Skip to content

feat: add media type for unknown file type #53

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
aftersnow wants to merge 1 commit into from
Closed

feat: add media type for unknown file type #53

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions docs/spec.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,14 @@ The image manifest of model artifacts follows the [OCI Image Manifest Specificat

- `application/vnd.cnai.model.dataset.v1.tar+zstd`: The layer is a [tar archive][tar-archive] that includes datasets that may be needed for the lifecycle of AI/ML models. The archive is compressed with [zstd][rfc8478].

- `application/vnd.cnai.model.unknown.v1.raw`: The layer is an unarchived, uncompressed unknown model file type. If the files are large, implementations are RECOMMENDED to use this media type. If the type of model file cannot be determined, this media type can be used.

- `application/vnd.cnai.model.unknown.v1.tar`: The layer is a [tar archive][tar-archive] that includes unknown model file types. If the type of model file cannot be determined, this media type can be used.

- `application/vnd.cnai.model.unknown.v1.tar+gzip`: The layer is a [tar archive][tar-archive] that includes unknown model file types. The archive is compressed with [gzip][rfc1952_2]. If the type of model file cannot be determined, this media type can be used.

- `application/vnd.cnai.model.unknown.v1.tar+zstd`: The layer is a [tar archive][tar-archive] that includes unknown model file types. The archive is compressed with [zstd][rfc8478]. If the type of model file cannot be determined, this media type can be used.

- **`annotations`** _string-string map_

This OPTIONAL property contains arbitrary attributes for the layer. For metadata specific to models, implementations SHOULD use the predefined annotation keys as outlined in the [Layer Annotation Keys](./annotations.md#layer-annotation-keys).
Expand Down Expand Up @@ -137,6 +145,12 @@ The `application/vnd.cnai.model.weight.v1.tar+gzip` represents an `application/v

The `application/vnd.cnai.model.weight.v1.tar+zstd` represents an `application/vnd.cnai.model.weight.v1.tar` payload which has been compressed with the [zstd][rfc8478] algorithm. The mediaTypes `application/vnd.cnai.model.weight.config.v1.tar+zstd`, `application/vnd.cnai.model.doc.v1.tar+zstd`, `application/vnd.cnai.model.code.v1.tar+zstd`, `application/vnd.cnai.model.dataset.v1.tar+zstd` refer to the zstd compressed payloads of their corresponding type.

### `unknown` Media Types

The `application/vnd.cnai.model.unknown.v1.tar` media type and its friends represent an file type which is unknown to the implementation. In some cases, the model package system may not know the type of the file, or the file is not config, weight, doc, code or dataset. For instance, the file is a binary file, or the file is a model file but the type is not known, but it is still a valid model file, which is required by the downstream model serving/deployment platform.

The `application/vnd.cnai.model.unknown.v1.tar+gzip` and `application/vnd.cnai.model.unknown.v1.tar+zstd` media types represent the gzip and zstd compressed payloads of the `application/vnd.cnai.model.unknown.v1.tar` media type. If the file is large, implementations are RECOMMENDED to use `application/vnd.cnai.model.unknown.v1.raw` media type.

### File Attributes

Where supported, MUST include file attributes
Expand Down
12 changes: 12 additions & 0 deletions specs-go/v1/mediatype.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,4 +84,16 @@ const (

// MediaTypeModelDatasetZstd specifies the media type for zstd compressed model datasets, including datasets that may be needed throughout the lifecycle of AI/ML models.
MediaTypeModelDatasetZstd = "application/vnd.cnai.model.dataset.v1.tar+zstd"

// MediaTypeModelUnknownRaw specifies the media type for an unarchived, uncompressed unknown file type.
MediaTypeModelUnknownRaw = "application/vnd.cnai.model.unknown.v1.raw"

// MediaTypeModelUnknown specifies the media type for an unknown file type.
MediaTypeModelUnknown = "application/vnd.cnai.model.unknown.v1.tar"

// MediaTypeModelUnknownGzip specifies the media type for a gzipped unknown file type.
MediaTypeModelUnknownGzip = "application/vnd.cnai.model.unknown.v1.tar+gzip"

// MediaTypeModelUnknownZstd specifies the media type for a zstd compressed unknown file type.
MediaTypeModelUnknownZstd = "application/vnd.cnai.model.unknown.v1.tar+zstd"
)