feat: add the model processor proposal #257

chlins · 2025-01-15T09:36:03Z

vivek

Thanks @chlins for authoring this proposal. Overall it looks good. The key aspects of the proposal make it appealing. My takeaways:

This proposal brings AI models as first class citizen in Harbor. The user stories to use this CNAI model spec which follows OCI spec to surface AI model
Uses existing OCI spec (artifactType and mediaType ) along with following CNAI model format https://github.com/CloudNativeAI/model-spec/blob/main/docs/spec.md
No additional APIs needed as it leverages additions API - This is great

I left a comment regarding future model format changes and how new additions can be handled in generic way without requiring code changes. PTAL, I might be off as new to Harbor at code level, as long as we can handle such use case, its all looking good.

proposals/new/AI-model-processor.md

steven-zou · 2025-02-17T11:46:16Z

proposals/new/AI-model-processor.md

+
+## Architecture
+
+There will be a new AI model processor added to the artifact processors in Harbor core.


It's better to give details about how the new processor processes the data following the model spec into the Harbor artifact object which supports the related APIs shown in the API sections.
|---->[CONFIG]-
[model spec] ----> (processor)----> [artifact object] ------>（API)
|---->[LAYERS]-

proposals/new/AI-model-processor.md

s3rj1k · 2025-02-20T14:59:31Z

Just for reference, I am using below Modelfile (https://huggingface.co/google/gemma-2-2b), Modelfile is inside artifact directory (gemma-2-2b/Modelfile)

# https://huggingface.co/google/gemma-2-2b/tree/main

NAME gemma-2-2b
ARCH transformer
FAMILY gemma
FORMAT safetensors
PARAMSIZE 2
PRECISION bf16
QUANTIZATION none

CONFIG config.json
CONFIG generation_config.json
CONFIG special_tokens_map.json
CONFIG tokenizer_config.json
CONFIG tokenizer.json
CONFIG model.safetensors.index.json

MODEL tokenizer.model
MODEL *.safetensors

Below how oras sees above model after it is pushed into registry

# oras manifest fetch domain.net/library/gemma-2-2b:latest | jq
{
  "schemaVersion": 2,
  "mediaType": "application/vnd.oci.image.manifest.v1+json",
  "artifactType": "application/vnd.cnai.model.manifest.v1+json",
  "config": {
    "mediaType": "application/vnd.cnai.model.config.v1+json",
    "digest": "sha256:c9f3f1a5131634d8a8c59f2085bf6745a6b2f1f13bd80019d5635f74b57e2372",
    "size": 264
  },
  "layers": [
    {
      "mediaType": "application/vnd.cnai.model.doc.v1.tar",
      "digest": "sha256:f2bd54bd4695332a5c9fffb5bd736aa3a47a933c03bd4fa1fca508c9f607a1a1",
      "size": 27648,
      "annotations": {
        "org.cnai.model.filepath": "README.md"
      }
    },
    {
      "mediaType": "application/vnd.cnai.model.weight.config.v1.tar",
      "digest": "sha256:0ad46a0e2c2bfb03aeacef30f5cfdf08fc57e1e2109902594a955d1501d6f998",
      "size": 2048,
      "annotations": {
        "org.cnai.model.filepath": "generation_config.json"
      }
    },
    {
      "mediaType": "application/vnd.cnai.model.weight.config.v1.tar",
      "digest": "sha256:4c09b0c73a63882fbbd689a085ec52e83f42f97a2c07ffaaeecdab14ca922aaf",
      "size": 2560,
      "annotations": {
        "org.cnai.model.filepath": "special_tokens_map.json"
      }
    },
    {
      "mediaType": "application/vnd.cnai.model.weight.config.v1.tar",
      "digest": "sha256:821088b4b19de1ee9564acc8983345a68150839da75755f50fbb78c8013ad0ea",
      "size": 48128,
      "annotations": {
        "org.cnai.model.filepath": "tokenizer_config.json"
      }
    },
    {
      "mediaType": "application/vnd.cnai.model.weight.config.v1.tar",
      "digest": "sha256:3e86b2521cd1e5516a4c7aa09efeb94af5cfb6d842bcb6f4b84ff798ede8a4ea",
      "size": 17527296,
      "annotations": {
        "org.cnai.model.filepath": "tokenizer.json"
      }
    },
    {
      "mediaType": "application/vnd.cnai.model.weight.config.v1.tar",
      "digest": "sha256:d85985abcd1e05389abb9088251ca703c3307f946aa693d549f0967558f709f4",
      "size": 26112,
      "annotations": {
        "org.cnai.model.filepath": "model.safetensors.index.json"
      }
    },
    {
      "mediaType": "application/vnd.cnai.model.weight.config.v1.tar",
      "digest": "sha256:dbc9cf744f8ff3e596924765d8a91c7bed4b7939c73b18a373905b1b31d81b85",
      "size": 2560,
      "annotations": {
        "org.cnai.model.filepath": "config.json"
      }
    },
    {
      "mediaType": "application/vnd.cnai.model.weight.v1.tar",
      "digest": "sha256:746911d5850fdd7bc058de4f294c02d282f4e99422df18d31dfb83beb7aa8eed",
      "size": 4242944,
      "annotations": {
        "org.cnai.model.filepath": "tokenizer.model"
      }
    },
    {
      "mediaType": "application/vnd.cnai.model.weight.v1.tar",
      "digest": "sha256:2d1be53f694599fe05d2f57ba6b9028e766f8a585225d87643ad984c351dd7b3",
      "size": 4992578048,
      "annotations": {
        "org.cnai.model.filepath": "model-00001-of-00003.safetensors"
      }
    },
    {
      "mediaType": "application/vnd.cnai.model.weight.v1.tar",
      "digest": "sha256:5fa6ff435737b221e75b771450c9c09b0a3695c73815776450adab71b24ab4a1",
      "size": 4983444992,
      "annotations": {
        "org.cnai.model.filepath": "model-00002-of-00003.safetensors"
      }
    },
    {
      "mediaType": "application/vnd.cnai.model.weight.v1.tar",
      "digest": "sha256:703f96fc5ba4fee31dd14b537da139da015bb4d3351a212209050c2e68acfc75",
      "size": 481383424,
      "annotations": {
        "org.cnai.model.filepath": "model-00003-of-00003.safetensors"
      }
    }
  ]
}

as opposed to Kitfile from KitOps

# oras manifest fetch domain.net/library/gemma-2-2b:latest | jq
{
  "schemaVersion": 2,
  "config": {
    "mediaType": "application/vnd.kitops.modelkit.config.v1+json",
    "digest": "sha256:4b58400f17b967152ab8ac022946896cd55575f6076e2f79781f1e6e21ffbd0a",
    "size": 2170
  },
  "layers": [
    {
      "mediaType": "application/vnd.kitops.modelkit.model.v1.tar",
      "digest": "sha256:f3b5a4b1c6065dbdeac2afa977d60b110bb84a10455f6e0d6c94c33b7a97cdeb",
      "size": 4992578048
    },
    {
      "mediaType": "application/vnd.kitops.modelkit.modelpart.v1.tar",
      "digest": "sha256:b22bcdad944d6588dd92314753482cc2c2265e5231425e2846529a51a9b2393a",
      "size": 4983444992
    },
    {
      "mediaType": "application/vnd.kitops.modelkit.modelpart.v1.tar",
      "digest": "sha256:d8554286602a841a31bd9a88d3fa2ec85d6aa8b1eb9b585efb10681d22beae35",
      "size": 481383424
    },
    {
      "mediaType": "application/vnd.kitops.modelkit.modelpart.v1.tar",
      "digest": "sha256:d41f170f157a76a8e6fe223726bace7d1b3eb865532488297d96bb5a37adfdcc",
      "size": 2560
    },
    {
      "mediaType": "application/vnd.kitops.modelkit.modelpart.v1.tar",
      "digest": "sha256:d77ba08b9e8b7c7cd4dfe49f0df71f17d7f9173280ea23b3e8ad35e4a2eef1bd",
      "size": 2048
    },
    {
      "mediaType": "application/vnd.kitops.modelkit.modelpart.v1.tar",
      "digest": "sha256:fa1f3ea472adaaf49d62295483bc7423cccdc62e67064b0acc3a2103866a5057",
      "size": 26112
    },
    {
      "mediaType": "application/vnd.kitops.modelkit.modelpart.v1.tar",
      "digest": "sha256:2428f162d376dd61b077463d15ba0a04300697ba888686515d93afdd6fdeb403",
      "size": 2560
    },
    {
      "mediaType": "application/vnd.kitops.modelkit.modelpart.v1.tar",
      "digest": "sha256:5d6bc03de874683603f9bae07c7c79195cad9821e3ed5484361ec59c7191e5ae",
      "size": 17527296
    },
    {
      "mediaType": "application/vnd.kitops.modelkit.modelpart.v1.tar",
      "digest": "sha256:d922ba9526e31d91f7d757c4634ab5d861ed433c19a9906faf7715825a025713",
      "size": 48128
    },
    {
      "mediaType": "application/vnd.kitops.modelkit.code.v1.tar",
      "digest": "sha256:79495fe9531fb5f2eb8181a64ffe6659613f10254d1ad7bc5ced38f098550453",
      "size": 4245504
    }
  ],
  "annotations": {
    "ml.kitops.modelkit.cli-version": "1.1.0"
  }
}

Signed-off-by: chlins <[email protected]>

reasonerjt

lgtm

vivek

lgtm

proposals/new/AI-model-processor.md

s3rj1k · 2025-02-24T09:46:10Z

We also should consider changing model name, reference here https://github.com/goharbor/harbor/pull/21663/files#r1967296025

"MODEL" -> "CNAI" for ArtifactTypeCNAI

Harbor can have multiple ML model processors for different specs, having this "MODEL" is so general, it will look strange when other models specs processors will be added.

chlins · 2025-03-05T12:30:00Z

We also should consider changing model name, reference here https://github.com/goharbor/harbor/pull/21663/files#r1967296025

"MODEL" -> "CNAI" for ArtifactTypeCNAI

Harbor can have multiple ML model processors for different specs, having this "MODEL" is so general, it will look strange when other models specs processors will be added.

@s3rj1k Sure, we can change this name to "CNAI" for future extension.

bupd

lgtm

wy65701436

lgtm

chlins requested review from a team as code owners January 15, 2025 09:36

github-actions bot assigned MinerYang, stonezdj and wy65701436 Jan 15, 2025

vivek suggested changes Jan 24, 2025

View reviewed changes

proposals/new/AI-model-processor.md Show resolved Hide resolved

proposals/new/AI-model-processor.md Show resolved Hide resolved

reasonerjt reviewed Jan 26, 2025

View reviewed changes

chlins force-pushed the feat/ai-model-processor branch 2 times, most recently from fc4e42c to 6c82d47 Compare February 17, 2025 08:58

steven-zou reviewed Feb 17, 2025

View reviewed changes

feat: add the model processor proposal

11106e9

Signed-off-by: chlins <[email protected]>

chlins force-pushed the feat/ai-model-processor branch from 6c82d47 to 11106e9 Compare February 21, 2025 02:54

reasonerjt approved these changes Feb 21, 2025

View reviewed changes

s3rj1k mentioned this pull request Feb 21, 2025

Add CNAI processor goharbor/harbor#21659

Closed

vivek approved these changes Feb 21, 2025

View reviewed changes

s3rj1k reviewed Feb 22, 2025

View reviewed changes

proposals/new/AI-model-processor.md Show resolved Hide resolved

stonezdj approved these changes Mar 13, 2025

View reviewed changes

bupd approved these changes Mar 14, 2025

View reviewed changes

wy65701436 approved these changes Mar 20, 2025

View reviewed changes

Merge branch 'main' into feat/ai-model-processor

be1d932

chlins merged commit a4208b1 into goharbor:main Mar 20, 2025
1 of 3 checks passed

chlins mentioned this pull request Sep 25, 2025

ModelPack Integration Phase 2 goharbor/harbor#22386

Open


		## Architecture

		There will be a new AI model processor added to the artifact processors in Harbor core.

feat: add the model processor proposal #257

feat: add the model processor proposal #257

Uh oh!

Conversation

chlins commented Jan 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vivek left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

steven-zou Feb 17, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

s3rj1k commented Feb 20, 2025

Uh oh!

reasonerjt left a comment

Choose a reason for hiding this comment

Uh oh!

vivek left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

s3rj1k commented Feb 24, 2025

Uh oh!

chlins commented Mar 5, 2025

Uh oh!

bupd left a comment

Choose a reason for hiding this comment

Uh oh!

wy65701436 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

chlins commented Jan 15, 2025 •

edited

Loading

vivek left a comment •

edited

Loading