Failed to load ColpaliEmbeddingModel due to missing safetensors files.

[reviver134]

(venv) jayant@reviver:/mnt/c/Users/jayan/OneDrive/Desktop/morphik/morphik-core$ HF_HUB_DOWNLOAD_TIMEOUT=60000 uv run start_server.py
warning: VIRTUAL_ENV=venv does not match the project environment path .venv and will be ignored; use --active to target the active environment instead
2025-08-15 04:03:22 - root - INFO - Starting existing Redis container (morphik-redis)...
2025-08-15 04:03:22 - root - INFO - Redis container started.
Ollama is running and will be used for: embedding, completion, rules, graph, parser.vision
2025-08-15 04:03:22 - root - INFO - Waiting for Redis to be available at localhost:6379...
2025-08-15 04:03:22 - root - INFO - Redis is accepting connections.
2025-08-15 04:03:22 - root - INFO - Starting ARQ worker...
2025-08-15 04:03:22 - root - INFO - ARQ worker started with PID: 1027
2025-08-15 04:03:22 - root - INFO - Worker logs available at: /mnt/c/Users/jayan/OneDrive/Desktop/morphik/morphik-core/logs/worker.log
2025-08-15 04:03:22 - root - INFO - Starting Uvicorn server...
2025-08-15 04:03:34 - httpx - INFO - HTTP Request: GET https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json "HTTP/1.1 200 OK"
/mnt/c/Users/jayan/OneDrive/Desktop/morphik/morphik-core/.venv/lib/python3.11/site-packages/pydantic/_internal/_fields.py:198: UserWarning: Field name "schema" in "CompletionRequest" shadows an attribute in parent "BaseModel"
warnings.warn(
/mnt/c/Users/jayan/OneDrive/Desktop/morphik/morphik-core/.venv/lib/python3.11/site-packages/pydantic/_internal/_fields.py:198: UserWarning: Field name "schema" in "MetadataExtractionRule" shadows an attribute in parent "BaseRule"
warnings.warn(
/mnt/c/Users/jayan/OneDrive/Desktop/morphik/morphik-core/.venv/lib/python3.11/site-packages/pydantic/_internal/_fields.py:198: UserWarning: Field name "schema" in "CompletionQueryRequest" shadows an attribute in parent "RetrieveRequest"
warnings.warn(
/mnt/c/Users/jayan/OneDrive/Desktop/morphik/morphik-core/.venv/lib/python3.11/site-packages/pydantic/_internal/_fields.py:198: UserWarning: Field name "schema" in "MetadataExtractionRuleRequest" shadows an attribute in parent "BaseModel"
warnings.warn(
2025-08-15 04:05:23 - core.database.postgres_database - INFO - Initializing PostgreSQL connection pool with size=10, max_overflow=15, pool_recycle=3600s
2025-08-15 04:05:24 - core.vector_store.pgvector_store - INFO - Initializing vector store database engine with pool size=10, max_overflow=15, pool_recycle=3600s
2025-08-15 04:05:24 - core.vector_store.pgvector_store - INFO - Created vector store database engine successfully
2025-08-15 04:05:24 - core.embedding.litellm_embedding - INFO - Initialized LiteLLM embedding model with model_key=ollama_embedding, config={'model_name': 'ollama/nomic-embed-text', 'api_base': 'http://localhost:11434'}
2025-08-15 04:05:24 - core.services_init - INFO - Initialized LiteLLM embedding model with model key: ollama_embedding
2025-08-15 04:05:24 - core.completion.litellm_completion - INFO - Initialized LiteLLM completion model with model_key=ollama_qwen_vision, config={'model_name': 'ollama_chat/qwen2.5vl:latest', 'api_base': 'http://localhost:11434', 'vision': True}, is_ollama_direct=True
2025-08-15 04:05:24 - core.services_init - INFO - Initialized LiteLLM completion model with model key: ollama_qwen_vision
2025-08-15 04:05:24 - core.services_init - INFO - Initializing ColPali in local mode
2025-08-15 04:05:24 - core.embedding.colpali_embedding_model - INFO - Initializing ColpaliEmbeddingModel with device: cuda
Traceback (most recent call last):
File "/mnt/c/Users/jayan/OneDrive/Desktop/morphik/morphik-core/start_server.py", line 331, in
main()
File "/mnt/c/Users/jayan/OneDrive/Desktop/morphik/morphik-core/start_server.py", line 319, in main
uvicorn.run(
File "/mnt/c/Users/jayan/OneDrive/Desktop/morphik/morphik-core/.venv/lib/python3.11/site-packages/uvicorn/main.py", line 580, in run
server.run()
File "/mnt/c/Users/jayan/OneDrive/Desktop/morphik/morphik-core/.venv/lib/python3.11/site-packages/uvicorn/server.py", line 67, in run
return asyncio.run(self.serve(sockets=sockets))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/jayant/.local/share/uv/python/cpython-3.11.13-linux-x86_64-gnu/lib/python3.11/asyncio/runners.py", line 190, in run
return runner.run(main)
^^^^^^^^^^^^^^^^
File "/home/jayant/.local/share/uv/python/cpython-3.11.13-linux-x86_64-gnu/lib/python3.11/asyncio/runners.py", line 118, in run
return self._loop.run_until_complete(task)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/jayant/.local/share/uv/python/cpython-3.11.13-linux-x86_64-gnu/lib/python3.11/asyncio/base_events.py", line 654, in run_until_complete
return future.result()
^^^^^^^^^^^^^^^
File "/mnt/c/Users/jayan/OneDrive/Desktop/morphik/morphik-core/.venv/lib/python3.11/site-packages/uvicorn/server.py", line 71, in serve
await self._serve(sockets)
File "/mnt/c/Users/jayan/OneDrive/Desktop/morphik/morphik-core/.venv/lib/python3.11/site-packages/uvicorn/server.py", line 78, in _serve
config.load()
File "/mnt/c/Users/jayan/OneDrive/Desktop/morphik/morphik-core/.venv/lib/python3.11/site-packages/uvicorn/config.py", line 436, in load
self.loaded_app = import_from_string(self.app)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/c/Users/jayan/OneDrive/Desktop/morphik/morphik-core/.venv/lib/python3.11/site-packages/uvicorn/importer.py", line 19, in import_from_string
module = importlib.import_module(module_str)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/jayant/.local/share/uv/python/cpython-3.11.13-linux-x86_64-gnu/lib/python3.11/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "", line 1204, in _gcd_import
File "", line 1176, in _find_and_load
File "", line 1147, in _find_and_load_unlocked
File "", line 690, in _load_unlocked
File "", line 940, in exec_module
File "", line 241, in _call_with_frames_removed
File "/mnt/c/Users/jayan/OneDrive/Desktop/morphik/morphik-core/core/api.py", line 34, in
from core.routes.cache import router as cache_router
File "/mnt/c/Users/jayan/OneDrive/Desktop/morphik/morphik-core/core/routes/cache.py", line 12, in
from core.services_init import document_service
File "/mnt/c/Users/jayan/OneDrive/Desktop/morphik/morphik-core/core/services_init.py", line 143, in
colpali_embedding_model = ColpaliEmbeddingModel()
^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/c/Users/jayan/OneDrive/Desktop/morphik/morphik-core/core/embedding/colpali_embedding_model.py", line 25, in init
self.model = ColQwen2_5.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/c/Users/jayan/OneDrive/Desktop/morphik/morphik-core/.venv/lib/python3.11/site-packages/transformers/modeling_utils.py", line 279, in _wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/mnt/c/Users/jayan/OneDrive/Desktop/morphik/morphik-core/.venv/lib/python3.11/site-packages/transformers/modeling_utils.py", line 4260, in from_pretrained
checkpoint_files, sharded_metadata = _get_resolved_checkpoint_files(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/c/Users/jayan/OneDrive/Desktop/morphik/morphik-core/.venv/lib/python3.11/site-packages/transformers/modeling_utils.py", line 1152, in _get_resolved_checkpoint_files
checkpoint_files, sharded_metadata = get_checkpoint_shard_files(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/c/Users/jayan/OneDrive/Desktop/morphik/morphik-core/.venv/lib/python3.11/site-packages/transformers/utils/hub.py", line 1115, in get_checkpoint_shard_files
cached_filenames = cached_files(
^^^^^^^^^^^^^
File "/mnt/c/Users/jayan/OneDrive/Desktop/morphik/morphik-core/.venv/lib/python3.11/site-packages/transformers/utils/hub.py", line 517, in cached_files
raise EnvironmentError(
OSError: tsystems/colqwen2.5-3b-base does not appear to have files named ('model-00001-of-00002.safetensors', 'model-00002-of-00002.safetensors'). Checkout 'https://huggingface.co/tsystems/colqwen2.5-3b-base/tree/main'for available files.
2025-08-15 04:07:11 - root - INFO - Stopping ARQ worker (PID: 1027)...
2025-08-15 04:07:16 - root - WARNING - ARQ worker did not terminate gracefully, sending SIGKILL.
2025-08-15 04:07:16 - root - INFO - ARQ worker killed.

[onestardao]

Here’s a compact comment you can post on that issue:

Looks like a model-asset mismatch rather than core code.
Problem Map: Model intake pipeline → “expected safetensors, cached/selected variant has .bin / partial repo”.
Quick checks: pip show safetensors (install/upgrade if missing) and clear the HF cache for this repo (huggingface-cli delete-cache --pattern <repo-id>), then re-pull.
Pin the exact repo and revision that actually contains *.safetensors (many ColPali forks have only .bin on main, safetensors on a tag/branch).
If you must stay on that repo, point the loader to the correct subfolder or pass a revision with safetensors.
Avoid synced paths (e.g., OneDrive) to rule out file locks; try a local path.
If helpful, I keep a small semantic firewall checklist for these intake issues (no infra changes). Ping me and I’ll share the link.