Support env HF_ENDPOINT?

[yufeng97]

Feature request

Is it possible to support HuggingFace mirror website? Such as env HF_ENDPOINT . Like huggingface_hub library, it has a environment variable HF_ENDPOINT which can use huggingface mirror website to download models.

export HF_ENDPOINT=https://hf-mirror.com
huggingface-cli download --resume-download gpt2 --local-dir gpt2

https://hf-mirror.com/

Motivation

It's difficult to download model in China. Use HF mirror to accelerate model download.

Your contribution

Moral support

[nbroad1881]

You can download the model locally and then use TEI to load the local model.

model=/path/to/model/weights
volume=/path/to/model

docker run --gpus all  -p 8080:80 -v $volume:/data --pull always ghcr.io/huggingface/text-embeddings-inference:1.5 --model-id /data/weights

[r0kk]

Hi @nbroad1881 could you give example for BAAI--bge-reranker-large I could not make it work. Also it still tries to download model.safetensors which is still blocked. I work for crop and all those things are blocked, so it would be greatly appreciated🙏.

[nbroad1881]

@r0kk ,

If it is blocked, then there is nothing that can be done until it gets unblocked.

If you have a local copy, copy the code I have above.

You can download the model locally and then use TEI to load the local model.

model=/path/to/model/weights
volume=/path/to/model

docker run --gpus all -p 8080:80 -v $volume:/data --pull always ghcr.io/huggingface/text-embeddings-inference:1.6 --model-id /data/weights