Releases: huggingface/huggingface_hub
[v0.24.2] Fix create empty commit PR should not fail
See #2413 for more details.
Creating an empty commit on a PR was failing due to a revision
parameter been quoted twice. This patch release fixes it.
Full Changelog: v0.24.1...v0.24.2
[v0.24.1] Handle [DONE] signal from TGI + remove logic for "non-TGI servers"
This release fixes 2 things:
- handle
"[DONE]"
message in chat stream (related to TGI update huggingface/text-generation-inference#2221) - remove the "non-TGI" logic in chat completion since all models support server-side rendering now that even transformers-backed models are TGI-server.
See #2410 for more details.
Full Changelog: v0.24.0...v0.24.1
v0.24.0: Inference, serialization and optimizations
⚡️ OpenAI-compatible inference client!
The InferenceClient
's chat completion API is now fully compliant with OpenAI
client. This means it's a drop-in replacement in your script:
- from openai import OpenAI
+ from huggingface_hub import InferenceClient
- client = OpenAI(
+ client = InferenceClient(
base_url=...,
api_key=...,
)
output = client.chat.completions.create(
model="meta-llama/Meta-Llama-3-8B-Instruct",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Count to 10"},
],
stream=True,
max_tokens=1024,
)
for chunk in output:
print(chunk.choices[0].delta.content)
Why switching to InferenceClient
if you already use OpenAI
then? Because it's better integrated with HF services, such as the Serverless Inference API and Dedicated Endpoints. Check out the more detailed answer in this HF Post.
For more details about OpenAI compatibility, check out this guide's section.
- True OpenAI drop-in replacement by InferenceClient by @Wauplin in #2384
- Promote chat_completion in inference guide by @Wauplin in #2366
(other) InferenceClient
improvements
Some new parameters have been added to the InferenceClient
, following the latest changes in our Inference API:
prompt_name
,truncate
andnormalize
infeature_extraction
model_id
andresponse_format
, inchat_completion
adapter_id
intext_generation
hypothesis_template
andmulti_labels
inzero_shot_classification
Of course, all of those changes are also available in the AsyncInferenceClient
async equivalent 🤗
- Support truncate and normalize in InferenceClient by @Wauplin in #2270
- Add
prompt_name
to feature-extraction + update types by @Wauplin in #2363 - Send model_id in ChatCompletion request by @Wauplin in #2302
- improve client.zero_shot_classification() by @MoritzLaurer in #2340
- [InferenceClient] Add support for
adapter_id
(text-generation) andresponse_format
(chat-completion) by @Wauplin in #2383
Added helpers for TGI servers:
get_endpoint_info
to get information about an endpoint (running model, framework, etc.). Only available on TGI/TEI-powered models.health_check
to check health status of the server. Only available on TGI/TEI-powered models and only for InferenceEndpoint or local deployment. For serverless InferenceAPI, it's better to useget_model_status
.
Other fixes:
image_to_text
output type has been fixed- use
wait-for-model
to avoid been rate limited while model is not loaded - add
proxies
support
- Fix InferenceClient.image_to_text output value by @Wauplin in #2285
- Fix always None in text_generation output by @Wauplin in #2316
- Add wait-for-model header when sending request to Inference API by @Wauplin in #2318
- Add proxy support on async client by @noech373 in #2350
- Remove jinja tips + fix typo in chat completion docstring by @Wauplin in #2368
💾 Serialization
The serialization module introduced in v0.22.x
has been improved to become the preferred way to serialize a torch model to disk. It handles how of the box sharding and safe serialization (using safetensors
) with subtleties to work with shared layers. This logic was previously scattered in libraries like transformers
, diffusers
, accelerate
and safetensors
. The goal of centralizing it in huggingface_hub
is to allow any external library to safely benefit from the same naming convention, making it easier to manage for end users.
>>> from huggingface_hub import save_torch_model
>>> model = ... # A PyTorch model
# Save state dict to "path/to/folder". The model will be split into shards of 5GB each and saved as safetensors.
>>> save_torch_model(model, "path/to/folder")
# Or save the state dict manually
>>> from huggingface_hub import save_torch_state_dict
>>> save_torch_state_dict(model.state_dict(), "path/to/folder")
More details in the serialization package reference.
- Serialization: support saving torch state dict to disk by @Wauplin in #2314
- Handle shared layers in
save_torch_state_dict
+ addsave_torch_model
by @Wauplin in #2373
Some helpers related to serialization have been made public for reuse in external libraries:
get_torch_storage_id
get_torch_storage_size
- Support
max_shard_size
as string insplit_state_dict_into_shards_factory
by @SunMarc in #2286 - Make get_torch_storage_id public by @Wauplin in #2304
📁 HfFileSystem
The HfFileSystem
has been improved to optimize calls, especially when listing files from a repo. This is especially useful for large datasets like HuggingFaceFW/fineweb for faster processing and reducing risk of being rate limited.
- [HfFileSystem] Less /paths-info calls by @lhoestq in #2271
- Update token type definition and arg description in
hf_file_system.py
by @lappemic in #2278 - [HfFileSystem] Faster
fs.walk()
by @lhoestq in #2346
Thanks to @lappemic, HfFileSystem
methods are now properly documented. Check it out here!
✨ HfApi & CLI improvements
Commit API
A new mechanism has been introduced to prevent empty commits if no changes have been detected. Enabled by default in upload_file
, upload_folder
, create_commit
and the huggingface-cli upload
command. There is no way to force an empty commit.
Resource groups
Resource Groups allow organizations administrators to group related repositories together, and manage access to those repos. It is now possible to specify a resource group ID when creating a repo:
from huggingface_hub import create_repo
create_repo("my-secret-repo", private=True, resource_group_id="66670e5163145ca562cb1988")
Webhooks API
Webhooks allow you to listen for new changes on specific repos or to all repos belonging to particular set of users/organizations (not just your repos, but any repo). With the Webhooks API you can create, enable, disable, delete, update, and list webhooks from a script!
from huggingface_hub import create_webhook
# Example: Creating a webhook
webhook = create_webhook(
url="https://webhook.site/your-custom-url",
watched=[{"type": "user", "name": "your-username"}, {"type": "org", "name": "your-org-name"}],
domains=["repo", "discussion"],
secret="your-secret"
)
Search API
The search API has been slightly improved. It is now possible to:
- filter datasets by tags
- filter which attributes should be returned in
model_info
/list_models
(and similarly for datasets/Spaces). For example, you can ask the server to returndownloadsAllTime
for all models.
>>> from huggingface_hub import list_models
>>> for model in list_models(library="transformers", expand="downloadsAllTime", sort="downloads", limit=5):
... print(model.id, model.downloads_all_time)
MIT/ast-finetuned-audioset-10-10-0.4593 1676502301
sentence-transformers/all-MiniLM-L12-v2 115588145
sentence-transformers/all-MiniLM-L6-v2 250790748
google-bert/bert-base-uncased 1476913254
openai/clip-vit-large-patch14 590557280
- Support filtering datasets by tags by @Wauplin in #2266
- Support
expand
parameter inxxx_info
andlist_xxxs
(model/dataset/Space) by @Wauplin in #2333 - Add InferenceStatus to ExpandModelProperty_T by @Wauplin in #2388
- Do not mention gitalyUid in expand parameter by @Wauplin in #2395
CLI
It is now possible to delete files from a repo using the command line:
Delete a folder:
>>> huggingface-cli repo-files Wauplin/my-cool-model delete folder/
Files correctly deleted from repo. Commit: https://huggingface.co/Wauplin/my-cool-mo...
Use Unix-style wildcards to delete sets of files:
>>> huggingface-cli repo-files Wauplin/my-cool-model delete *.txt folder/*.bin
Files correctly deleted from repo. Commit: https://huggingface.co/Wauplin/my-cool-mo...
- fix/issue 2090 : Add a
repo_files
command, with recursive deletion. by @OlivierKessler01 in #2280
ModelHubMixin
The ModelHubMixin
, allowing for quick integration of external libraries with the Hub have been updated to fix some existing bugs and ease its use. Learn how to integrate your library from this guide.
- Don't override 'config' in model_kwargs by @alexander-soare in #2274
- Support custom kwargs for model card in save_pretrained by @qubvel in #2310
- ModelHubMixin: Fix attributes lost in inheritance by @Wauplin in #2305
- Fix ModelHubMixin coders by @gorold in #2291
- Hot-fix: do not share tags between
ModelHubMixin
siblings by @Wauplin in #2394 - Fix: correctly encode/decode config in ModelHubMixin if custom coders by @Wauplin in #2337
🌐 📚 Documentation
Efforts from the Korean-speaking community continued to translate guides and package references to KO! Check out the result here.
- 🌐 [i18n-KO] Trans...
[v0.23.5] Hot-fix: do not share tags between ModelHubMixin siblings
See #2394 for more details.
Full Changelog: v0.23.4...v0.23.5
[v0.23.4] Patch: fix encoders issues in ModelHubMixin
Includes:
- 🌐 [i18n-KO] Translated guides/integrations.md to Korean #2256 #2256
- Don't override 'config' in model_kwargs #2274
- Fix ModelHubMixin coders #2291
- ModelHubMixin: Fix attributes lost in inheritance #2305
- Support custom kwargs for model card in save_pretrained #2310
Full Changelog: v0.23.3...v0.23.4
[v0.23.3] Patch: fix details not returned in `InferenceClient.text_generation`
Release 0.23.0 introduced a breaking change in InferenceClient.text_generation
. When details=True
is passed, the details
attribute in the output is always None. The patch release fixes this. See #2316 for more details.
Full Changelog: v0.23.2...v0.23.3
[v0.23.2] Patch: Support `max_shard_size` as string in `split_state_dict_into_shards_factory`
split_state_dict_into_shards_factory
now accepts string values as max_shard_size
(ex: "5MB"
), in addition to integer values. Related PR: #2286.
Full Changelog: v0.23.1...v0.23.2
v0.23.1 hot-fix: optimize HTTP calls in `HfFileSystem`
See #2271 for more details.
Full Changelog: v0.23.0...v0.23.1
v0.23.0: LLMs with tools, seamless downloads, and much more!
📁 Seamless download to local dir
The 0.23.0
release comes with a big revamp of the download process, especially when it comes to downloading to a local directory. Previously the process was still involving the cache directory and symlinks which led to misconceptions and a suboptimal user experience. The new workflow involves a .cache/huggingface/
folder, similar to the .git/
one, that keeps track of the progress of a download. The main features are:
- no symlinks
- no local copy
- don't re-download when not necessary
- same behavior on both Unix and Windows
- unrelated to cache-system
Example to download q4 GGUF file for microsoft/Phi-3-mini-4k-instruct-gguf:
# Download q4 GGUF file from
huggingface-cli download microsoft/Phi-3-mini-4k-instruct-gguf Phi-3-mini-4k-instruct-q4.gguf --local-dir=data/phi3
With this addition, interrupted downloads are now resumable! This applies both for downloads in local and cache directories which should greatly improve UX for users with slow/unreliable connections. In this regard, the resume_download
parameter is now deprecated (not relevant anymore).
- Revamp download to local dir process by @Wauplin in #2223
- Rename
.huggingface/
folder to.cache/huggingface/
by @Wauplin in #2262
💡 Grammar and Tools in InferenceClient
It is now possible to provide a list of tools when chatting with a model using the InferenceClient
! This major improvement has been made possible thanks to TGI that handle them natively.
>>> from huggingface_hub import InferenceClient
# Ask for weather in the next days using tools
>>> client = InferenceClient("meta-llama/Meta-Llama-3-70B-Instruct")
>>> messages = [
... {"role": "system", "content": "Don't make assumptions about what values to plug into functions. Ask for clarification if a user request is ambiguous."},
... {"role": "user", "content": "What's the weather like the next 3 days in San Francisco, CA?"},
... ]
>>> tools = [
... {
... "type": "function",
... "function": {
... "name": "get_current_weather",
... "description": "Get the current weather",
... "parameters": {
... "type": "object",
... "properties": {
... "location": {
... "type": "string",
... "description": "The city and state, e.g. San Francisco, CA",
... },
... "format": {
... "type": "string",
... "enum": ["celsius", "fahrenheit"],
... "description": "The temperature unit to use. Infer this from the users location.",
... },
... },
... "required": ["location", "format"],
... },
... },
... },
... ...
... ]
>>> response = client.chat_completion(
... model="meta-llama/Meta-Llama-3-70B-Instruct",
... messages=messages,
... tools=tools,
... tool_choice="auto",
... max_tokens=500,
... )
>>> response.choices[0].message.tool_calls[0].function
ChatCompletionOutputFunctionDefinition(
arguments={
'location': 'San Francisco, CA',
'format': 'fahrenheit',
'num_days': 3
},
name='get_n_day_weather_forecast',
description=None
)
It is also possible to provide grammar rules to the text_generation
task. This ensures that the output follows a precise JSON Schema specification or matches a regular expression. For more details about it, check out the Guidance guide from Text-Generation-Inference docs.
⚙️ Other
Mention more chat-completion
task instead of conversation
in documentation.
chat-completion
relies on server-side rendering in all cases, including when model is transformers
-backed. Previously it was only the case for TGI-backed models and templates were rendered client-side otherwise.
Improved logic to determine whether a model is served via TGI or transformers
.
🌐 📚 Korean community is on fire!
The PseudoLab team is a non-profit dedicated to make AI more accessible in the Korean-speaking community. In the past few weeks, their team of contributors managed to translated (almost) entirely the huggingface_hub
documentation. Huge shout-out to the coordination on this task! Documentation can be accessed here.
- 🌐 [i18n-KO] Translated
guides/webhooks_server.md
to Korean by @nuatmochoi in #2145 - 🌐 [i18n-KO] Translated
reference/login.md
to Korean by @SeungAhSon in #2151 - 🌐 [i18n-KO] Translated
package_reference/tensorboard.md
to Korean by @fabxoe in #2173 - 🌐 [i18n-KO] Translated
package_reference/inference_client.md
to Korean by @cjfghk5697 in #2178 - 🌐 [i18n-KO] Translated
reference/inference_endpoints.md
to Korean by @harheem in #2180 - 🌐 [i18n-KO] Translated
package_reference/file_download.md
to Korean by @seoyoung-3060 in #2184 - 🌐 [i18n-KO] Translated
package_reference/cache.md
to Korean by @nuatmochoi in #2191 - 🌐 [i18n-KO] Translated
package_reference/collections.md
to Korean by @boyunJang in #2214 - 🌐 [i18n-KO] Translated
package_reference/inference_types.md
to Korean by @fabxoe in #2171 - 🌐 [i18n-KO] Translated
guides/upload.md
to Korean by @junejae in #2139 - 🌐 [i18n-KO] Translated
reference/repository.md
to Korean by @junejae in #2189 - 🌐 [i18n-KO] Translated
package_reference/space_runtime.md
to Korean by @boyunJang in #2213 - 🌐 [i18n-KO] Translated
guides/repository.md
to Korean by @cjfghk5697 in #2124 - 🌐 [i18n-KO] Translated
guides/model_cards.md
to Korean" by @SeungAhSon in #2128 - 🌐 [i18n-KO] Translated
guides/community.md
to Korean by @seoulsky-field in #2126 - 🌐 [i18n-KO] Translated
guides/cli.md
to Korean by @harheem in #2131 - 🌐 [i18n-KO] Translated
guides/search.md
to Korean by @seoyoung-3060 in #2134 - 🌐 [i18n-KO] Translated
guides/inference.md
to Korean by @boyunJang in #2130 - 🌐 [i18n-KO] Translated
guides/manage-spaces.md
to Korean by @boyunJang in #2220 - 🌐 [i18n-KO] Translating
guides/hf_file_system.md
to Korean by @heuristicwave in #2146 - 🌐 [i18n-KO] Translated
package_reference/hf_api.md
to Korean by @fabxoe in #2165 - 🌐 [i18n-KO] Translated
package_reference/mixins.md
to Korean by @fabxoe in #2166 - 🌐 [i18n-KO] Translated
guides/inference_endpoints.md
to Korean by @usr-bin-ksh in #2164 - 🌐 [i18n-KO] Translated
package_reference/utilities.md
to Korean by @cjfghk5697 in #2196 - fix ko docs by @Wauplin (direct commit on main)
- 🌐 [i18n-KO] Translated package_reference/serialization.md to Korean by @seoyoung-3060 in #2233
- 🌐 [i18n-KO] Translated package_reference/hf_file_system.md to Korean by @SeungAhSon in #2174
🛠️ Misc improvements
User API
@bilgehanertan added support for 2 new routes:
get_user_overview
to retrieve high-level information about a user: username, avatar, number of models/datasets/Spaces, number of likes and upvotes, number of interactions in discussion, etc.
- User API endpoints by @bilgehanertan in #2147
CLI tag
@bilgehanertan added a new command to the CLI to handle tags. It is now possible to:
- tag a repo
>>> huggingface-cli tag Wauplin/my-cool-model v1.0
You are about to create tag v1.0 on model Wauplin/my-cool-model
Tag v1.0 created on Wauplin/my-cool-model
- retrieve the list of tags for a repo
>>> huggingface-cli tag Wauplin/gradio-space-ci -l --repo-type space
Tags for space Wauplin/gradio-space-ci:
0.2.2
0.2.1
0.2.0
0.1.2
0.0.2
0.0.1
- delete a tag on a repo
>>> huggingface-cli tag -d Wauplin/my-cool-model v1.0
You are about to delete tag v1.0 on model Wauplin/my-cool-model
Proceed? [Y/n] y
Tag v1.0 deleted on Wauplin/my-cool-model
For more details, check out the CLI guide.
- CLI Tag Functionality by @bilgehanertan in #2172
🧩 ModelHubMixin
This ModelHubMixin
got a set of nice improvement to generate model cards and handle custom data types in the config.json
file. More info in the integration guide.
ModelHubMixin
: more metadata + arbitrary config types + proper guide by @Wauplin in #2230- Fix ModelHubMixin when class is a dataclass by @Wauplin in #2159
- Do not document private attributes of ModelHubMixin by @Wauplin in #2216
- Add support for pipeline_tag in ModelHubMixin by @Wauplin in #2228
⚙️ Other
In a shared environment, it is now possible to set a custom path HF_TOKEN_PATH
as environment variable so that each user of the cluster has their own access token.
Thanks to @Y4suyuki and @lappemic, most custom errors defined in huggingface_hub
are now aggregated in the same module. This makes it very easy...
[v0.22.2] Hot-fix: correctly handle proxies
Full Changelog: v0.22.1...v0.22.2