Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow for Private HF_Lora URLs with Query Params for Auth #1

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

Averylamp
Copy link

@Averylamp Averylamp commented Sep 27, 2024

Please let me know if you have any PR requests or things you'd like to be different.

This change allows a user to pass a custom hf_access_token with the hf_lora url in order to load a custom lora from a private repo.

It follows the existing url path match ^https?://huggingface.co and extracts query parameters to pass directly to pipe.load_lora_weights This allows

https://huggingface.co/<HF_SLUG>/<WEIGHTS_NAME>
https://huggingface.co/<HF_SLUG>?token=<INSERT_HF_ACCESS_TOKEN>&weight_name=<WEIGHTS_NAME>
https://huggingface.co/<HF_SLUG>/<WEIGHTS_NAME>?token=<INSERT_HF_ACCESS_TOKEN>

And other loading parameters to be specified keeping backwards compatibility.

Figured using the existing huggingface url match and url parsing would be a flexible way to pass anything to the load statement without changing anything else, but let me know if you'd like it done a different way

This change allows a user to pass a custom `hf_access_token` with the hf_lora url in order to load a custom lora from a private repo.

It follows the existing url path match `^https?://huggingface.co` and extracts query parameters to pass directly to `pipe.load_lora_weights`
This allows `token=<INSERT_HF_ACCESS_TOKEN>&weight_name=<WEIGHTS_NAME>` and other loading parameters to be specified
@Averylamp Averylamp changed the title Allow for private HF_Lora urls with query params for loading Allow for Private HF_Lora URLs with Query Params for Auth Sep 27, 2024
Fix double quote escape
fix parseql to be a flat dictionary
@Averylamp
Copy link
Author

FYI will test it with a private deployment to make sure it works, but threw up a PR to get a first read

@gregorym
Copy link

@Averylamp Were you able to test it?

@Averylamp
Copy link
Author

I did test it awhile back, but then decided that I'd rather use an fp8 compatible lora solution. Saw this outstanding PR replicate/cog-flux#24 and then didn't continue for this one because it seems that it will be duplicate work after cog-flux is updated.
I'd love to get an FP8 Flux Dev Lora on H100s solution to use, but am not sure which repo is correct or what the timeline would be on it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants