New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Stop generate() when reaching text sequence #15

Open

kripper opened this issue Aug 25, 2023 · 0 comments

kripper commented Aug 25, 2023

First, congratulations for integrating the CTranslate2 implementation with HF Transformers. This was necessary.

Question: Is there any way to stop generate() when reaching a specific text sequence?
We need this to deal with this issue: https://huggingface.co/clibrain/Llama-2-7b-ft-instruct-es/discussions/1

Somehing like a callback to cancel generate() would also be usefull (when a user triggers a cancel).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment