Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add instructions about setting TGI(S) parameters #74

Open
bdattoma opened this issue Sep 5, 2023 · 2 comments
Open

Add instructions about setting TGI(S) parameters #74

bdattoma opened this issue Sep 5, 2023 · 2 comments
Assignees
@dtrifiro
Labels
kind/documentation Improvements or additions to documentation

Comments

@bdattoma
Copy link
Contributor

bdattoma commented Sep 5, 2023 •
edited
Loading

Users may need to set particular TGI(S) parameter when using Caikit+TGIS runtime on KServe. An example is the model timeout parameter which can be necessary to be tweaked based on the model size.

We should document the procedure in our docs

Additionally, in a future UI effort, this option should be present in the user interface
@bdattoma
Copy link
Contributor Author

bdattoma commented Sep 5, 2023

@MelissaFlinn @Jooho

@heyselbi heyselbi added the kind/documentation Improvements or additions to documentation label Sep 6, 2023
@heyselbi heyselbi moved this from New/Backlog to To-do/Groomed in ODH Model Serving Planning Sep 6, 2023
@heyselbi heyselbi moved this from To-do/Groomed to In Progress in ODH Model Serving Planning Oct 26, 2023
@dtrifiro
Copy link
Contributor

Related #115

@heyselbi heyselbi moved this from In Progress to To-do/Groomed in ODH Model Serving Planning Nov 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dtrifiro dtrifiro

Labels
kind/documentation Improvements or additions to documentation
Projects
Internal tracking
Status: No status
ODH Feature Tracking
Status: No status
ODH Model Serving Planning
Status: To-do/Groomed
Milestone
No milestone
Development

No branches or pull requests
3 participants
@dtrifiro @heyselbi @bdattoma