Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Python: Token usage from Azure OpenAI streaming chat completion #8996

Open
TaoChenOSU opened this issue Sep 25, 2024 · 2 comments · May be fixed by #9753
Open

Python: Token usage from Azure OpenAI streaming chat completion #8996

TaoChenOSU opened this issue Sep 25, 2024 · 2 comments · May be fixed by #9753
Assignees
@TaoChenOSU
Labels
ai connector Anything related to AI connectors python Pull requests for the Python Semantic Kernel sk team issue A tag to denote issues that where created by the Semantic Kernel team (i.e., not the community)

Comments

@TaoChenOSU
Copy link
Contributor

Currently, OpenAI provides the functionality to return token usage through their Streaming Chat Completion API. However, this feature has not yet been incorporated into Azure OpenAI. Upon experimenting with the stream_options parameter with the OpenAI Python SDK, I noticed that using this parameter may lead to errors when using the latest Azure OpenAI GA API version: 2024-06-01.

This issue has been created to monitor their progress, enabling us to implement the feature within our Azure OpenAI connector as soon as the Azure team integrates support for this functionality.
@TaoChenOSU TaoChenOSU added python Pull requests for the Python Semantic Kernel ai connector Anything related to AI connectors labels Sep 25, 2024
@TaoChenOSU TaoChenOSU self-assigned this Sep 25, 2024
@github-actions github-actions bot changed the title New Feature: Token usage from Azure OpenAI streaming chat completion Python: New Feature: Token usage from Azure OpenAI streaming chat completion Sep 25, 2024
@moonbox3 moonbox3 removed the triage label Sep 26, 2024
@schneidenbach
Copy link

Curious if there's been any update - and will a separate issue be opened for the .NET library?

@evchaki evchaki added the sk team issue A tag to denote issues that where created by the Semantic Kernel team (i.e., not the community) label Oct 18, 2024
@RogerBarreto
Copy link
Member

@schneidenbach , yes our latest update to use the beta.2 of Azure.AI.OpenAI package resolves their issue.

Our tracking issue was:

@TaoChenOSU TaoChenOSU linked a pull request Nov 20, 2024 that will close this issue
Python: Introduced a new condition to yield StreamingChatMessageContent directly when usage data is available. #9753
Open
4 tasks
@TaoChenOSU TaoChenOSU changed the title Python: New Feature: Token usage from Azure OpenAI streaming chat completion Python: Token usage from Azure OpenAI streaming chat completion Nov 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@TaoChenOSU TaoChenOSU

Labels
ai connector Anything related to AI connectors python Pull requests for the Python Semantic Kernel sk team issue A tag to denote issues that where created by the Semantic Kernel team (i.e., not the community)
Projects
Semantic Kernel
Status: No status
Milestone
No milestone
6 participants
@schneidenbach @TaoChenOSU @RogerBarreto @moonbox3 @evchaki @markwallace-microsoft