Stream "API calls quota exceeded"

[JadynWong]

The streaming chat frequently triggers the API quota exceeded error. Any suggestions for optimisation?

2025-09-04 07:40:10.680 +00:00 [ERR] System.Exception: Exception of type 'Microsoft.Teams.Common.Http.HttpException' was thrown.
 ---> "API calls quota exceeded"
   --- End of inner exception stack trace ---
   at Microsoft.Teams.Common.Extensions.TaskExtensions.Retry[T](Task`1 task, Int32 max, Int32 delay)
   at Microsoft.Teams.Common.Extensions.TaskExtensions.Retry[T](Task`1 task, Int32 max, Int32 delay)
   at Microsoft.Teams.Common.Extensions.TaskExtensions.Retry[T](Task`1 task, Int32 max, Int32 delay)
   at Microsoft.Teams.Common.Extensions.TaskExtensions.Retry[T](Task`1 task, Int32 max, Int32 delay)
   at Microsoft.Teams.Plugins.AspNetCore.AspNetCorePlugin.Stream.Close()
   at Microsoft.Teams.Apps.App.Process(ISenderPlugin sender, ActivityEvent event, CancellationToken cancellationToken)

[Message]
public async Task OnMessage(IContext<MessageActivity> context, [Context] MessageActivity activity, [Context] IContext.Client client)
{
  var state = State.From(context);

  using var scope = _serviceScopeFactory.CreateScope();
  
  var chatPrompt = scope.ServiceProvider.GetRequiredService<OpenAIChatPrompt>();

  var contents = new List<IContent>();
  if (!string.IsNullOrWhiteSpace(context.Activity.Text))
  {
      contents.Add(new TextContent() { Text = context.Activity.Text });
  }
  if (context.Activity.Attachments?.Count > 0)
  {
      _logger.LogInformation("Attachment count: {Count}", context.Activity.Attachments.Count);
      await ProcessAttachments(context, contents);
  }

  var response = await chatPrompt.Send(
      content: [.. contents],
      options: new() { Messages = state.Messages },
      onChunk: (chunk) => Task.Run(() =>
      {
         context.Stream.Emit(chunk);
      }, context.CancellationToken),
      cancellationToken: context.CancellationToken
  );
  
  var activityWithResponse = new MessageActivity();
  activityWithResponse.AddAIGenerated();
  context.Stream.Emit(activityWithResponse);
}

[JadynWong]

I used the following optimisation method: merging messages over time before emitting them.

var tempQueue = new Queue<string>();

var queueProcessingTask = Task.Run(async () =>
{
    while (!context.CancellationToken.IsCancellationRequested)
    {
        if (tempQueue.Count > 0)
        {
            // Merge all available chunks
            var merged = string.Empty;
            while (tempQueue.Count > 0)
            {
                merged += tempQueue.Dequeue();
            }
            if (!string.IsNullOrEmpty(merged))
            {
                context.Stream.Emit(merged);
            }
        }
        await Task.Delay(Random.Shared.Next(50, 100), context.CancellationToken);
    }
}, context.CancellationToken);

var response = await chatPrompt.Send(
    content: [.. contents],
    options: new() { Messages = state.Messages },
    onChunk: (chunk) => Task.Run(() =>
    {
       tempQueue.Enqueue(chunk);
    }, context.CancellationToken),
    cancellationToken: context.CancellationToken
);

// Wait for all chunks to be emitted before continuing
// Allow some time for the last chunks to be processed
await Task.Delay(200, context.CancellationToken);

[MehakBindra]

Fixed with release- #118

[JadynWong]

Fixed with release- #118

The version used to report the issue is v2.0.0-preview.10, and PR #103 is already included in preview.10.
Are there any other changes for streaming?