Skip to content

[Proposal]: Standardize Non-blocking Behavior for ChatModelBase Implementations #571

@wuji1428

Description

@wuji1428

AgentScope-Java is an open-source project. To involve a broader community, we recommend asking your questions in English.

Description

Currently, the five primary implementations of ChatModelBase (OpenAIChatModel, AnthropicChatModel, DashScopeChatModel, GeminiChatModel, and OllamaChatModel) exhibit inconsistent behavior when handling non-streaming calls.

The Core Issue

In non-streaming mode (stream=false), different implementation classes exhibit different blocking behaviors towards the calling thread:

Implementation Non-streaming
OpenAIChatModel Blocking
AnthropicChatModel Non-blocking
DashScopeChatModel Blocking
GeminiChatModel Non-blocking
OllamaChatModel Blocking

Why This Needs to be Addressed

  1. Semantic Inconsistency: The same client code behaves differently depending on the underlying model. This leads to unpredictable application performance and inconsistent API semantics.
  2. Developer Experience: Users expect a unified Reactive interface (returning Flux/Mono) to follow consistent non-blocking patterns.
  3. Maintenance Complexity: It increases the cognitive load for developers integrating multiple models.

Proposal: Standardize on Non-blocking Mode

I propose that all ChatModelBase implementations should be updated to ensure that non-streaming calls do not block the calling thread.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Status

    Backlog

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions