Fix unpickleable fields due to openai client #79
Conversation
| openai_model: str = Field(alias="model") | ||
| openai_api_key: Optional[str] = Field( | ||
| default=os.getenv("OPENAI_API_KEY"), alias="api_key" | ||
| ) | ||
| max_tokens: Optional[int] = 1000 | ||
| splitter: Optional[str] = None | ||
|
|
||
| _client: OpenAI = None | ||
| @computed_field | ||
| def _client(self) -> OpenAI: |
There was a problem hiding this comment.
why do we need computed_field here but create a client on the fly in the Async version? let's use the unique approach if possible
There was a problem hiding this comment.
because the sync runtime actually uses its _client, but the async runtime does not, it directly sends post requests in async_create_completion. Even if it did use the python client lib instead of requests, it wouldn't make sense for the class to carry a single client instance because it would create them on demand to use them concurrently
There was a problem hiding this comment.
@niklub if you feel strongly about this I'll change it, but I think it's a meaningful difference that the async openai runtime doesn't need a self._client
matt-bernstein
force-pushed
the
fix/unpickleable-fields
branch
from
35627d7 to
013c75a
Compare
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #79 +/- ##
==========================================
+ Coverage 53.22% 53.25% +0.03%
==========================================
Files 42 42
Lines 1704 1703 -1
==========================================
Hits 907 907
+ Misses 797 796 -1 ☔ View full report in Codecov by Sentry. |
Bugfix -
/submitrequests to the server were failing with the following stack trace:The root cause is that the
OpenAIclient object inAsyncOpenAIChatRuntimeandOpenAIChatRuntimewas attempted to be serialized usingpickle.dumps(), which is not supported after upgrading the openai client lib: openai/openai-python#837The solution was to make it a computed field in the sync client, and remove it from the async client as it is unused.