Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support DeepSeek reasoning_content field #1173

Open
tadamcz opened this issue Jan 22, 2025 · 8 comments
Open

Support DeepSeek reasoning_content field #1173

tadamcz opened this issue Jan 22, 2025 · 8 comments

Comments

@tadamcz
Copy link
Contributor

tadamcz commented Jan 22, 2025

reasoning_content:The content of the CoT, which is at the same level as content in the output structure.

https://api-docs.deepseek.com/guides/reasoning_model

We'd need to store that in appropriate Inspect data structures.

@jjallaire
Copy link
Collaborator

This could be added to the ModelOutput class.

@tadamcz
Copy link
Contributor Author

tadamcz commented Jan 22, 2025

ChatCompletionChoice, I think, rather than directly on ModelOutput? If you haven't got anyone working on this yet, I can take a crack at it

@jjallaire
Copy link
Collaborator

Sorry, yes. ChatCompletionChoice is the right container. I'm wonder if this actually belongs on ChatMessageAssistant then? (as then UIs which deal with lists of messages could show reasoning_content?)

@tadamcz
Copy link
Contributor Author

tadamcz commented Jan 23, 2025

I'm wonder if this actually belongs on ChatMessageAssistant then?

Yes, it's debatable. Let me play around with the DeepSeek API and think about it some more. Then I'll open a PR. Might not get to it this week though.

@tadamcz
Copy link
Contributor Author

tadamcz commented Jan 28, 2025

Blocked because the DeepSeek API has been very difficult to access in the last few days

@menhguin
Copy link
Contributor

menhguin commented Feb 1, 2025

hello, i already have a barebones implementation on deepseek on my fork based off the grok and openai implementations. unfortunately yes, chat API barely works nowadays let alone reasoning for me to test things.
https://github.com/menhguin/inspect_ai/blob/main/src/inspect_ai/model/_providers/deepseek.py

@tadamcz
Copy link
Contributor Author

tadamcz commented Feb 1, 2025

Feel free to PR the change if you think it's ready

@jjallaire
Copy link
Collaborator

PR here: #1229

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants