Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: The LLM doesn't understand the Finish tool very well #6406

Open
1 task done
enyst opened this issue Jan 22, 2025 · 0 comments
Open
1 task done

[Bug]: The LLM doesn't understand the Finish tool very well #6406

enyst opened this issue Jan 22, 2025 · 0 comments
Labels
bug Something isn't working

Comments

@enyst
Copy link
Collaborator

enyst commented Jan 22, 2025

Is there an existing issue for the same bug?

  • I have checked the existing issues.

Describe the bug and reproduction steps

Looking at trajectories, specially on evals, but also on those in the resolver I think, I seem to see too often that when the LLM is happy with the job, it sends a MessageAction, not a FinishAction. We insert then a fake user message to remind it to can call the Finish tool if it's done, and then it calls the Finish tool. That's one more iteration, often.

Maybe we can explain better or more clear that it's supposed to call Finish when it thinks the task is done.

OpenHands Installation

Docker command in README

OpenHands Version

No response

Operating System

None

Logs, Errors, Screenshots, and Additional Context

No response

@enyst enyst added the bug Something isn't working label Jan 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant