Skip to content

Issues: clp-research/clembench

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

28 Open 40 Closed
28 Open 40 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Captured exception is unused
#149 opened Jan 21, 2025 by peter-k4rsten
[feature] Make a pip installable library of clembench framework
#147 opened Jan 14, 2025 by sherzod-hakimov
[feature] Reduce the Python requirements to have minimal version where only the requirement components are left
#146 opened Jan 14, 2025 by sherzod-hakimov
[feature] programmatic way to enquire about available instances
#145 opened Jan 9, 2025 by davidschlangen
optimise packaging and calling of benchmark
#140 opened Dec 11, 2024 by davidschlangen
@phisad
5
[games] make separation between games and worlds clearer
#133 opened Nov 8, 2024 by davidschlangen
2
[documentation] run pdoc or similar to automatically create API documentation documentation Improvements or additions to documentation
#125 opened Sep 16, 2024 by davidschlangen
6
[documentation] Add "how_to_contribute" doc documentation Improvements or additions to documentation
#124 opened Sep 16, 2024 by davidschlangen
@sherzod-hakimov
[backends] Benchmark configuration file to change settings such as openai-compatible API backend enhancement New feature or request
#112 opened Aug 7, 2024 by Gnurro
2
Missing logs for episodes with exceptions enhancement New feature or request
#110 opened Jul 31, 2024 by Gnurro
1
[analysis] LLM Calculator: inference parameters, like latency etc. - find the best models based on filters
#100 opened Jul 9, 2024 by davidschlangen
@sherzod-hakimov
2
[framework] game registry enhancement New feature or request
#99 opened Jul 9, 2024 by davidschlangen
@AnneBeyer
20
[wordle] fix re-prompt prompts
#98 opened Jul 2, 2024 by davidschlangen v2
1
[games] make “abort” consistent
#95 opened Jun 20, 2024 by davidschlangen
[games] Do another round of prompt criticism / unification. For clembench 1.6
#87 opened May 1, 2024 by davidschlangen
1
[framework] rethink response parsing
#86 opened Apr 30, 2024 by davidschlangen
[leaderboard] rethink approach to deciding on which models to test
#83 opened Apr 19, 2024 by davidschlangen
1
Proper step-by-step guide for game creation
#82 opened Apr 18, 2024 by Gnurro
7
[framework] Integrate External API Backends (Together) that support many models
#78 opened Apr 11, 2024 by sherzod-hakimov
1
[games] Open questions on private/shared game
#71 opened Mar 26, 2024 by briemadu
2
model responses differ when run locally and when run via fastchat API
#70 opened Mar 19, 2024 by davidschlangen
[structure] separate framework from benchmark (from games?) / make pip installable
#62 opened Feb 29, 2024 by davidschlangen
1
[framework] remove special handling of programmatic Players breaking Framework users need to adjust their code enhancement New feature or request
#60 opened Feb 29, 2024 by phisad
@phisad
[scoring] ensure that all scores use the full range 0:100
#58 opened Feb 22, 2024 by davidschlangen
1
[framework] future-proof: allow any kind of data in conversation
#56 opened Feb 21, 2024 by davidschlangen
2
ProTip! Mix and match filters to narrow down what you’re looking for.