Skip to content

Issues: UKGovernmentBEIS/inspect_ai

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

41 Open 246 Closed
41 Open 246 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Using multiple models from the same provider with different base urls and api keys
#1238 opened Feb 4, 2025 by NanoSean
1
add sandbox parameter to bash and python tools
#1237 opened Feb 4, 2025 by jjallaire
Log viewer: scroll bar resets when using the scorer drop-down
#1220 opened Jan 30, 2025 by tadamcz
@dragonstyle
[Feature request] Support for LiteLLM and LiteLLM proxy
#1213 opened Jan 30, 2025 by danrr
Execute --runslow tests in CI
#1212 opened Jan 30, 2025 by jjallaire-aisi
@art-dsit
1
validate that there is no container_name in docker compose files
#1203 opened Jan 28, 2025 by jjallaire
verify that log_dir is writeable before running eval
#1202 opened Jan 28, 2025 by jjallaire
inspect-ai-vscode adds unnecessary entries to .gitignore.
#1194 opened Jan 27, 2025 by tobiasraabe
1
Tutorial: Mathematics dataset not available in HF anymore
#1183 opened Jan 24, 2025 by guiem
Broader notion of retryable exception
#1174 opened Jan 22, 2025 by tadamcz
10
Support DeepSeek reasoning_content field
#1173 opened Jan 22, 2025 by tadamcz
8
Feature Request: Grouped output metrics & exportable result files for benchmarks
#1172 opened Jan 22, 2025 by owenparsons
@dragonstyle
3
Enable sandboxes to connect to existing Docker or k8s instances
#1153 opened Jan 19, 2025 by jjallaire
@craigwalton-dsit
Enforcing cost limits
#1152 opened Jan 19, 2025 by jjallaire
@craigwalton-dsit
Broader notions of "model" for evals
#1149 opened Jan 19, 2025 by jjallaire
@jjallaire
Bridge to convert from compose.yaml to HELM values
#1148 opened Jan 19, 2025 by jjallaire
@craigwalton-dsit
1
Improve task scheduler to not require model as an axis
#1146 opened Jan 19, 2025 by jjallaire
@jjallaire
Direct linking to samples and transcript entries in Inspect View
#1145 opened Jan 19, 2025 by jjallaire
@dragonstyle
Checkpointing and restarting from checkpoints
#1144 opened Jan 19, 2025 by jjallaire
@jjallaire
Log annotation UI
#1143 opened Jan 19, 2025 by jjallaire
@dragonstyle
Live updating log viewer
#1142 opened Jan 19, 2025 by jjallaire
@dragonstyle
Recording pauses for rate-limiting and transient errors
#1139 opened Jan 19, 2025 by jjallaire
@jjallaire
Feature request: global token cap
#1135 opened Jan 17, 2025 by edkins
GAIA web browser tool returns <empty> when affected by packet loss
#988 opened Dec 12, 2024 by alex-treebeard
viewer: display cost estimates in usage tab
#980 opened Dec 10, 2024 by jjallaire-aisi
ProTip! Add no:assignee to see everything that’s not assigned.