Skip to content

Issues: open-compass/T-Eval

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clear current search query, filters, and sorts
36 Open 18 Closed
36 Open 18 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

数据链接打不开
#59 opened Jul 15, 2024 by cobraheleah updated Jan 20, 2025
2
AttributeError: 'GenerationConfig' object has no attribute '_eos_token_tensor'
#65 opened Jan 19, 2025 by LiaoNiuyu updated Jan 19, 2025
BUG
#64 opened Jan 13, 2025 by Jasperzkk updated Jan 13, 2025
执行plan任务会卡住，没有输出执行结果文件
#62 opened Nov 8, 2024 by 1207408359 updated Nov 8, 2024
Failed to convert pandas DataFrame to Arrow Table from file '/home/####/.cache/huggingface/hub/datasets--lovesnowbest--T-Eval/snapshots/af355ab2b62cdbae4262ac41c7529ffeae395012/data/instruct_v2.json' with error <class 'pyarrow.lib.ArrowInvalid'>: ('cannot mix list and non-list, non-null values', 'Conversion failed for column query_60_0_3 with type object')
#61 opened Nov 3, 2024 by OfficerChul updated Nov 3, 2024
能否根据结果文件汇总指标得到overall结果的脚本？
#60 opened Aug 8, 2024 by Cppowboy updated Aug 8, 2024
如何计算的分数
#58 opened Jul 13, 2024 by hmppt updated Jul 13, 2024
Opensource Model not found ——Nanbeige-Agent-32B
#57 opened Jun 13, 2024 by tttonytan updated Jun 13, 2024
llama2模板有问题
#55 opened Jun 1, 2024 by li-aolong updated Jun 1, 2024
推理速度慢
#54 opened May 28, 2024 by lihaibineric updated May 30, 2024
测试结果不完整
#52 opened May 10, 2024 by Mrak6192 updated May 15, 2024
2
Can not eval when set batch_size>1
#48 opened Apr 9, 2024 by dkqkxx updated May 15, 2024
2
请问能否提供一份完全对齐openai输入格式的测试数据
#49 opened Apr 11, 2024 by Watebear updated May 14, 2024
2
Llama2 7b chat 模型，输入长度超过 4096
#44 opened Mar 28, 2024 by Watebear updated May 14, 2024
1
Review测评指标失真，Qwen被严重低估了
#53 opened May 14, 2024 by fengzhu1 updated May 14, 2024
1
对数据集case数的疑问
#50 opened Apr 12, 2024 by AmberXu98 updated Apr 26, 2024
2
是否支持qwen1.5，复现结果差距较大
#51 opened Apr 23, 2024 by Little-girl-1992 updated Apr 23, 2024
How to use multi-gpu to test?
#42 opened Mar 21, 2024 by seanxuu updated Mar 21, 2024
qwen14B测试python test.py 报错
#40 opened Mar 21, 2024 by chococatsrin updated Mar 21, 2024
qwen1.5 tokenizer错误
#39 opened Mar 21, 2024 by chococatsrin updated Mar 21, 2024
qwen-14b评测结果疑问
#38 opened Mar 11, 2024 by Fenglly updated Mar 11, 2024
API model
#34 opened Mar 4, 2024 by Fenglly updated Mar 8, 2024
3
Evaluate Claude 3
#37 opened Mar 5, 2024 by stalkermustang updated Mar 5, 2024
关于plan_json_v1_zh.json数据文件答案问题
#36 opened Mar 5, 2024 by 13416157913 updated Mar 5, 2024
关于plan_json_v1_zh.json数据文件答案问题
#35 opened Mar 5, 2024 by 13416157913 updated Mar 5, 2024
ProTip! no:milestone will show everything without a milestone.