Skip to content

Issues: embeddings-benchmark/mteb

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

113 Open 480 Closed
113 Open 480 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Add benchmark on word order enhancement New feature or request new-benchmark
#1699 opened Jan 3, 2025 by KennethEnevoldsen
openbmb/MiniCPM-Embedding still fails
#1696 opened Jan 2, 2025 by Muennighoff
BUCC fails
#1694 opened Jan 2, 2025 by Muennighoff
Unittests to check model loading?
#1690 opened Jan 2, 2025 by Muennighoff
1
3
[mieb] SketchyI2IRetrieval hangs in the beginning mieb The image extension of MTEB
#1686 opened Jan 2, 2025 by Muennighoff
voyage fails
#1681 opened Jan 2, 2025 by Muennighoff
2
LLM2Vec fails
#1679 opened Jan 2, 2025 by Muennighoff
Reranker models do not work with cli
#1671 opened Jan 1, 2025 by Muennighoff
1
BeastyZ/e5-R-mistral-7b fails
#1661 opened Jan 1, 2025 by Muennighoff
1
consciousAI/cai-stellaris-text-embeddings fails
#1660 opened Jan 1, 2025 by Muennighoff
1
Hum-Works/lodestone-base-4096-v1 fails
#1658 opened Jan 1, 2025 by Muennighoff
1
zeta-alpha-ai/Zeta-Alpha-E5-Mistral cannot be loaded
#1655 opened Jan 1, 2025 by Muennighoff
4
infgrad/jasper_en_vision_language_v1 fails on MassiveScenarioClassification
#1653 opened Jan 1, 2025 by Muennighoff
openai/text-embedding-3-large fails on sts22
#1650 opened Jan 1, 2025 by Muennighoff
error when runing e5_v in mieb bug Something isn't working mieb The image extension of MTEB
#1647 opened Dec 31, 2024 by afalf
3
autodetect dataset contamination leaderboard issues related to the leaderboard
#1636 opened Dec 27, 2024 by KennethEnevoldsen
Annotate models trained on commercial-use-friendly licensed data enhancement New feature or request leaderboard issues related to the leaderboard
#1634 opened Dec 27, 2024 by nramrakhiyani
5
Adding enhanced cross-lingual retrieval benchmark by merging retrieval pools from different languages
#1625 opened Dec 24, 2024 by yjoonjang
1
Integrate ModernBERT new-model Questions related to adding a new model to the benchmark
#1624 opened Dec 23, 2024 by Samoed
1
4
Leaderboard: Add ONNX and GGUF to frameworks to allow us to filter based on these leaderboard issues related to the leaderboard
#1622 opened Dec 22, 2024 by KennethEnevoldsen
1
Normalize image encoders interface to the text encoder interface
#1606 opened Dec 17, 2024 by KennethEnevoldsen
1
leaderboard v2 - add autoupdate leaderboard issues related to the leaderboard
#1605 opened Dec 16, 2024 by KennethEnevoldsen
@orionw
5
Investigate performance discrepancies in gte-Qwen and NV-embed models
#1600 opened Dec 16, 2024 by isaac-chung
10
Add Support for Index-Based Retrieval for ColBERT enhancement New feature or request
#1593 opened Dec 14, 2024 by sam-hey
Integrate UniBench mieb The image extension of MTEB new-benchmark
#1590 opened Dec 14, 2024 by Samoed
ProTip! Adding no:label will show everything without a label.