Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

voyage fails #1681

Open
Muennighoff opened this issue Jan 2, 2025 · 2 comments
Open

voyage fails #1681

Muennighoff opened this issue Jan 2, 2025 · 2 comments

Comments

@Muennighoff
Copy link
Contributor 

2025-01-02 01:18:05.884113: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:485] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2025-01-02 01:18:05.898899: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:8454] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2025-01-02 01:18:05.903020: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1452] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
INFO:mteb.cli:Running with parameters: Namespace(model='voyageai/voyage-large-2-instruct', task_types=None, categories=None, tasks=['XQuADRetrieval'], languages=None, benchmarks=None, device=None, output_folder='/data/niklas/results/results', verbosity=2, co2_tracker=True, eval_splits=None, model_revision=None, batch_size=64, overwrite=False, save_predictions=False, func=<function run at 0x7fd207adbd00>)
INFO:mteb.evaluation.MTEB:

## Evaluating 1 tasks:
─────────────────────────────── Selected tasks  ────────────────────────────────
Retrieval
    - XQuADRetrieval, s2p, multilingual 12 / 12 Subsets


INFO:mteb.evaluation.MTEB:

********************** Evaluating XQuADRetrieval **********************
Overwrite dataset info from restored data version if exists.
INFO:datasets.builder:Overwrite dataset info from restored data version if exists.
Loading Dataset info from /data/huggingface/datasets/google___xquad/xquad.ar/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583
INFO:datasets.info:Loading Dataset info from /data/huggingface/datasets/google___xquad/xquad.ar/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583
Found cached dataset xquad (/data/huggingface/datasets/google___xquad/xquad.ar/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583)
INFO:datasets.builder:Found cached dataset xquad (/data/huggingface/datasets/google___xquad/xquad.ar/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583)
Loading Dataset info from /data/huggingface/datasets/google___xquad/xquad.ar/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583
INFO:datasets.info:Loading Dataset info from /data/huggingface/datasets/google___xquad/xquad.ar/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583
Loading cached processed dataset at /data/huggingface/datasets/google___xquad/xquad.ar/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583/cache-6a017b34aa9256ff.arrow
INFO:datasets.arrow_dataset:Loading cached processed dataset at /data/huggingface/datasets/google___xquad/xquad.ar/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583/cache-6a017b34aa9256ff.arrow
Overwrite dataset info from restored data version if exists.
INFO:datasets.builder:Overwrite dataset info from restored data version if exists.
Loading Dataset info from /data/huggingface/datasets/google___xquad/xquad.de/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583
INFO:datasets.info:Loading Dataset info from /data/huggingface/datasets/google___xquad/xquad.de/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583
Found cached dataset xquad (/data/huggingface/datasets/google___xquad/xquad.de/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583)
INFO:datasets.builder:Found cached dataset xquad (/data/huggingface/datasets/google___xquad/xquad.de/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583)
Loading Dataset info from /data/huggingface/datasets/google___xquad/xquad.de/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583
INFO:datasets.info:Loading Dataset info from /data/huggingface/datasets/google___xquad/xquad.de/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583
Loading cached processed dataset at /data/huggingface/datasets/google___xquad/xquad.de/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583/cache-a3af057a3a8638fe.arrow
INFO:datasets.arrow_dataset:Loading cached processed dataset at /data/huggingface/datasets/google___xquad/xquad.de/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583/cache-a3af057a3a8638fe.arrow
Overwrite dataset info from restored data version if exists.
INFO:datasets.builder:Overwrite dataset info from restored data version if exists.
Loading Dataset info from /data/huggingface/datasets/google___xquad/xquad.el/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583
INFO:datasets.info:Loading Dataset info from /data/huggingface/datasets/google___xquad/xquad.el/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583
Found cached dataset xquad (/data/huggingface/datasets/google___xquad/xquad.el/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583)
INFO:datasets.builder:Found cached dataset xquad (/data/huggingface/datasets/google___xquad/xquad.el/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583)
Loading Dataset info from /data/huggingface/datasets/google___xquad/xquad.el/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583
INFO:datasets.info:Loading Dataset info from /data/huggingface/datasets/google___xquad/xquad.el/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583
Loading cached processed dataset at /data/huggingface/datasets/google___xquad/xquad.el/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583/cache-c5485ef1bdb4fa3b.arrow
INFO:datasets.arrow_dataset:Loading cached processed dataset at /data/huggingface/datasets/google___xquad/xquad.el/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583/cache-c5485ef1bdb4fa3b.arrow
Overwrite dataset info from restored data version if exists.
INFO:datasets.builder:Overwrite dataset info from restored data version if exists.
Loading Dataset info from /data/huggingface/datasets/google___xquad/xquad.en/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583
INFO:datasets.info:Loading Dataset info from /data/huggingface/datasets/google___xquad/xquad.en/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583
Found cached dataset xquad (/data/huggingface/datasets/google___xquad/xquad.en/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583)
INFO:datasets.builder:Found cached dataset xquad (/data/huggingface/datasets/google___xquad/xquad.en/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583)
Loading Dataset info from /data/huggingface/datasets/google___xquad/xquad.en/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583
INFO:datasets.info:Loading Dataset info from /data/huggingface/datasets/google___xquad/xquad.en/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583
Loading cached processed dataset at /data/huggingface/datasets/google___xquad/xquad.en/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583/cache-30f06d2235fe7933.arrow
INFO:datasets.arrow_dataset:Loading cached processed dataset at /data/huggingface/datasets/google___xquad/xquad.en/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583/cache-30f06d2235fe7933.arrow
Overwrite dataset info from restored data version if exists.
INFO:datasets.builder:Overwrite dataset info from restored data version if exists.
Loading Dataset info from /data/huggingface/datasets/google___xquad/xquad.es/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583
INFO:datasets.info:Loading Dataset info from /data/huggingface/datasets/google___xquad/xquad.es/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583
Found cached dataset xquad (/data/huggingface/datasets/google___xquad/xquad.es/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583)
INFO:datasets.builder:Found cached dataset xquad (/data/huggingface/datasets/google___xquad/xquad.es/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583)
Loading Dataset info from /data/huggingface/datasets/google___xquad/xquad.es/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583
INFO:datasets.info:Loading Dataset info from /data/huggingface/datasets/google___xquad/xquad.es/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583
Loading cached processed dataset at /data/huggingface/datasets/google___xquad/xquad.es/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583/cache-fcdd5eb0181d537e.arrow
INFO:datasets.arrow_dataset:Loading cached processed dataset at /data/huggingface/datasets/google___xquad/xquad.es/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583/cache-fcdd5eb0181d537e.arrow
Overwrite dataset info from restored data version if exists.
INFO:datasets.builder:Overwrite dataset info from restored data version if exists.
Loading Dataset info from /data/huggingface/datasets/google___xquad/xquad.hi/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583
INFO:datasets.info:Loading Dataset info from /data/huggingface/datasets/google___xquad/xquad.hi/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583
Found cached dataset xquad (/data/huggingface/datasets/google___xquad/xquad.hi/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583)
INFO:datasets.builder:Found cached dataset xquad (/data/huggingface/datasets/google___xquad/xquad.hi/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583)
Loading Dataset info from /data/huggingface/datasets/google___xquad/xquad.hi/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583
INFO:datasets.info:Loading Dataset info from /data/huggingface/datasets/google___xquad/xquad.hi/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583
Loading cached processed dataset at /data/huggingface/datasets/google___xquad/xquad.hi/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583/cache-4f36d325a204a113.arrow
INFO:datasets.arrow_dataset:Loading cached processed dataset at /data/huggingface/datasets/google___xquad/xquad.hi/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583/cache-4f36d325a204a113.arrow
Overwrite dataset info from restored data version if exists.
INFO:datasets.builder:Overwrite dataset info from restored data version if exists.
Loading Dataset info from /data/huggingface/datasets/google___xquad/xquad.ro/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583
INFO:datasets.info:Loading Dataset info from /data/huggingface/datasets/google___xquad/xquad.ro/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583
Found cached dataset xquad (/data/huggingface/datasets/google___xquad/xquad.ro/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583)
INFO:datasets.builder:Found cached dataset xquad (/data/huggingface/datasets/google___xquad/xquad.ro/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583)
Loading Dataset info from /data/huggingface/datasets/google___xquad/xquad.ro/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583
INFO:datasets.info:Loading Dataset info from /data/huggingface/datasets/google___xquad/xquad.ro/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583
Loading cached processed dataset at /data/huggingface/datasets/google___xquad/xquad.ro/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583/cache-27d9ef10b44ddf79.arrow
INFO:datasets.arrow_dataset:Loading cached processed dataset at /data/huggingface/datasets/google___xquad/xquad.ro/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583/cache-27d9ef10b44ddf79.arrow
Overwrite dataset info from restored data version if exists.
INFO:datasets.builder:Overwrite dataset info from restored data version if exists.
Loading Dataset info from /data/huggingface/datasets/google___xquad/xquad.ru/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583
INFO:datasets.info:Loading Dataset info from /data/huggingface/datasets/google___xquad/xquad.ru/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583
Found cached dataset xquad (/data/huggingface/datasets/google___xquad/xquad.ru/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583)
INFO:datasets.builder:Found cached dataset xquad (/data/huggingface/datasets/google___xquad/xquad.ru/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583)
Loading Dataset info from /data/huggingface/datasets/google___xquad/xquad.ru/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583
INFO:datasets.info:Loading Dataset info from /data/huggingface/datasets/google___xquad/xquad.ru/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583
Loading cached processed dataset at /data/huggingface/datasets/google___xquad/xquad.ru/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583/cache-7f3c87071d48c03a.arrow
INFO:datasets.arrow_dataset:Loading cached processed dataset at /data/huggingface/datasets/google___xquad/xquad.ru/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583/cache-7f3c87071d48c03a.arrow
Overwrite dataset info from restored data version if exists.
INFO:datasets.builder:Overwrite dataset info from restored data version if exists.
Loading Dataset info from /data/huggingface/datasets/google___xquad/xquad.th/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583
INFO:datasets.info:Loading Dataset info from /data/huggingface/datasets/google___xquad/xquad.th/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583
Found cached dataset xquad (/data/huggingface/datasets/google___xquad/xquad.th/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583)
INFO:datasets.builder:Found cached dataset xquad (/data/huggingface/datasets/google___xquad/xquad.th/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583)
Loading Dataset info from /data/huggingface/datasets/google___xquad/xquad.th/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583
INFO:datasets.info:Loading Dataset info from /data/huggingface/datasets/google___xquad/xquad.th/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583
Loading cached processed dataset at /data/huggingface/datasets/google___xquad/xquad.th/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583/cache-2bf05a6acd6d114c.arrow
INFO:datasets.arrow_dataset:Loading cached processed dataset at /data/huggingface/datasets/google___xquad/xquad.th/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583/cache-2bf05a6acd6d114c.arrow
Overwrite dataset info from restored data version if exists.
INFO:datasets.builder:Overwrite dataset info from restored data version if exists.
Loading Dataset info from /data/huggingface/datasets/google___xquad/xquad.tr/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583
INFO:datasets.info:Loading Dataset info from /data/huggingface/datasets/google___xquad/xquad.tr/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583
Found cached dataset xquad (/data/huggingface/datasets/google___xquad/xquad.tr/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583)
INFO:datasets.builder:Found cached dataset xquad (/data/huggingface/datasets/google___xquad/xquad.tr/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583)
Loading Dataset info from /data/huggingface/datasets/google___xquad/xquad.tr/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583
INFO:datasets.info:Loading Dataset info from /data/huggingface/datasets/google___xquad/xquad.tr/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583
Loading cached processed dataset at /data/huggingface/datasets/google___xquad/xquad.tr/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583/cache-c4dfc0427774aace.arrow
INFO:datasets.arrow_dataset:Loading cached processed dataset at /data/huggingface/datasets/google___xquad/xquad.tr/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583/cache-c4dfc0427774aace.arrow
Overwrite dataset info from restored data version if exists.
INFO:datasets.builder:Overwrite dataset info from restored data version if exists.
Loading Dataset info from /data/huggingface/datasets/google___xquad/xquad.vi/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583
INFO:datasets.info:Loading Dataset info from /data/huggingface/datasets/google___xquad/xquad.vi/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583
Found cached dataset xquad (/data/huggingface/datasets/google___xquad/xquad.vi/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583)
INFO:datasets.builder:Found cached dataset xquad (/data/huggingface/datasets/google___xquad/xquad.vi/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583)
Loading Dataset info from /data/huggingface/datasets/google___xquad/xquad.vi/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583
INFO:datasets.info:Loading Dataset info from /data/huggingface/datasets/google___xquad/xquad.vi/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583
Loading cached processed dataset at /data/huggingface/datasets/google___xquad/xquad.vi/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583/cache-ac3efd33e1264bce.arrow
INFO:datasets.arrow_dataset:Loading cached processed dataset at /data/huggingface/datasets/google___xquad/xquad.vi/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583/cache-ac3efd33e1264bce.arrow
Overwrite dataset info from restored data version if exists.
INFO:datasets.builder:Overwrite dataset info from restored data version if exists.
Loading Dataset info from /data/huggingface/datasets/google___xquad/xquad.zh/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583
INFO:datasets.info:Loading Dataset info from /data/huggingface/datasets/google___xquad/xquad.zh/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583
Found cached dataset xquad (/data/huggingface/datasets/google___xquad/xquad.zh/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583)
INFO:datasets.builder:Found cached dataset xquad (/data/huggingface/datasets/google___xquad/xquad.zh/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583)
Loading Dataset info from /data/huggingface/datasets/google___xquad/xquad.zh/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583
INFO:datasets.info:Loading Dataset info from /data/huggingface/datasets/google___xquad/xquad.zh/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583
Loading cached processed dataset at /data/huggingface/datasets/google___xquad/xquad.zh/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583/cache-ea6c623f3b415adb.arrow
INFO:datasets.arrow_dataset:Loading cached processed dataset at /data/huggingface/datasets/google___xquad/xquad.zh/0.0.0/51adfef1c1287aab1d2d91b5bead9bcfb9c68583/cache-ea6c623f3b415adb.arrow
INFO:mteb.abstasks.AbsTaskRetrieval:Subset: ar
INFO:mteb.evaluation.evaluators.RetrievalEvaluator:Encoding Queries.
INFO:mteb.evaluation.evaluators.RetrievalEvaluator:Sorting Corpus by document length (Longest first)...
INFO:mteb.evaluation.evaluators.RetrievalEvaluator:Encoding Corpus in batches... Warning: This might take a while!
INFO:mteb.evaluation.evaluators.RetrievalEvaluator:Encoding Batch 1/1...
ERROR:mteb.evaluation.MTEB:Error while evaluating XQuADRetrieval: The request body is not valid JSON, or some arguments were not specified properly. In particular, Value 'passage' supplied for argument 'input_type' is not valid -- accepted values are 'query' or 'document'
Traceback (most recent call last):
  File "/env/lib/conda/gritkto/bin/mteb", line 8, in <module>
    sys.exit(main())
  File "/data/niklas/mteb/mteb/cli.py", line 387, in main
    args.func(args)
  File "/data/niklas/mteb/mteb/cli.py", line 145, in run
    eval.run(
  File "/data/niklas/mteb/mteb/evaluation/MTEB.py", line 623, in run
    raise e
  File "/data/niklas/mteb/mteb/evaluation/MTEB.py", line 576, in run
    results, tick, tock = self._run_eval(
  File "/data/niklas/mteb/mteb/evaluation/MTEB.py", line 304, in _run_eval
    results = task.evaluate(
  File "/data/niklas/mteb/mteb/abstasks/AbsTaskRetrieval.py", line 339, in evaluate
    scores[hf_subset] = self._evaluate_subset(
  File "/data/niklas/mteb/mteb/abstasks/AbsTaskRetrieval.py", line 348, in _evaluate_subset
    results = retriever(corpus, queries)
  File "/data/niklas/mteb/mteb/evaluation/evaluators/RetrievalEvaluator.py", line 478, in __call__
    return self.retriever.search(
  File "/data/niklas/mteb/mteb/evaluation/evaluators/RetrievalEvaluator.py", line 159, in search
    sub_corpus_embeddings = self.model.encode(
  File "/data/niklas/mteb/mteb/evaluation/evaluators/RetrievalEvaluator.py", line 414, in encode
    return self.encode_corpus(
  File "/data/niklas/mteb/mteb/evaluation/evaluators/RetrievalEvaluator.py", line 394, in encode_corpus
    corpus_embeddings = self.model.encode(
  File "/data/niklas/mteb/mteb/models/voyage_models.py", line 101, in encode
    return self._batched_encode(sentences, batch_size, input_type)
  File "/data/niklas/mteb/mteb/models/voyage_models.py", line 128, in _batched_encode
    self._embed_func(
  File "/data/niklas/mteb/mteb/models/voyage_models.py", line 58, in wrapper
    result = func(*args, **kwargs)
  File "/data/niklas/mteb/mteb/models/voyage_models.py", line 25, in wrapper
    result = func(*args, **kwargs)
  File "/env/lib/conda/gritkto/lib/python3.10/site-packages/voyageai/client.py", line 68, in embed
    for attempt in self.retry_controller:
  File "/env/lib/conda/gritkto/lib/python3.10/site-packages/tenacity/__init__.py", line 435, in __iter__
    do = self.iter(retry_state=retry_state)
  File "/env/lib/conda/gritkto/lib/python3.10/site-packages/tenacity/__init__.py", line 368, in iter
    result = action(retry_state)
  File "/env/lib/conda/gritkto/lib/python3.10/site-packages/tenacity/__init__.py", line 390, in <lambda>
    self._add_action_func(lambda rs: rs.outcome.result())
  File "/env/lib/conda/gritkto/lib/python3.10/concurrent/futures/_base.py", line 451, in result
    return self.__get_result()
  File "/env/lib/conda/gritkto/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
  File "/env/lib/conda/gritkto/lib/python3.10/site-packages/voyageai/client.py", line 70, in embed
    response = voyageai.Embedding.create(
  File "/env/lib/conda/gritkto/lib/python3.10/site-packages/voyageai/api_resources/embedding.py", line 22, in create
    response = super().create(*args, **kwargs)
  File "/env/lib/conda/gritkto/lib/python3.10/site-packages/voyageai/api_resources/api_resource.py", line 47, in create
    response = requestor.request(
  File "/env/lib/conda/gritkto/lib/python3.10/site-packages/voyageai/api_resources/api_requestor.py", line 147, in request
    resp = self._interpret_response(result)
  File "/env/lib/conda/gritkto/lib/python3.10/site-packages/voyageai/api_resources/api_requestor.py", line 408, in _interpret_response
    return self._interpret_response_line(
  File "/env/lib/conda/gritkto/lib/python3.10/site-packages/voyageai/api_resources/api_requestor.py", line 463, in _interpret_response_line
    raise self.handle_error_response(rbody, rcode, resp.data, rheaders)
voyageai.error.InvalidRequestError: The request body is not valid JSON, or some arguments were not specified properly. In particular, Value 'passage' supplied for argument 'input_type' is not valid -- accepted values are 'query' or 'document'
@Samoed
Copy link
Collaborator

Samoed commented Jan 2, 2025

I can't reproduce the error with latest mteb

@isaac-chung
Copy link
Collaborator

I ran into a rate limit error when trying to reproduce this. But since the error message is the same as in #1652, I wonder if the latest main branch would resolve this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@Samoed @isaac-chung @Muennighoff