Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reranker models do not work with cli #1671

Open
Muennighoff opened this issue Jan 1, 2025 · 0 comments · May be fixed by #1676
Open

Reranker models do not work with cli #1671

Muennighoff opened this issue Jan 1, 2025 · 0 comments · May be fixed by #1676

Comments

@Muennighoff
Copy link
Contributor

Issue appears to be here

model = mteb.get_model(args.model, args.model_revision, device=device)

2025-01-01 17:38:32.069261: I tensorflow/core/util/port.cc:153] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2025-01-01 17:38:32.081509: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:485] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2025-01-01 17:38:32.094956: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:8454] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2025-01-01 17:38:32.098822: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1452] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2025-01-01 17:38:32.109108: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F AVX512_VNNI AVX512_BF16 AVX512_FP16 AVX_VNNI AMX_TILE AMX_INT8 AMX_BF16 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
INFO:mteb.cli:Running with parameters: Namespace(model='jhu-clsp/FollowIR-7B', task_types=None, categories=None, tasks=['TwitterHjerneRetrieval'], languages=None, benchmarks=None, device=None, output_folder='/data/niklas/results/results', verbosity=2, co2_tracker=True, eval_splits=None, model_revision=None, batch_size=16, overwrite=False, save_predictions=False, func=<function run at 0x7fd28b9af9a0>)
Traceback (most recent call last):
  File "/env/lib/conda/gritkto/bin/mteb", line 8, in <module>
    sys.exit(main())
  File "/data/niklas/mteb/mteb/cli.py", line 387, in main
    args.func(args)
  File "/data/niklas/mteb/mteb/cli.py", line 123, in run
    model = mteb.get_model(args.model, args.model_revision, device=device, trust_remote_code=True)
  File "/data/niklas/mteb/mteb/models/overview.py", line 150, in get_model
    model = meta.load_model(**kwargs)
  File "/data/niklas/mteb/mteb/model_meta.py", line 128, in load_model
    model: Encoder = loader(**kwargs)  # type: ignore
  File "/data/niklas/mteb/mteb/models/rerankers_custom.py", line 174, in _loader
    return loader_inner()
  File "/data/niklas/mteb/mteb/models/rerankers_custom.py", line 172, in loader_inner
    return wrapper(**_kwargs, **kwargs)
  File "/data/niklas/mteb/mteb/models/rerankers_monot5_based.py", line 258, in __init__
    super().__init__(model_name_or_path, **kwargs)
  File "/data/niklas/mteb/mteb/models/rerankers_monot5_based.py", line 146, in __init__
    super().__init__(model_name_or_path, **kwargs)
TypeError: RerankerWrapper.__init__() got an unexpected keyword argument 'device'

@Samoed Samoed linked a pull request Jan 1, 2025 that will close this issue
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant