Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

infgrad/jasper_en_vision_language_v1 fails on MassiveScenarioClassification #1653

Open
Muennighoff opened this issue Jan 1, 2025 · 0 comments
Open

infgrad/jasper_en_vision_language_v1 fails on MassiveScenarioClassification #1653

Muennighoff opened this issue Jan 1, 2025 · 0 comments

Comments

@Muennighoff
Copy link
Contributor

also the not loaded weights part looks a scary - is this fine?

2025-01-01 03:02:31.039017: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:485] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
2025-01-01 03:02:31.052914: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:8454] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
2025-01-01 03:02:31.057117: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1452] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
INFO:mteb.cli:Running with parameters: Namespace(model='infgrad/jasper_en_vision_language_v1', task_types=None, categories=None, tasks=['MassiveScenarioClassification'], languages=None, benchmarks=None, device=None, output_folder='/data/niklas/results/results', verbosity=2, co2_tracker=True, eval_splits=None, model_revision=None, batch_size=64, overwrite=False, save_predictions=False, func=<function run at 0x7f5cd8939630>)
Some weights of the model checkpoint at infgrad/jasper_en_vision_language_v1 were not used when initializing JasperVL: ['vision_model.vision_model.embeddings.patch_embedding.bias', 'vision_model.vision_model.embeddings.patch_embedding.weight', 'vision_model.vision_model.embeddings.position_embedding.weight', 'vision_model.vision_model.encoder.layers.0.layer_norm1.bias', 'vision_model.vision_model.encoder.layers.0.layer_norm1.weight', 'vision_model.vision_model.encoder.layers.0.layer_norm2.bias', 'vision_model.vision_model.encoder.layers.0.layer_norm2.weight', 'vision_model.vision_model.encoder.layers.0.mlp.fc1.bias', 'vision_model.vision_model.encoder.layers.0.mlp.fc1.weight', 'vision_model.vision_model.encoder.layers.0.mlp.fc2.bias', 'vision_model.vision_model.encoder.layers.0.mlp.fc2.weight', 'vision_model.vision_model.encoder.layers.0.self_attn.k_proj.bias', 'vision_model.vision_model.encoder.layers.0.self_attn.k_proj.weight', 'vision_model.vision_model.encoder.layers.0.self_attn.out_proj.bias', 'vision_model.vision_model.encoder.layers.0.self_attn.out_proj.weight', 'vision_model.vision_model.encoder.layers.0.self_attn.q_proj.bias', 'vision_model.vision_model.encoder.layers.0.self_attn.q_proj.weight', 'vision_model.vision_model.encoder.layers.0.self_attn.v_proj.bias', 'vision_model.vision_model.encoder.layers.0.self_attn.v_proj.weight', 'vision_model.vision_model.encoder.layers.1.layer_norm1.bias', 'vision_model.vision_model.encoder.layers.1.layer_norm1.weight', 'vision_model.vision_model.encoder.layers.1.layer_norm2.bias', 'vision_model.vision_model.encoder.layers.1.layer_norm2.weight', 'vision_model.vision_model.encoder.layers.1.mlp.fc1.bias', 'vision_model.vision_model.encoder.layers.1.mlp.fc1.weight', 'vision_model.vision_model.encoder.layers.1.mlp.fc2.bias', 'vision_model.vision_model.encoder.layers.1.mlp.fc2.weight', 'vision_model.vision_model.encoder.layers.1.self_attn.k_proj.bias', 'vision_model.vision_model.encoder.layers.1.self_attn.k_proj.weight', 'vision_model.vision_model.encoder.layers.1.self_attn.out_proj.bias', 'vision_model.vision_model.encoder.layers.1.self_attn.out_proj.weight', 'vision_model.vision_model.encoder.layers.1.self_attn.q_proj.bias', 'vision_model.vision_model.encoder.layers.1.self_attn.q_proj.weight', 'vision_model.vision_model.encoder.layers.1.self_attn.v_proj.bias', 'vision_model.vision_model.encoder.layers.1.self_attn.v_proj.weight', 'vision_model.vision_model.encoder.layers.10.layer_norm1.bias', 'vision_model.vision_model.encoder.layers.10.layer_norm1.weight', 'vision_model.vision_model.encoder.layers.10.layer_norm2.bias', 'vision_model.vision_model.encoder.layers.10.layer_norm2.weight', 'vision_model.vision_model.encoder.layers.10.mlp.fc1.bias', 'vision_model.vision_model.encoder.layers.10.mlp.fc1.weight', 'vision_model.vision_model.encoder.layers.10.mlp.fc2.bias', 'vision_model.vision_model.encoder.layers.10.mlp.fc2.weight', 'vision_model.vision_model.encoder.layers.10.self_attn.k_proj.bias', 'vision_model.vision_model.encoder.layers.10.self_attn.k_proj.weight', 'vision_model.vision_model.encoder.layers.10.self_attn.out_proj.bias', 'vision_model.vision_model.encoder.layers.10.self_attn.out_proj.weight', 'vision_model.vision_model.encoder.layers.10.self_attn.q_proj.bias', 'vision_model.vision_model.encoder.layers.10.self_attn.q_proj.weight', 'vision_model.vision_model.encoder.layers.10.self_attn.v_proj.bias', 'vision_model.vision_model.encoder.layers.10.self_attn.v_proj.weight', 'vision_model.vision_model.encoder.layers.11.layer_norm1.bias', 'vision_model.vision_model.encoder.layers.11.layer_norm1.weight', 'vision_model.vision_model.encoder.layers.11.layer_norm2.bias', 'vision_model.vision_model.encoder.layers.11.layer_norm2.weight', 'vision_model.vision_model.encoder.layers.11.mlp.fc1.bias', 'vision_model.vision_model.encoder.layers.11.mlp.fc1.weight', 'vision_model.vision_model.encoder.layers.11.mlp.fc2.bias', 'vision_model.vision_model.encoder.layers.11.mlp.fc2.weight', 'vision_model.vision_model.encoder.layers.11.self_attn.k_proj.bias', 'vision_model.vision_model.encoder.layers.11.self_attn.k_proj.weight', 'vision_model.vision_model.encoder.layers.11.self_attn.out_proj.bias', 'vision_model.vision_model.encoder.layers.11.self_attn.out_proj.weight', 'vision_model.vision_model.encoder.layers.11.self_attn.q_proj.bias', 'vision_model.vision_model.encoder.layers.11.self_attn.q_proj.weight', 'vision_model.vision_model.encoder.layers.11.self_attn.v_proj.bias', 'vision_model.vision_model.encoder.layers.11.self_attn.v_proj.weight', 'vision_model.vision_model.encoder.layers.12.layer_norm1.bias', 'vision_model.vision_model.encoder.layers.12.layer_norm1.weight', 'vision_model.vision_model.encoder.layers.12.layer_norm2.bias', 'vision_model.vision_model.encoder.layers.12.layer_norm2.weight', 'vision_model.vision_model.encoder.layers.12.mlp.fc1.bias', 'vision_model.vision_model.encoder.layers.12.mlp.fc1.weight', 'vision_model.vision_model.encoder.layers.12.mlp.fc2.bias', 'vision_model.vision_model.encoder.layers.12.mlp.fc2.weight', 'vision_model.vision_model.encoder.layers.12.self_attn.k_proj.bias', 'vision_model.vision_model.encoder.layers.12.self_attn.k_proj.weight', 'vision_model.vision_model.encoder.layers.12.self_attn.out_proj.bias', 'vision_model.vision_model.encoder.layers.12.self_attn.out_proj.weight', 'vision_model.vision_model.encoder.layers.12.self_attn.q_proj.bias', 'vision_model.vision_model.encoder.layers.12.self_attn.q_proj.weight', 'vision_model.vision_model.encoder.layers.12.self_attn.v_proj.bias', 'vision_model.vision_model.encoder.layers.12.self_attn.v_proj.weight', 'vision_model.vision_model.encoder.layers.13.layer_norm1.bias', 'vision_model.vision_model.encoder.layers.13.layer_norm1.weight', 'vision_model.vision_model.encoder.layers.13.layer_norm2.bias', 'vision_model.vision_model.encoder.layers.13.layer_norm2.weight', 'vision_model.vision_model.encoder.layers.13.mlp.fc1.bias', 'vision_model.vision_model.encoder.layers.13.mlp.fc1.weight', 'vision_model.vision_model.encoder.layers.13.mlp.fc2.bias', 'vision_model.vision_model.encoder.layers.13.mlp.fc2.weight', 'vision_model.vision_model.encoder.layers.13.self_attn.k_proj.bias', 'vision_model.vision_model.encoder.layers.13.self_attn.k_proj.weight', 'vision_model.vision_model.encoder.layers.13.self_attn.out_proj.bias', 'vision_model.vision_model.encoder.layers.13.self_attn.out_proj.weight', 'vision_model.vision_model.encoder.layers.13.self_attn.q_proj.bias', 'vision_model.vision_model.encoder.layers.13.self_attn.q_proj.weight', 'vision_model.vision_model.encoder.layers.13.self_attn.v_proj.bias', 'vision_model.vision_model.encoder.layers.13.self_attn.v_proj.weight', 'vision_model.vision_model.encoder.layers.14.layer_norm1.bias', 'vision_model.vision_model.encoder.layers.14.layer_norm1.weight', 'vision_model.vision_model.encoder.layers.14.layer_norm2.bias', 'vision_model.vision_model.encoder.layers.14.layer_norm2.weight', 'vision_model.vision_model.encoder.layers.14.mlp.fc1.bias', 'vision_model.vision_model.encoder.layers.14.mlp.fc1.weight', 'vision_model.vision_model.encoder.layers.14.mlp.fc2.bias', 'vision_model.vision_model.encoder.layers.14.mlp.fc2.weight', 'vision_model.vision_model.encoder.layers.14.self_attn.k_proj.bias', 'vision_model.vision_model.encoder.layers.14.self_attn.k_proj.weight', 'vision_model.vision_model.encoder.layers.14.self_attn.out_proj.bias', 'vision_model.vision_model.encoder.layers.14.self_attn.out_proj.weight', 'vision_model.vision_model.encoder.layers.14.self_attn.q_proj.bias', 'vision_model.vision_model.encoder.layers.14.self_attn.q_proj.weight', 'vision_model.vision_model.encoder.layers.14.self_attn.v_proj.bias', 'vision_model.vision_model.encoder.layers.14.self_attn.v_proj.weight', 'vision_model.vision_model.encoder.layers.15.layer_norm1.bias', 'vision_model.vision_model.encoder.layers.15.layer_norm1.weight', 'vision_model.vision_model.encoder.layers.15.layer_norm2.bias', 'vision_model.vision_model.encoder.layers.15.layer_norm2.weight', 'vision_model.vision_model.encoder.layers.15.mlp.fc1.bias', 'vision_model.vision_model.encoder.layers.15.mlp.fc1.weight', 'vision_model.vision_model.encoder.layers.15.mlp.fc2.bias', 'vision_model.vision_model.encoder.layers.15.mlp.fc2.weight', 'vision_model.vision_model.encoder.layers.15.self_attn.k_proj.bias', 'vision_model.vision_model.encoder.layers.15.self_attn.k_proj.weight', 'vision_model.vision_model.encoder.layers.15.self_attn.out_proj.bias', 'vision_model.vision_model.encoder.layers.15.self_attn.out_proj.weight', 'vision_model.vision_model.encoder.layers.15.self_attn.q_proj.bias', 'vision_model.vision_model.encoder.layers.15.self_attn.q_proj.weight', 'vision_model.vision_model.encoder.layers.15.self_attn.v_proj.bias', 'vision_model.vision_model.encoder.layers.15.self_attn.v_proj.weight', 'vision_model.vision_model.encoder.layers.16.layer_norm1.bias', 'vision_model.vision_model.encoder.layers.16.layer_norm1.weight', 'vision_model.vision_model.encoder.layers.16.layer_norm2.bias', 'vision_model.vision_model.encoder.layers.16.layer_norm2.weight', 'vision_model.vision_model.encoder.layers.16.mlp.fc1.bias', 'vision_model.vision_model.encoder.layers.16.mlp.fc1.weight', 'vision_model.vision_model.encoder.layers.16.mlp.fc2.bias', 'vision_model.vision_model.encoder.layers.16.mlp.fc2.weight', 'vision_model.vision_model.encoder.layers.16.self_attn.k_proj.bias', 'vision_model.vision_model.encoder.layers.16.self_attn.k_proj.weight', 'vision_model.vision_model.encoder.layers.16.self_attn.out_proj.bias', 'vision_model.vision_model.encoder.layers.16.self_attn.out_proj.weight', 'vision_model.vision_model.encoder.layers.16.self_attn.q_proj.bias', 'vision_model.vision_model.encoder.layers.16.self_attn.q_proj.weight', 'vision_model.vision_model.encoder.layers.16.self_attn.v_proj.bias', 'vision_model.vision_model.encoder.layers.16.self_attn.v_proj.weight', 'vision_model.vision_model.encoder.layers.17.layer_norm1.bias', 'vision_model.vision_model.encoder.layers.17.layer_norm1.weight', 'vision_model.vision_model.encoder.layers.17.layer_norm2.bias', 'vision_model.vision_model.encoder.layers.17.layer_norm2.weight', 'vision_model.vision_model.encoder.layers.17.mlp.fc1.bias', 'vision_model.vision_model.encoder.layers.17.mlp.fc1.weight', 'vision_model.vision_model.encoder.layers.17.mlp.fc2.bias', 'vision_model.vision_model.encoder.layers.17.mlp.fc2.weight', 'vision_model.vision_model.encoder.layers.17.self_attn.k_proj.bias', 'vision_model.vision_model.encoder.layers.17.self_attn.k_proj.weight', 'vision_model.vision_model.encoder.layers.17.self_attn.out_proj.bias', 'vision_model.vision_model.encoder.layers.17.self_attn.out_proj.weight', 'vision_model.vision_model.encoder.layers.17.self_attn.q_proj.bias', 'vision_model.vision_model.encoder.layers.17.self_attn.q_proj.weight', 'vision_model.vision_model.encoder.layers.17.self_attn.v_proj.bias', 'vision_model.vision_model.encoder.layers.17.self_attn.v_proj.weight', 'vision_model.vision_model.encoder.layers.18.layer_norm1.bias', 'vision_model.vision_model.encoder.layers.18.layer_norm1.weight', 'vision_model.vision_model.encoder.layers.18.layer_norm2.bias', 'vision_model.vision_model.encoder.layers.18.layer_norm2.weight', 'vision_model.vision_model.encoder.layers.18.mlp.fc1.bias', 'vision_model.vision_model.encoder.layers.18.mlp.fc1.weight', 'vision_model.vision_model.encoder.layers.18.mlp.fc2.bias', 'vision_model.vision_model.encoder.layers.18.mlp.fc2.weight', 'vision_model.vision_model.encoder.layers.18.self_attn.k_proj.bias', 'vision_model.vision_model.encoder.layers.18.self_attn.k_proj.weight', 'vision_model.vision_model.encoder.layers.18.self_attn.out_proj.bias', 'vision_model.vision_model.encoder.layers.18.self_attn.out_proj.weight', 'vision_model.vision_model.encoder.layers.18.self_attn.q_proj.bias', 'vision_model.vision_model.encoder.layers.18.self_attn.q_proj.weight', 'vision_model.vision_model.encoder.layers.18.self_attn.v_proj.bias', 'vision_model.vision_model.encoder.layers.18.self_attn.v_proj.weight', 'vision_model.vision_model.encoder.layers.19.layer_norm1.bias', 'vision_model.vision_model.encoder.layers.19.layer_norm1.weight', 'vision_model.vision_model.encoder.layers.19.layer_norm2.bias', 'vision_model.vision_model.encoder.layers.19.layer_norm2.weight', 'vision_model.vision_model.encoder.layers.19.mlp.fc1.bias', 'vision_model.vision_model.encoder.layers.19.mlp.fc1.weight', 'vision_model.vision_model.encoder.layers.19.mlp.fc2.bias', 'vision_model.vision_model.encoder.layers.19.mlp.fc2.weight', 'vision_model.vision_model.encoder.layers.19.self_attn.k_proj.bias', 'vision_model.vision_model.encoder.layers.19.self_attn.k_proj.weight', 'vision_model.vision_model.encoder.layers.19.self_attn.out_proj.bias', 'vision_model.vision_model.encoder.layers.19.self_attn.out_proj.weight', 'vision_model.vision_model.encoder.layers.19.self_attn.q_proj.bias', 'vision_model.vision_model.encoder.layers.19.self_attn.q_proj.weight', 'vision_model.vision_model.encoder.layers.19.self_attn.v_proj.bias', 'vision_model.vision_model.encoder.layers.19.self_attn.v_proj.weight', 'vision_model.vision_model.encoder.layers.2.layer_norm1.bias', 'vision_model.vision_model.encoder.layers.2.layer_norm1.weight', 'vision_model.vision_model.encoder.layers.2.layer_norm2.bias', 'vision_model.vision_model.encoder.layers.2.layer_norm2.weight', 'vision_model.vision_model.encoder.layers.2.mlp.fc1.bias', 'vision_model.vision_model.encoder.layers.2.mlp.fc1.weight', 'vision_model.vision_model.encoder.layers.2.mlp.fc2.bias', 'vision_model.vision_model.encoder.layers.2.mlp.fc2.weight', 'vision_model.vision_model.encoder.layers.2.self_attn.k_proj.bias', 'vision_model.vision_model.encoder.layers.2.self_attn.k_proj.weight', 'vision_model.vision_model.encoder.layers.2.self_attn.out_proj.bias', 'vision_model.vision_model.encoder.layers.2.self_attn.out_proj.weight', 'vision_model.vision_model.encoder.layers.2.self_attn.q_proj.bias', 'vision_model.vision_model.encoder.layers.2.self_attn.q_proj.weight', 'vision_model.vision_model.encoder.layers.2.self_attn.v_proj.bias', 'vision_model.vision_model.encoder.layers.2.self_attn.v_proj.weight', 'vision_model.vision_model.encoder.layers.20.layer_norm1.bias', 'vision_model.vision_model.encoder.layers.20.layer_norm1.weight', 'vision_model.vision_model.encoder.layers.20.layer_norm2.bias', 'vision_model.vision_model.encoder.layers.20.layer_norm2.weight', 'vision_model.vision_model.encoder.layers.20.mlp.fc1.bias', 'vision_model.vision_model.encoder.layers.20.mlp.fc1.weight', 'vision_model.vision_model.encoder.layers.20.mlp.fc2.bias', 'vision_model.vision_model.encoder.layers.20.mlp.fc2.weight', 'vision_model.vision_model.encoder.layers.20.self_attn.k_proj.bias', 'vision_model.vision_model.encoder.layers.20.self_attn.k_proj.weight', 'vision_model.vision_model.encoder.layers.20.self_attn.out_proj.bias', 'vision_model.vision_model.encoder.layers.20.self_attn.out_proj.weight', 'vision_model.vision_model.encoder.layers.20.self_attn.q_proj.bias', 'vision_model.vision_model.encoder.layers.20.self_attn.q_proj.weight', 'vision_model.vision_model.encoder.layers.20.self_attn.v_proj.bias', 'vision_model.vision_model.encoder.layers.20.self_attn.v_proj.weight', 'vision_model.vision_model.encoder.layers.21.layer_norm1.bias', 'vision_model.vision_model.encoder.layers.21.layer_norm1.weight', 'vision_model.vision_model.encoder.layers.21.layer_norm2.bias', 'vision_model.vision_model.encoder.layers.21.layer_norm2.weight', 'vision_model.vision_model.encoder.layers.21.mlp.fc1.bias', 'vision_model.vision_model.encoder.layers.21.mlp.fc1.weight', 'vision_model.vision_model.encoder.layers.21.mlp.fc2.bias', 'vision_model.vision_model.encoder.layers.21.mlp.fc2.weight', 'vision_model.vision_model.encoder.layers.21.self_attn.k_proj.bias', 'vision_model.vision_model.encoder.layers.21.self_attn.k_proj.weight', 'vision_model.vision_model.encoder.layers.21.self_attn.out_proj.bias', 'vision_model.vision_model.encoder.layers.21.self_attn.out_proj.weight', 'vision_model.vision_model.encoder.layers.21.self_attn.q_proj.bias', 'vision_model.vision_model.encoder.layers.21.self_attn.q_proj.weight', 'vision_model.vision_model.encoder.layers.21.self_attn.v_proj.bias', 'vision_model.vision_model.encoder.layers.21.self_attn.v_proj.weight', 'vision_model.vision_model.encoder.layers.22.layer_norm1.bias', 'vision_model.vision_model.encoder.layers.22.layer_norm1.weight', 'vision_model.vision_model.encoder.layers.22.layer_norm2.bias', 'vision_model.vision_model.encoder.layers.22.layer_norm2.weight', 'vision_model.vision_model.encoder.layers.22.mlp.fc1.bias', 'vision_model.vision_model.encoder.layers.22.mlp.fc1.weight', 'vision_model.vision_model.encoder.layers.22.mlp.fc2.bias', 'vision_model.vision_model.encoder.layers.22.mlp.fc2.weight', 'vision_model.vision_model.encoder.layers.22.self_attn.k_proj.bias', 'vision_model.vision_model.encoder.layers.22.self_attn.k_proj.weight', 'vision_model.vision_model.encoder.layers.22.self_attn.out_proj.bias', 'vision_model.vision_model.encoder.layers.22.self_attn.out_proj.weight', 'vision_model.vision_model.encoder.layers.22.self_attn.q_proj.bias', 'vision_model.vision_model.encoder.layers.22.self_attn.q_proj.weight', 'vision_model.vision_model.encoder.layers.22.self_attn.v_proj.bias', 'vision_model.vision_model.encoder.layers.22.self_attn.v_proj.weight', 'vision_model.vision_model.encoder.layers.23.layer_norm1.bias', 'vision_model.vision_model.encoder.layers.23.layer_norm1.weight', 'vision_model.vision_model.encoder.layers.23.layer_norm2.bias', 'vision_model.vision_model.encoder.layers.23.layer_norm2.weight', 'vision_model.vision_model.encoder.layers.23.mlp.fc1.bias', 'vision_model.vision_model.encoder.layers.23.mlp.fc1.weight', 'vision_model.vision_model.encoder.layers.23.mlp.fc2.bias', 'vision_model.vision_model.encoder.layers.23.mlp.fc2.weight', 'vision_model.vision_model.encoder.layers.23.self_attn.k_proj.bias', 'vision_model.vision_model.encoder.layers.23.self_attn.k_proj.weight', 'vision_model.vision_model.encoder.layers.23.self_attn.out_proj.bias', 'vision_model.vision_model.encoder.layers.23.self_attn.out_proj.weight', 'vision_model.vision_model.encoder.layers.23.self_attn.q_proj.bias', 'vision_model.vision_model.encoder.layers.23.self_attn.q_proj.weight', 'vision_model.vision_model.encoder.layers.23.self_attn.v_proj.bias', 'vision_model.vision_model.encoder.layers.23.self_attn.v_proj.weight', 'vision_model.vision_model.encoder.layers.24.layer_norm1.bias', 'vision_model.vision_model.encoder.layers.24.layer_norm1.weight', 'vision_model.vision_model.encoder.layers.24.layer_norm2.bias', 'vision_model.vision_model.encoder.layers.24.layer_norm2.weight', 'vision_model.vision_model.encoder.layers.24.mlp.fc1.bias', 'vision_model.vision_model.encoder.layers.24.mlp.fc1.weight', 'vision_model.vision_model.encoder.layers.24.mlp.fc2.bias', 'vision_model.vision_model.encoder.layers.24.mlp.fc2.weight', 'vision_model.vision_model.encoder.layers.24.self_attn.k_proj.bias', 'vision_model.vision_model.encoder.layers.24.self_attn.k_proj.weight', 'vision_model.vision_model.encoder.layers.24.self_attn.out_proj.bias', 'vision_model.vision_model.encoder.layers.24.self_attn.out_proj.weight', 'vision_model.vision_model.encoder.layers.24.self_attn.q_proj.bias', 'vision_model.vision_model.encoder.layers.24.self_attn.q_proj.weight', 'vision_model.vision_model.encoder.layers.24.self_attn.v_proj.bias', 'vision_model.vision_model.encoder.layers.24.self_attn.v_proj.weight', 'vision_model.vision_model.encoder.layers.25.layer_norm1.bias', 'vision_model.vision_model.encoder.layers.25.layer_norm1.weight', 'vision_model.vision_model.encoder.layers.25.layer_norm2.bias', 'vision_model.vision_model.encoder.layers.25.layer_norm2.weight', 'vision_model.vision_model.encoder.layers.25.mlp.fc1.bias', 'vision_model.vision_model.encoder.layers.25.mlp.fc1.weight', 'vision_model.vision_model.encoder.layers.25.mlp.fc2.bias', 'vision_model.vision_model.encoder.layers.25.mlp.fc2.weight', 'vision_model.vision_model.encoder.layers.25.self_attn.k_proj.bias', 'vision_model.vision_model.encoder.layers.25.self_attn.k_proj.weight', 'vision_model.vision_model.encoder.layers.25.self_attn.out_proj.bias', 'vision_model.vision_model.encoder.layers.25.self_attn.out_proj.weight', 'vision_model.vision_model.encoder.layers.25.self_attn.q_proj.bias', 'vision_model.vision_model.encoder.layers.25.self_attn.q_proj.weight', 'vision_model.vision_model.encoder.layers.25.self_attn.v_proj.bias', 'vision_model.vision_model.encoder.layers.25.self_attn.v_proj.weight', 'vision_model.vision_model.encoder.layers.26.layer_norm1.bias', 'vision_model.vision_model.encoder.layers.26.layer_norm1.weight', 'vision_model.vision_model.encoder.layers.26.layer_norm2.bias', 'vision_model.vision_model.encoder.layers.26.layer_norm2.weight', 'vision_model.vision_model.encoder.layers.26.mlp.fc1.bias', 'vision_model.vision_model.encoder.layers.26.mlp.fc1.weight', 'vision_model.vision_model.encoder.layers.26.mlp.fc2.bias', 'vision_model.vision_model.encoder.layers.26.mlp.fc2.weight', 'vision_model.vision_model.encoder.layers.26.self_attn.k_proj.bias', 'vision_model.vision_model.encoder.layers.26.self_attn.k_proj.weight', 'vision_model.vision_model.encoder.layers.26.self_attn.out_proj.bias', 'vision_model.vision_model.encoder.layers.26.self_attn.out_proj.weight', 'vision_model.vision_model.encoder.layers.26.self_attn.q_proj.bias', 'vision_model.vision_model.encoder.layers.26.self_attn.q_proj.weight', 'vision_model.vision_model.encoder.layers.26.self_attn.v_proj.bias', 'vision_model.vision_model.encoder.layers.26.self_attn.v_proj.weight', 'vision_model.vision_model.encoder.layers.3.layer_norm1.bias', 'vision_model.vision_model.encoder.layers.3.layer_norm1.weight', 'vision_model.vision_model.encoder.layers.3.layer_norm2.bias', 'vision_model.vision_model.encoder.layers.3.layer_norm2.weight', 'vision_model.vision_model.encoder.layers.3.mlp.fc1.bias', 'vision_model.vision_model.encoder.layers.3.mlp.fc1.weight', 'vision_model.vision_model.encoder.layers.3.mlp.fc2.bias', 'vision_model.vision_model.encoder.layers.3.mlp.fc2.weight', 'vision_model.vision_model.encoder.layers.3.self_attn.k_proj.bias', 'vision_model.vision_model.encoder.layers.3.self_attn.k_proj.weight', 'vision_model.vision_model.encoder.layers.3.self_attn.out_proj.bias', 'vision_model.vision_model.encoder.layers.3.self_attn.out_proj.weight', 'vision_model.vision_model.encoder.layers.3.self_attn.q_proj.bias', 'vision_model.vision_model.encoder.layers.3.self_attn.q_proj.weight', 'vision_model.vision_model.encoder.layers.3.self_attn.v_proj.bias', 'vision_model.vision_model.encoder.layers.3.self_attn.v_proj.weight', 'vision_model.vision_model.encoder.layers.4.layer_norm1.bias', 'vision_model.vision_model.encoder.layers.4.layer_norm1.weight', 'vision_model.vision_model.encoder.layers.4.layer_norm2.bias', 'vision_model.vision_model.encoder.layers.4.layer_norm2.weight', 'vision_model.vision_model.encoder.layers.4.mlp.fc1.bias', 'vision_model.vision_model.encoder.layers.4.mlp.fc1.weight', 'vision_model.vision_model.encoder.layers.4.mlp.fc2.bias', 'vision_model.vision_model.encoder.layers.4.mlp.fc2.weight', 'vision_model.vision_model.encoder.layers.4.self_attn.k_proj.bias', 'vision_model.vision_model.encoder.layers.4.self_attn.k_proj.weight', 'vision_model.vision_model.encoder.layers.4.self_attn.out_proj.bias', 'vision_model.vision_model.encoder.layers.4.self_attn.out_proj.weight', 'vision_model.vision_model.encoder.layers.4.self_attn.q_proj.bias', 'vision_model.vision_model.encoder.layers.4.self_attn.q_proj.weight', 'vision_model.vision_model.encoder.layers.4.self_attn.v_proj.bias', 'vision_model.vision_model.encoder.layers.4.self_attn.v_proj.weight', 'vision_model.vision_model.encoder.layers.5.layer_norm1.bias', 'vision_model.vision_model.encoder.layers.5.layer_norm1.weight', 'vision_model.vision_model.encoder.layers.5.layer_norm2.bias', 'vision_model.vision_model.encoder.layers.5.layer_norm2.weight', 'vision_model.vision_model.encoder.layers.5.mlp.fc1.bias', 'vision_model.vision_model.encoder.layers.5.mlp.fc1.weight', 'vision_model.vision_model.encoder.layers.5.mlp.fc2.bias', 'vision_model.vision_model.encoder.layers.5.mlp.fc2.weight', 'vision_model.vision_model.encoder.layers.5.self_attn.k_proj.bias', 'vision_model.vision_model.encoder.layers.5.self_attn.k_proj.weight', 'vision_model.vision_model.encoder.layers.5.self_attn.out_proj.bias', 'vision_model.vision_model.encoder.layers.5.self_attn.out_proj.weight', 'vision_model.vision_model.encoder.layers.5.self_attn.q_proj.bias', 'vision_model.vision_model.encoder.layers.5.self_attn.q_proj.weight', 'vision_model.vision_model.encoder.layers.5.self_attn.v_proj.bias', 'vision_model.vision_model.encoder.layers.5.self_attn.v_proj.weight', 'vision_model.vision_model.encoder.layers.6.layer_norm1.bias', 'vision_model.vision_model.encoder.layers.6.layer_norm1.weight', 'vision_model.vision_model.encoder.layers.6.layer_norm2.bias', 'vision_model.vision_model.encoder.layers.6.layer_norm2.weight', 'vision_model.vision_model.encoder.layers.6.mlp.fc1.bias', 'vision_model.vision_model.encoder.layers.6.mlp.fc1.weight', 'vision_model.vision_model.encoder.layers.6.mlp.fc2.bias', 'vision_model.vision_model.encoder.layers.6.mlp.fc2.weight', 'vision_model.vision_model.encoder.layers.6.self_attn.k_proj.bias', 'vision_model.vision_model.encoder.layers.6.self_attn.k_proj.weight', 'vision_model.vision_model.encoder.layers.6.self_attn.out_proj.bias', 'vision_model.vision_model.encoder.layers.6.self_attn.out_proj.weight', 'vision_model.vision_model.encoder.layers.6.self_attn.q_proj.bias', 'vision_model.vision_model.encoder.layers.6.self_attn.q_proj.weight', 'vision_model.vision_model.encoder.layers.6.self_attn.v_proj.bias', 'vision_model.vision_model.encoder.layers.6.self_attn.v_proj.weight', 'vision_model.vision_model.encoder.layers.7.layer_norm1.bias', 'vision_model.vision_model.encoder.layers.7.layer_norm1.weight', 'vision_model.vision_model.encoder.layers.7.layer_norm2.bias', 'vision_model.vision_model.encoder.layers.7.layer_norm2.weight', 'vision_model.vision_model.encoder.layers.7.mlp.fc1.bias', 'vision_model.vision_model.encoder.layers.7.mlp.fc1.weight', 'vision_model.vision_model.encoder.layers.7.mlp.fc2.bias', 'vision_model.vision_model.encoder.layers.7.mlp.fc2.weight', 'vision_model.vision_model.encoder.layers.7.self_attn.k_proj.bias', 'vision_model.vision_model.encoder.layers.7.self_attn.k_proj.weight', 'vision_model.vision_model.encoder.layers.7.self_attn.out_proj.bias', 'vision_model.vision_model.encoder.layers.7.self_attn.out_proj.weight', 'vision_model.vision_model.encoder.layers.7.self_attn.q_proj.bias', 'vision_model.vision_model.encoder.layers.7.self_attn.q_proj.weight', 'vision_model.vision_model.encoder.layers.7.self_attn.v_proj.bias', 'vision_model.vision_model.encoder.layers.7.self_attn.v_proj.weight', 'vision_model.vision_model.encoder.layers.8.layer_norm1.bias', 'vision_model.vision_model.encoder.layers.8.layer_norm1.weight', 'vision_model.vision_model.encoder.layers.8.layer_norm2.bias', 'vision_model.vision_model.encoder.layers.8.layer_norm2.weight', 'vision_model.vision_model.encoder.layers.8.mlp.fc1.bias', 'vision_model.vision_model.encoder.layers.8.mlp.fc1.weight', 'vision_model.vision_model.encoder.layers.8.mlp.fc2.bias', 'vision_model.vision_model.encoder.layers.8.mlp.fc2.weight', 'vision_model.vision_model.encoder.layers.8.self_attn.k_proj.bias', 'vision_model.vision_model.encoder.layers.8.self_attn.k_proj.weight', 'vision_model.vision_model.encoder.layers.8.self_attn.out_proj.bias', 'vision_model.vision_model.encoder.layers.8.self_attn.out_proj.weight', 'vision_model.vision_model.encoder.layers.8.self_attn.q_proj.bias', 'vision_model.vision_model.encoder.layers.8.self_attn.q_proj.weight', 'vision_model.vision_model.encoder.layers.8.self_attn.v_proj.bias', 'vision_model.vision_model.encoder.layers.8.self_attn.v_proj.weight', 'vision_model.vision_model.encoder.layers.9.layer_norm1.bias', 'vision_model.vision_model.encoder.layers.9.layer_norm1.weight', 'vision_model.vision_model.encoder.layers.9.layer_norm2.bias', 'vision_model.vision_model.encoder.layers.9.layer_norm2.weight', 'vision_model.vision_model.encoder.layers.9.mlp.fc1.bias', 'vision_model.vision_model.encoder.layers.9.mlp.fc1.weight', 'vision_model.vision_model.encoder.layers.9.mlp.fc2.bias', 'vision_model.vision_model.encoder.layers.9.mlp.fc2.weight', 'vision_model.vision_model.encoder.layers.9.self_attn.k_proj.bias', 'vision_model.vision_model.encoder.layers.9.self_attn.k_proj.weight', 'vision_model.vision_model.encoder.layers.9.self_attn.out_proj.bias', 'vision_model.vision_model.encoder.layers.9.self_attn.out_proj.weight', 'vision_model.vision_model.encoder.layers.9.self_attn.q_proj.bias', 'vision_model.vision_model.encoder.layers.9.self_attn.q_proj.weight', 'vision_model.vision_model.encoder.layers.9.self_attn.v_proj.bias', 'vision_model.vision_model.encoder.layers.9.self_attn.v_proj.weight', 'vision_model.vision_model.head.attention.in_proj_bias', 'vision_model.vision_model.head.attention.in_proj_weight', 'vision_model.vision_model.head.attention.out_proj.bias', 'vision_model.vision_model.head.attention.out_proj.weight', 'vision_model.vision_model.head.layernorm.bias', 'vision_model.vision_model.head.layernorm.weight', 'vision_model.vision_model.head.mlp.fc1.bias', 'vision_model.vision_model.head.mlp.fc1.weight', 'vision_model.vision_model.head.mlp.fc2.bias', 'vision_model.vision_model.head.mlp.fc2.weight', 'vision_model.vision_model.head.probe', 'vision_model.vision_model.post_layernorm.bias', 'vision_model.vision_model.post_layernorm.weight']
- This IS expected if you are initializing JasperVL from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing JasperVL from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
INFO:mteb.evaluation.MTEB:

## Evaluating 1 tasks:
─────────────────────────────── Selected tasks  ────────────────────────────────
Classification
    - MassiveScenarioClassification, s2s, multilingual 51 / 51 Subsets


INFO:mteb.evaluation.MTEB:

********************** Evaluating MassiveScenarioClassification **********************
Loading Dataset Infos from /env/lib/conda/gritkto/lib/python3.10/site-packages/datasets/packaged_modules/json
INFO:datasets.info:Loading Dataset Infos from /env/lib/conda/gritkto/lib/python3.10/site-packages/datasets/packaged_modules/json
Overwrite dataset info from restored data version if exists.
INFO:datasets.builder:Overwrite dataset info from restored data version if exists.
Loading Dataset info from /data/huggingface/datasets/mteb___amazon_massive_scenario/default/0.0.0/fad2c6e8459f9e1c45d9315f4953d921437d70f8
INFO:datasets.info:Loading Dataset info from /data/huggingface/datasets/mteb___amazon_massive_scenario/default/0.0.0/fad2c6e8459f9e1c45d9315f4953d921437d70f8
Found cached dataset amazon_massive_scenario (/data/huggingface/datasets/mteb___amazon_massive_scenario/default/0.0.0/fad2c6e8459f9e1c45d9315f4953d921437d70f8)
INFO:datasets.builder:Found cached dataset amazon_massive_scenario (/data/huggingface/datasets/mteb___amazon_massive_scenario/default/0.0.0/fad2c6e8459f9e1c45d9315f4953d921437d70f8)
Loading Dataset info from /data/huggingface/datasets/mteb___amazon_massive_scenario/default/0.0.0/fad2c6e8459f9e1c45d9315f4953d921437d70f8
INFO:datasets.info:Loading Dataset info from /data/huggingface/datasets/mteb___amazon_massive_scenario/default/0.0.0/fad2c6e8459f9e1c45d9315f4953d921437d70f8
INFO:mteb.abstasks.AbsTaskClassification:
Task: MassiveScenarioClassification, split: validation, subset: sq. Running...
INFO:mteb.abstasks.AbsTaskClassification:========== Experiment 1/10 ==========
INFO:mteb.evaluation.evaluators.ClassificationEvaluator:Fitting logistic regression classifier...
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
	- Avoid using `tokenizers` before the fork if possible
	- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
INFO:mteb.evaluation.evaluators.ClassificationEvaluator:Evaluating...
INFO:mteb.abstasks.AbsTaskClassification:========== Experiment 2/10 ==========
INFO:mteb.evaluation.evaluators.ClassificationEvaluator:Fitting logistic regression classifier...
INFO:mteb.evaluation.evaluators.ClassificationEvaluator:Evaluating...
INFO:mteb.abstasks.AbsTaskClassification:========== Experiment 3/10 ==========
INFO:mteb.evaluation.evaluators.ClassificationEvaluator:Fitting logistic regression classifier...
INFO:mteb.evaluation.evaluators.ClassificationEvaluator:Evaluating...
INFO:mteb.abstasks.AbsTaskClassification:========== Experiment 4/10 ==========
INFO:mteb.evaluation.evaluators.ClassificationEvaluator:Fitting logistic regression classifier...
INFO:mteb.evaluation.evaluators.ClassificationEvaluator:Evaluating...
INFO:mteb.abstasks.AbsTaskClassification:========== Experiment 5/10 ==========
INFO:mteb.evaluation.evaluators.ClassificationEvaluator:Fitting logistic regression classifier...
INFO:mteb.evaluation.evaluators.ClassificationEvaluator:Evaluating...
INFO:mteb.abstasks.AbsTaskClassification:========== Experiment 6/10 ==========
INFO:mteb.evaluation.evaluators.ClassificationEvaluator:Fitting logistic regression classifier...
INFO:mteb.evaluation.evaluators.ClassificationEvaluator:Evaluating...
INFO:mteb.abstasks.AbsTaskClassification:========== Experiment 7/10 ==========
INFO:mteb.evaluation.evaluators.ClassificationEvaluator:Fitting logistic regression classifier...
INFO:mteb.evaluation.evaluators.ClassificationEvaluator:Evaluating...
INFO:mteb.abstasks.AbsTaskClassification:========== Experiment 8/10 ==========
INFO:mteb.evaluation.evaluators.ClassificationEvaluator:Fitting logistic regression classifier...
INFO:mteb.evaluation.evaluators.ClassificationEvaluator:Evaluating...
INFO:mteb.abstasks.AbsTaskClassification:========== Experiment 9/10 ==========
INFO:mteb.evaluation.evaluators.ClassificationEvaluator:Fitting logistic regression classifier...
INFO:mteb.evaluation.evaluators.ClassificationEvaluator:Evaluating...
INFO:mteb.abstasks.AbsTaskClassification:========== Experiment 10/10 ==========
INFO:mteb.evaluation.evaluators.ClassificationEvaluator:Fitting logistic regression classifier...
INFO:mteb.evaluation.evaluators.ClassificationEvaluator:Evaluating...
INFO:mteb.abstasks.AbsTaskClassification:
Task: MassiveScenarioClassification, split: validation, subset: pt. Running...
INFO:mteb.abstasks.AbsTaskClassification:========== Experiment 1/10 ==========
INFO:mteb.evaluation.evaluators.ClassificationEvaluator:Fitting logistic regression classifier...
INFO:mteb.evaluation.evaluators.ClassificationEvaluator:Evaluating...
INFO:mteb.abstasks.AbsTaskClassification:========== Experiment 2/10 ==========
INFO:mteb.evaluation.evaluators.ClassificationEvaluator:Fitting logistic regression classifier...
INFO:mteb.evaluation.evaluators.ClassificationEvaluator:Evaluating...
INFO:mteb.abstasks.AbsTaskClassification:========== Experiment 3/10 ==========
INFO:mteb.evaluation.evaluators.ClassificationEvaluator:Fitting logistic regression classifier...
INFO:mteb.evaluation.evaluators.ClassificationEvaluator:Evaluating...
INFO:mteb.abstasks.AbsTaskClassification:========== Experiment 4/10 ==========
INFO:mteb.evaluation.evaluators.ClassificationEvaluator:Fitting logistic regression classifier...
INFO:mteb.evaluation.evaluators.ClassificationEvaluator:Evaluating...
INFO:mteb.abstasks.AbsTaskClassification:========== Experiment 5/10 ==========
INFO:mteb.evaluation.evaluators.ClassificationEvaluator:Fitting logistic regression classifier...
INFO:mteb.evaluation.evaluators.ClassificationEvaluator:Evaluating...
INFO:mteb.abstasks.AbsTaskClassification:========== Experiment 6/10 ==========
INFO:mteb.evaluation.evaluators.ClassificationEvaluator:Fitting logistic regression classifier...
INFO:mteb.evaluation.evaluators.ClassificationEvaluator:Evaluating...
INFO:mteb.abstasks.AbsTaskClassification:========== Experiment 7/10 ==========
INFO:mteb.evaluation.evaluators.ClassificationEvaluator:Fitting logistic regression classifier...
INFO:mteb.evaluation.evaluators.ClassificationEvaluator:Evaluating...
INFO:mteb.abstasks.AbsTaskClassification:========== Experiment 8/10 ==========
INFO:mteb.evaluation.evaluators.ClassificationEvaluator:Fitting logistic regression classifier...
INFO:mteb.evaluation.evaluators.ClassificationEvaluator:Evaluating...
INFO:mteb.abstasks.AbsTaskClassification:========== Experiment 9/10 ==========
INFO:mteb.evaluation.evaluators.ClassificationEvaluator:Fitting logistic regression classifier...
INFO:mteb.evaluation.evaluators.ClassificationEvaluator:Evaluating...
INFO:mteb.abstasks.AbsTaskClassification:========== Experiment 10/10 ==========
INFO:mteb.evaluation.evaluators.ClassificationEvaluator:Fitting logistic regression classifier...
INFO:mteb.evaluation.evaluators.ClassificationEvaluator:Evaluating...
INFO:mteb.abstasks.AbsTaskClassification:
Task: MassiveScenarioClassification, split: validation, subset: ro. Running...
INFO:mteb.abstasks.AbsTaskClassification:========== Experiment 1/10 ==========
INFO:mteb.evaluation.evaluators.ClassificationEvaluator:Fitting logistic regression classifier...
INFO:mteb.evaluation.evaluators.ClassificationEvaluator:Evaluating...
ERROR:mteb.evaluation.MTEB:Error while evaluating MassiveScenarioClassification: Input X contains NaN.
LogisticRegression does not accept missing values encoded as NaN natively. For supervised learning, you might want to consider sklearn.ensemble.HistGradientBoostingClassifier and Regressor which accept missing values encoded as NaNs natively. Alternatively, it is possible to preprocess the data, for instance by using an imputer transformer in a pipeline or drop samples with missing values. See https://scikit-learn.org/stable/modules/impute.html You can find a list of all estimators that handle NaN values at the following page: https://scikit-learn.org/stable/modules/impute.html#estimators-that-handle-nan-values
Traceback (most recent call last):
  File "/env/lib/conda/gritkto/bin/mteb", line 8, in <module>
    sys.exit(main())
  File "/data/niklas/mteb/mteb/cli.py", line 387, in main
    args.func(args)
  File "/data/niklas/mteb/mteb/cli.py", line 145, in run
    eval.run(
  File "/data/niklas/mteb/mteb/evaluation/MTEB.py", line 623, in run
    raise e
  File "/data/niklas/mteb/mteb/evaluation/MTEB.py", line 562, in run
    results, tick, tock = self._run_eval(
  File "/data/niklas/mteb/mteb/evaluation/MTEB.py", line 304, in _run_eval
    results = task.evaluate(
  File "/data/niklas/mteb/mteb/abstasks/AbsTaskClassification.py", line 120, in evaluate
    scores[hf_subset] = self._evaluate_subset(
  File "/data/niklas/mteb/mteb/abstasks/AbsTaskClassification.py", line 196, in _evaluate_subset
    scores_exp, test_cache = evaluator(model, test_cache=test_cache)
  File "/data/niklas/mteb/mteb/evaluation/evaluators/ClassificationEvaluator.py", line 308, in __call__
    y_pred = clf.predict(X_test)
  File "/env/lib/conda/gritkto/lib/python3.10/site-packages/sklearn/linear_model/_base.py", line 351, in predict
    scores = self.decision_function(X)
  File "/env/lib/conda/gritkto/lib/python3.10/site-packages/sklearn/linear_model/_base.py", line 332, in decision_function
    X = self._validate_data(X, accept_sparse="csr", reset=False)
  File "/env/lib/conda/gritkto/lib/python3.10/site-packages/sklearn/base.py", line 633, in _validate_data
    out = check_array(X, input_name="X", **check_params)
  File "/env/lib/conda/gritkto/lib/python3.10/site-packages/sklearn/utils/validation.py", line 1049, in check_array
    _assert_all_finite(
  File "/env/lib/conda/gritkto/lib/python3.10/site-packages/sklearn/utils/validation.py", line 126, in _assert_all_finite
    _assert_all_finite_element_wise(
  File "/env/lib/conda/gritkto/lib/python3.10/site-packages/sklearn/utils/validation.py", line 175, in _assert_all_finite_element_wise
    raise ValueError(msg_err)
ValueError: Input X contains NaN.
LogisticRegression does not accept missing values encoded as NaN natively. For supervised learning, you might want to consider sklearn.ensemble.HistGradientBoostingClassifier and Regressor which accept missing values encoded as NaNs natively. Alternatively, it is possible to preprocess the data, for instance by using an imputer transformer in a pipeline or drop samples with missing values. See https://scikit-learn.org/stable/modules/impute.html You can find a list of all estimators that handle NaN values at the following page: https://scikit-learn.org/stable/modules/impute.html#estimators-that-handle-nan-values
/var/spool/slurmd/job810588/slurm_script: line 9: /data/niklas/mteb/failures/infgrad/jasper_en_vision_language_v1_MassiveScenarioClassification.txt: No such file or directory
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@Muennighoff