[sentence-transformers] Add sentencepiece dependency for running models with slow tokenizers only

[tomaarsen]

Hello!

Pull Request overview

• Add sentencepiece dependency for running models with slow tokenizers only

Details

See e.g. https://huggingface.co/rokn/slovlo-v1:

sentencepiece was removed as a required dependency for sentence-transformers, because most models don't require it anymore and it doesn't work well with Python 3.12 on Windows 11. Nowadays, we simply throw an error if it's required but not installed. However, the API Inference should have it installed to ensure that we can also run these models with slow tokenizers. I've verified locally that sentencepiece==0.2.0 works for e.g. that slovlo-v1 model.

• Tom Aarsen

[tomaarsen]

The same issue exists for some Camembert models, e.g.: https://huggingface.co/Photon-BR/sentence-camembert-large

• Tom Aarsen

[osanseviero]

Makes sense!