Huggingface model not working

[lucy0828]

I am trying to use OpenLLaMA for TextCat. I have already tested my code with OpenAI API and it is classifying the text very well.
However, if I run it with any models from Huggingface, the output of labels are always 0.

What might cause this issue?

[lucy0828]

This is my output

[rmitsch]

I recommend setting save_io = True in your config (in components.llm) and investigating your raw model output. OpenLLaMA 3b is a comparatively old and small model, so it's quite possible that it returns wrong/non-parseable outputs.