llama3.1:8b

[MyraBaba]

Hi,

How to set model to llama3.1:8b for Local Rag ?

I cant find a convenient way to do this

[WilliamEspegren]

Hey @MyraBaba, here is how you can run local Ollama models with Phidata :)

[MyraBaba]

@WilliamEspegren

(venvPhiData) redel@RedElephant:~/Projects/phidata$ python cookbook/llms/ollama/assistant.py
⠋ Working...
Traceback (most recent call last):
File "/home/redel/Projects/phidata/cookbook/llms/ollama/assistant.py", line 9, in
assistant.print_response("Share a quick healthy breakfast recipe.", markdown=True)
File "/home/redel/Projects/phidata/venvPhiData/lib/python3.10/site-packages/phi/assistant/assistant.py", line 1473, in print_response
for resp in self.run(message=message, messages=messages, stream=True, **kwargs):
File "/home/redel/Projects/phidata/venvPhiData/lib/python3.10/site-packages/phi/assistant/assistant.py", line 891, in _run
for response_chunk in self.llm.response_stream(messages=llm_messages):
File "/home/redel/Projects/phidata/venvPhiData/lib/python3.10/site-packages/phi/llm/ollama/chat.py", line 271, in response_stream
for response in self.invoke_stream(messages=messages):
File "/home/redel/Projects/phidata/venvPhiData/lib/python3.10/site-packages/phi/llm/ollama/chat.py", line 96, in invoke_stream
yield from self.client.chat(
File "/home/redel/Projects/phidata/venvPhiData/lib/python3.10/site-packages/ollama/_client.py", line 84, in _stream
raise ResponseError(e.response.text, e.response.status_code) from None
ollama._types.ResponseError: model "llama3" not found, try pulling it first

I am serving ollama at local llama3.1:8b but above errors given.

[WilliamEspegren]

@MyraBaba have you pulled it by running 'ollama run llama3.1'?

[MyraBaba]

ollama run llama3.1:8b is running

…

On 21 Aug 2024, at 21:39, William Espegren ***@***.***> wrote: @MyraBaba <https://github.com/MyraBaba> have you pulled it by running 'ollama run llama3.1'? — Reply to this email directly, view it on GitHub <#1103 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AEFRZH63AES65SGNIQPMASLZSTNFJAVCNFSM6AAAAABM3X2JLCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMBSG4ZDKNZWHE>. You are receiving this because you were mentioned.

[WilliamEspegren]

@MyraBaba can you share your code?

[reinside]

Hi,

How to set model to llama3.1:8b for Local Rag ?

I cant find a convenient way to do this

Hey,

Just edit the file, adding "llama3.1" in the models list:
cookbook/llms/ollama/rag/app.py