bug: Local models cannot recognize uploaded files

[imtuyethan]

Jan version

0.5.8-731

Describe the Bug

I tried the entire Llama family. These models seem like they're unable to detect or process uploaded PDF files, responding as if no file was provided despite the UI showing successful file uploads.

File Type: PDF
File Size: 169.56KB

Steps to Reproduce

1. Upload a PDF file
2. Requests to "summarize this"
3. Model responds that there is no text to summarize

Screenshots / Logs

OS: macOS (Darwin Kernel Version 23.2.0)
Hardware: Apple M2
Jan Version: v0.5.8-731
Memory: 16GB Total
Cortex Version: v1.0.3-rc5

What is your OS?

• MacOS
• Windows
• Linux

[louis-jan]

As I remember, we don't support to change model mid-thread for documents uploads. There would be an enhancement where we fix an embedding model to address this issue from cortex.cpp.

[sebastian-lempert]

Same for me:

Screenshots

PDF file used

Downloaded from Wikipedia: https://en.wikipedia.org/w/index.php?title=Special:DownloadAsPdf&page=Bitcoin&action=show-download-screen

System

• OS: macOS, ProductVersion: 15.1.1, BuildVersion: 24B91
• CPU: 2,6 GHz 6-Core Intel Core i7
• GPUs (I guess none of thes should / can be used for AI):
  1. AMD Radeon Pro 5300M 4 GB
  2. Intel UHD Graphics 630 1536 MB
• Jan: v0.5.8
• Memory: 16GB Total
• Cortex: v1.0.2 (according to cortex.log)
• Model: Llama 3.1 8B Instruct Q4

Steps to reproduce

1. Start Jan
2. Install model "Llama 3.1 8B Instruct Q4"
3. Follow these steps in order to be able to chat with PDFs using RAG: https://jan.ai/docs/tools/retrieval
4. Start a new Thread
5. Select model "Llama 3.1 8B Instruct Q4"
6. Upload PDF
7. Ask "Please summarize the text in this file" --> this will take forever
8. Ask "Please summarize the text in this file" again --> this will result in an answer like "You haven't provided a file for me to access or read from. If you'd like, you can copy and paste the contents of the file into this chat window, and I'll be happy to help summarize it for you!"