Need some help on my simple RAG setup that won't rag.

[woezel1980]

Hi, regardless of watching a few tutorials on youtube and using chatGPT I am still stuck.

My goal: create a RAG. It consists of a document store in Flowise, which uses nomic-embed-text to embed. I uploaded a simple text file with (translated from Dutch) the sentence: "Pancakes are blue, peanut butter is green". I succesfully upserted to my self hosted Postgres DB. I can see the entry there.

This is my Chat flow (see screenshot).

The problem is that the agent does not seem to "see" the text in the text file. My expectation would be that if I ask what color pancakes are, I get the answer "blue". But instead I get a general answer.

Who knows what's wrong?

[onestardao]

looks like your flow is wired but the retriever isn’t actually surfacing the embedded text back to the llm. this matches ProblemMap No.8 (retriever mismatch) — basically the query is bypassing the vector search and falling back to a generic completion.

the fix isn’t infra, it’s a semantic alignment step. if you want, i can share the checklist we use that ensures the retriever and embedding are bound correctly (it’s open in our problem map).

let me know if you want the link.

[woezel1980]

looks like your flow is wired but the retriever isn’t actually surfacing the embedded text back to the llm. this matches ProblemMap No.8 (retriever mismatch) — basically the query is bypassing the vector search and falling back to a generic completion.

the fix isn’t infra, it’s a semantic alignment step. if you want, i can share the checklist we use that ensures the retriever and embedding are bound correctly (it’s open in our problem map).

let me know if you want the link.

Hi onestardao, ( @onestardao )
Thanks for your reply.
I think you are describing the problem correctly as I too think the flow is correct. So there must be an issue with the retrieval and/or the matching with the LLM. This seems weird as both the embedding model (nomic-embed-text) as the llm (mistral) are both running via ollama and part of the same stack.

A semantic alignment step? Sounds complicated but I would like to know how to check this, as my problem still is not fixed. You are mentioning a link. Can I see it please?

[woezel1980]

Do you mean
https://github.com/onestardao/WFGY/blob/main/ProblemMap/README.md
?
Currently analyzing it.

[onestardao]

Yes, you have found my secret :P , you can use AI and ask "use wfgy from TXTOS to solve your Q" (also screenshot the 16 common problems map) I designed this for AI so every smart AI can understand my framework ^_^

[woezel1980]

ok thanks. I didn't find the solution using the tool agent and ended up using the conversational retrieval QA chain.

[onestardao]

perfect you already found the map. from your screenshot and the “general answer” symptom, this is almost always two things mixed:

• No.8 retriever mismatch. the query never truly binds to the retriever output. the graph falls back to a generic completion.
• No.5 semantic ≠ embedding. your store metric or chunk contract is off, so the right sentence never makes top-k.

here is a short sanity checklist you can run inside Flowise. it is fast and tells you where it breaks.

60-sec triage

1. Probe the retriever directly
   Call the retriever node with the exact query you asked the agent. Print ids and the first 200 chars.
   Expect top1 to contain “pancakes are blue”. If not, go to steps 2 and 3.

2. Metric alignment

   • nomic-embed-text is cosine oriented. Your vector store must be cosine too.
   • If it was L2 or dot, rebuild the index. Do not just flip a flag.

3. Chunk → embedding contract

   • Make sure the full fact is inside a single chunk.
   • Try chunk size 300–500, overlap 50–80.
   • Do not index titles only. No HTML boilerplate. Dedup near duplicates.

4. Guard the handoff

   • Add a tiny wrapper before the LLM: if retriever top1 is empty or weak, refuse to answer and surface the snippet.
   • Quick rule of thumb: if top1 similarity is below your store’s “good” range, do not let the LLM synthesize.
   • In WFGY we use ΔS ≤ 0.45 as the acceptance target. As a quick proxy you can use a cosine threshold around 0.82 and rerank top5.

5. If you go back to tool-agent

   • Ensure the agent sends a JSON object, not a plain string.

   • Schema like

     { "query": "pancakes color?", "filters": { "topic": "recipe" } }

     The retriever should receive exactly that object. No nested toolName. No extra wrapper text.

6. Force citations before prose
   Even in conversational QA, add one step: commit citations first, then plan, then answer. This alone kills a lot of “general answer” drift.

If you want the single page with the numbers and acceptance targets, it is here
Problem Map → https://github.com/onestardao/WFGY/blob/main/ProblemMap/README.md

If you share one retriever call log and your store metric, I can point to the exact No.X again and give a concrete fix order. Also, if you load TXT OS and ask any model “use WFGY to solve this bug,” it will route you to the right page and generate the patch steps.