How an AI assistant looks things up before it answers

Retrieval augmented generation, or RAG, keeps an AI model honest by letting it read before it writes. Your question is turned into a vector, the closest passages in a document library are found by distance, and those passages are placed in front of the language model so its answer can cite real sources.

Controls

Labels

Animation speed

Question

Top k

Step through it

Embed the questionAn embedding model turns the question into a list of numbers that captures its meaning.
Search the vector storeEvery document chunk has its own vector. The search finds the chunks closest to the question.
Pick the top kOnly the nearest few chunks are kept. Too few misses context; too many adds noise.
Assemble the promptThe question and the retrieved chunks are combined into one prompt with instructions to cite them.
Generate the answerThe language model writes an answer grounded in those passages.

A toy space with real math

Real embeddings have hundreds or thousands of dimensions, which no screen can draw. This model squeezes a small library of example farm passages into three dimensions so you can see the idea: passages about similar things sit near each other, and retrieval is literally measuring distance and keeping the nearest ones.

Why RAG matters for business

A language model on its own only knows what it saw in training, which goes stale and never included your private documents. RAG lets a company keep its knowledge in a store it controls and update it any time, while the model handles the language. It also makes answers auditable, because each one can point back to the passages it used.

Tuning tip: slide top k from 1 to 8 while asking the same question. Watch when the extra chunks start coming from the wrong topic. Production systems add a reranking step to catch exactly that.

Where Avid uses this

AvidRAG, our nine level agentic RAG architecture on Red Hat OpenShift, is the engine behind GovPulse.io and VerdantaIQ.

Sources

Keep exploring

Energy & HVACHow a heat pump moves heat uphill AgricultureHow a spray drone steers with four propellers AgricultureInside a centrifugal irrigation pump Energy & HVACGrowing crops under solar panels