Deploying RAG Systems in Production: Lessons Learned

RAG fails when the corpus is a junk drawer. We only retrieve from sources someone owns, and we show the passage next to the answer.

We have put retrieval on support desks, internal wikis, and product catalogs. The work is not the model name. The work is what you allow the model to see.

Before we index

We ask who updates the files. If the answer is "a shared drive from 2019," we stop. A RAG system that quotes a retired price list will cost more than a search box.

Owner A person who can delete a stale page.

Boundary What the model must never retrieve. HR folders. Another tenant.

Cite The UI shows the source. If we cannot cite, we do not answer.

In production

We keep embeddings and the app in accounts you own. We log the query, the chunks, and the override. When retrieval is empty, the product says so. It does not invent a paragraph.

Latency matters on a phone. We do not hide a 20-second wait behind a friendly spinner. If the job can be a nightly digest instead of a live chat, we will say that on the brief.

After launch

The same people who indexed the first set stay for the first weeks. New documents go through the owner, not a silent crawl. Ongoing work is scoped separately.

This is the same rule as the rest of the studio: you own the repo, the cloud, and the corpus.