RAG prompt

 

RAG prompt

What Is RAG

Retrieval-Augmented Generation means that the AI no longer relies only on its internal memory.

Instead, it works in two clear stages:

1. Retrieval — the model searches connected sources such as files, databases, or knowledge bases to find relevant pieces of information.
2. Generation — it then writes a clear, natural-language answer using those retrieved materials.

 You can use this as a system or template prompt, with placeholders filled by your app:

You are an assistant that answers questions using only the information provided in the context below.

Context:
{context}

When you answer:

• Use only the information in the context; do not rely on any outside knowledge.
• If the answer is not clearly contained in the context, say “I don’t know based on the provided documents.”
• First, briefly state the answer in 1–3 sentences.
• Then, if useful, add a short bullet list of key supporting facts drawn from the context.
• Do not mention the word “context” or refer to document IDs in your answer.

User question: {question}

Final answer:

Explanation of the components

• “Use only the information in the context” reduces hallucinations by forcing the model to ground answers in retrieved documents rather than its parametric memory.​
• Explicit fallback (“I don’t know…”) gives well‑behaved behavior when the retriever fails or the KB lacks the answer.​
• Answer + supporting bullets encourages both directness and traceability to the retrieved text, which is a common pattern in production RAG systems.​
• Hiding implementation details (no mention of “documents”, IDs, embeddings, etc.) keeps the UX clean while your backend handles retrieval and chunking.​

   Here is a concrete RAG-style prompt plus a realistic example of the model’s answer.

Full RAG prompt (Q&A with context)

System / template prompt:

You are an AI assistant that answers questions using only the information in the context provided.
If the answer is not clearly supported by the context, say:
“I don’t know based on the provided documents.”

Context:
{context}

User question:
{question}

Instructions:

• Use only the context above; ignore any outside knowledge.
• If the context is ambiguous or incomplete, say you don’t know.
• First give a direct answer in 1–3 sentences.
• Then give 2–4 bullet points citing the key supporting facts from the context, paraphrased.
• Do not mention that you are using retrieved documents or RAG.

Final answer:

Example input and output

Assume your retrieval step returned the following context and user question:

• context =
  “Document 1: The capital city of France is Paris. Paris is known for landmarks such as the Eiffel Tower and the Louvre Museum.
  Document 2: France is in Western Europe and uses the euro as its official currency.”
• question = “What is the capital of France, and where is it located geographically?”

The model’s answer (what you’d expect from this prompt):

Answer:
The capital of France is Paris, and France is located in Western Europe.
Paris is the main city and political center of the country.

Key supporting facts:

• The context states that the capital city of France is Paris.
• It also explains that France is in Western Europe.
• Paris is described as a major city with important landmarks, indicating its central national role.