Context injection assembles retrieved chunks into the LLM's prompt. How retrieved content is formatted, ordered, and combined with the user's question significantly affects answer accuracy and faithfulness.
System message — Instructions for how the model should use the context: "Answer using only the information provided below. If the answer is not in the context, say so."
Context block — Retrieved chunk texts, labeled with their source (document name, URL, or chunk number). Each chunk formatted for readability.
User question — Placed at the end where the model's attention is strongest. The question the LLM must answer using the provided context.
LLMs pay more attention to content at the beginning and end of long contexts — content buried in the middle receives less attention. Place the most relevant chunks at the beginning or end of the context block, not in the middle.
System message + retrieved chunks + user question + expected answer. All must fit within the model's token limit (4K–200K depending on the model).
Reduce k (fewer chunks), reduce chunk size, or summarize chunks before injection. A common heuristic: reserve half the context for retrieved content, half for the generated response.
"Answer using only the provided context. If the context does not contain enough information to answer the question, explicitly state this. Do not make assumptions beyond what is stated."
This anchors the model to retrieved content and reduces hallucination — even when the correct answer is present in the context.
Ask the AI assistant about prompt structure, faithfulness instructions, context ordering, or managing context window limits.