Teams often say they want strict document-grounded answers, then allow model priors to fill gaps. That is where hallucinations return. We rebuilt our retrieval stack with one hard rule: if support is missing in the corpus, the assistant must abstain and ask for clarification.
Guardrails that actually matter
First, constrain generation to quoted or tightly summarized retrieved chunks. Second, require citation IDs in the final answer object. Third, enforce confidence thresholds that block free-form responses when evidence is thin.
We also added an unsupported-claim detector that scans draft responses for entities and numbers not present in the retrieved passages. If it finds any, the answer is rewritten or refused.
Prompting pattern for strict mode
- Instruct the model to answer only from provided context.
- Require explicit citation markers for each claim.
- Define a mandatory fallback sentence when citations are insufficient.
- Block speculative language in post-processing checks.
Results after hardening
In offline evaluation, unsupported claims dropped sharply and refusal quality improved. Users accepted abstentions when the system explained what source was missing. That is a better experience than confident incorrect answers.
Recommendation: treat retrieval fidelity as a product requirement, not a prompt trick. If trust matters, refusal behavior and citation rigor are core features, not optional polish.