faithfulness

Chain-of-thought looks like the model showing its work, but it may not reflect the real computation. The signal is knowing CoT can be a post-hoc story, with evidence, and what that breaks for safety and oversight.

Preventing hallucinations is one job; catching the ones that slip through at runtime is another. The signal is the detection toolkit, why each signal is imperfect, and how you combine them into an action.

Most RAG systems ship with no real eval, which is why most RAG systems quietly degrade. The signal is decomposing evaluation into retrieval and generation, and measuring faithfulness separately from relevance. Here is the framework that makes RAG improvements measurable.

Citations turn a RAG answer from 'trust me' into something verifiable, and they are a real engineering problem. The signal is grounding each claim to a specific passage and verifying the attribution, not appending a bag of links at the end.

Real corpora contradict themselves (old vs new policy, different teams), and naive RAG silently picks one or blends them into a wrong answer. The signal is detecting conflict and resolving it by recency, authority, and transparency.

An LLM can cite a source that does not say what the answer claims. The signal is checking claim-by-claim entailment against the cited text, not just that a citation marker exists, and knowing what to do when grounding fails.

An answer can be correct and still unfaithful, stating things the retrieved context never said. The signal is scoring faithfulness against the context, not against truth, and knowing why LLM-judge faithfulness scores drift.

Two forces decide this design: documents longer than the context window, and summaries that must not lie. The strong answer pairs hierarchical map-reduce with claim-level grounding, then serves it cheaply at scale.