accuracy

Legal review is high-stakes and precision-critical: a missed clause or hallucinated citation has real consequences, and lawyers have been sanctioned for fabricated cites. The signal is grounding in the actual documents, citing exact passages, and lawyer-in-the-loop. Here is the design.

Naive 4-bit quantization can tank quality, and the reflex to give up and serve fp16 leaves a big speedup on the table. The accuracy is usually recoverable. Here is the ladder of fixes.

FP8 serving is fast and usually fine, until it silently degrades on the one workload your benchmark did not cover. The signal is knowing what FP8 breaks and how to validate it. Here is the recipe.