regression testing

Editing a prompt is a code change with no compiler and no unit test by default, so quality regressions ship invisibly. Treating prompts and models as testable artifacts is what separates a toy from a product. Here is the harness.

A green build still means nothing for model quality. The signal is an eval harness that runs deterministically in CI, compares against a frozen baseline, and blocks the merge on regressions. Here is how to make it fast, stable, and trusted.

Eyeballing a few outputs does not scale, and a prompt tweak that fixes one case quietly breaks ten. A real LLM eval pipeline is a versioned dataset, layered scorers, and a CI gate. Here is the architecture.