sample ratio mismatch

Running an A/B test is easy; running a valid one is hard. The signal is naming the traps that produce confidently wrong conclusions and the fix for each. Here is the checklist a senior experimenter carries.

A pre-existing gap between your groups means randomization or instrumentation is broken, and the whole experiment is suspect. Strong candidates name sample-ratio mismatch on sight. Here is the full diagnosis and the disciplined response.