82When do you use pairwise versus pointwise evaluation for LLM outputs, and what does each get wrong?▼mediumOpenAIMicrosoftCohere1 replies◆ premiumPairwise asks which is better, pointwise asks how good. The signal is knowing which is more reliable, how Elo/Bradley-Terry aggregates comparisons, and where each method quietly misleads.Open full answer →
113What is CUPED, why does it shrink experiment variance, and how does it compare to stratification and regression adjustment?▼hardMicrosoftNetflixMeta1 replies◆ premiumCUPED can cut the sample size an A/B test needs by half without touching validity. The signal is explaining why subtracting a pre-experiment covariate reduces variance but never biases the estimate. Here is the answer.Open full answer →
114Why does peeking at an A/B test inflate false positives, and how do sequential and always-valid tests fix it?▼hardNetflixUberMicrosoft1 replies◆ premiumChecking an experiment every day and stopping when it hits significance can triple your false-positive rate. The signal is knowing why, and the family of methods that make continuous monitoring valid. Here is the answer.Open full answer →
