How and when do you use synthetic data (LLM-generated) for training or fine-tuning?

Synthetic data is how teams get training data when real data is scarce, and it has sharp failure modes. The signal is naming when it helps, how to hold the quality line, and why training on model output across generations narrows the distribution.

Updated Aug 2026 · Grounded in real Applied AI Engineer interview loops and written to a senior-engineer editorial bar.