pretraining
Applied AI interview questions tagged pretraining, across every topic.
4 questions · 0 unlocked for you
Concepts behind "pretraining"
The curriculum that explains the ideas these questions test.
Core
Scaling LawsScaling laws say model loss falls predictably as a power law in parameters, data, and compute, which is why bigger models trained on more data reliably get better. The Chinchilla result showed that for a fixed compute budget you should scale parameters and training tokens together (roughly equally), meaning prior large models were under-trained. This reshaped how compute is allocated and why smaller, data-heavy models are competitive. Applied-AI interviews probe it because it underlies model-selection and the data-vs-size economics.🧠 Foundations of LLMs & GenAISign in
Core
Transfer LearningTransfer learning reuses a model pretrained on a large general corpus as the starting point for a new task, so you inherit learned features instead of training from scratch. The two modes are feature extraction (freeze the backbone, train only a new head) and fine-tuning (unfreeze some layers and keep training), and the choice turns on how much labeled data you have and how far the new domain has drifted. Applied AI interviews probe it because it is the default for vision and NLP when labels are scarce, and because candidates often fine-tune when they should freeze, or vice versa.📊 Evaluation & ML FoundationsSign in