← 🧠 Foundations of LLMs & GenAINEXT IN FOUNDATIONS OF LLMS & GENAIThe KV Cache→
Core
Constitutional AI and RLAIF
RLAIF (RL from AI Feedback) replaces human preference labels with AI-generated ones, scaling alignment past the human-labeling bottleneck. Constitutional AI is Anthropic's specific approach: the model critiques and revises its own outputs against a written set of principles (a constitution), generating the preference data from those principles. The win is scalability, consistency, and explicit, editable values; the risk is the AI judge's own biases. Applied-AI interviews probe it because it is how alignment scales and how values become explicit and auditable.
a free account unlocks the core curriculum tier · no card
RELATED CONCEPTS
PRACTICE THIS IN REAL QUESTIONS
LLM & GenAI FundamentalsWhat is Constitutional AI / RLAIF, and how does it differ from RLHF?→LLM & GenAI FundamentalsWalk through RLHF, then explain DPO and why it has largely displaced PPO-based RLHF.→LLM & GenAI FundamentalsExplain PPO and GRPO for LLM alignment. Why did GRPO drop the value model?→LLM & GenAI FundamentalsAfter RLHF, your model is safer but worse at hard tasks. How do you manage the alignment tax?→LLM & GenAI FundamentalsYour RLHF model games the reward model instead of being genuinely helpful. How do you stop reward hacking?→LLM & GenAI FundamentalsWhat is reward-model overoptimization, and how do you detect and bound it during RLHF?→
COMPANIES THAT ASSUME THIS