AppliedAIPrep logoAppliedAI/Prep

alibi

Applied AI interview questions tagged alibi, across every topic.

2 questions · 1 unlocked for you
Open full answer →
Open full answer →

Concepts behind "alibi"

The curriculum that explains the ideas these questions test.

Core
🧠 Foundations of LLMs & GenAISign in
Positional Encodings (RoPE and ALiBi)Attention is order-blind, so models inject token position separately. Modern LLMs use relative schemes: RoPE rotates query/key vectors by an angle proportional to position so the attention score depends only on the offset between tokens, and ALiBi adds a distance penalty to attention scores. Both extrapolate to longer sequences far better than learned absolute positions, which is why RoPE-with-scaling is how context windows get extended. Applied-AI interviews probe it because it explains how long-context models are built.
Explore the full curriculum →