← Machine Learning & Data Science / 19

mediumGoogle NVIDIA Meta

What causes vanishing and exploding gradients, and how do activations, initialization, and residuals fix them?

This question ties together why deep nets were hard to train and the cluster of tricks that fixed it. The signal is the multiplicative-gradient cause and naming the real fixes: ReLU, He/Xavier init, residuals, normalization. Here is the answer.

Updated Aug 2026 · Grounded in real Applied AI Engineer interview loops and written to a senior-engineer editorial bar.

This question ties together why deep nets were hard to train and the cluster of tricks that fixed it. The signal is the multiplicative-gradient cause and naming the real fixes: ReLU, He/Xavier init, residuals, normalization. Here is the answer.

more free answers with an account · no card

UP NEXT ON YOUR JOURNEY

Next in this trackHow do CNNs work? Explain convolution, pooling, and the receptive field.Next in this trackHow does a Vision Transformer (ViT) work, and when does it beat a CNN?Next in this trackHow do diffusion models work, and what do the VAE and U-Net do in latent diffusion (Stable Diffusion)?

DISCUSSION · 0

No comments yet — be the first to share your approach.