vanishing gradients

This question ties together why deep nets were hard to train and the cluster of tricks that fixed it. The signal is the multiplicative-gradient cause and naming the real fixes: ReLU, He/Xavier init, residuals, normalization. Here is the answer.

Sequence models are foundational and still asked, especially the gating that fixed RNNs and why attention won. The signal is connecting the vanishing-gradient story to the parallelism argument that let transformers ride the scaling wave.

Bad initialization makes deep nets fail to train at all, with activations that vanish or explode through depth. The signal is knowing why zero init is fatal and how to match Xavier or He to your activation function.