19What causes vanishing and exploding gradients, and how do activations, initialization, and residuals fix them?▼mediumGoogleNVIDIAMeta1 replies○ sign inThis question ties together why deep nets were hard to train and the cluster of tricks that fixed it. The signal is the multiplicative-gradient cause and naming the real fixes: ReLU, He/Xavier init, residuals, normalization. Here is the answer.Open full answer →
27What consumes GPU memory during training/inference, and how do you fit a model that doesn't?▼hardNVIDIAOpenAIMeta1 replies◆ premiumOOM is the most common wall in deep learning, and 'buy a bigger GPU' is the weakest answer. The signal is naming the memory consumers, knowing which one dominates, and matching the right lever to it.Open full answer →
