mediumGoogle Meta NVIDIA

Compare activation functions (sigmoid, tanh, ReLU, GELU, softmax) and when to use each.

A staple that checks whether you know why ReLU dethroned sigmoid and how to match the output activation to the loss. The signal is the vanishing-gradient story told correctly. Here is the answer.

Updated Aug 2026 · Grounded in real Applied AI Engineer interview loops and written to a senior-engineer editorial bar.

A staple that checks whether you know why ReLU dethroned sigmoid and how to match the output activation to the loss. The signal is the vanishing-gradient story told correctly. Here is the answer.

Unlock the other 750 answers · ₹2,000 / $25Your progress and mastery stay saved · 6 months · one payment · no auto-renew

UP NEXT ON YOUR JOURNEY

Next in this trackCompare batch, stochastic, and mini-batch gradient descent (and momentum).Next in this trackHow do you choose a loss function (MSE, MAE, Huber, cross-entropy, focal, contrastive)?Next in this trackHow do you evaluate a regression model (MAE, RMSE, R², MAPE)?

DISCUSSION · 0

No comments yet — be the first to share your approach.