lasso

Univariate ranking is the trap everyone falls into: it keeps redundant features and drops ones that only matter in combination. Here is the filter/wrapper/embedded breakdown plus mRMR and Boruta, and where each one bites.

Lasso gives sparsity, Ridge handles correlated features, and each fails where the other shines. Elastic Net blends them for a specific, common failure. Here is when it earns its two hyperparameters.