23What is data skew in a distributed job (Spark), and how do you fix it?▼hardDatabricksAmazonMeta1 replies◆ premiumThe top cause of mysteriously slow Spark jobs: one partition does most of the work while the rest idle. The signal is reading the symptom (a few straggler tasks) and reaching for the right fix, salting, broadcast, or AQE.Open full answer →
08How do you run hyperparameter optimization efficiently across a cluster of GPUs?▼mediumGoogleNVIDIADatabricks1 repliesunlockedGrid search is the wrong answer at scale. The signal is knowing why random beats grid, how Bayesian optimization and early-stopping schemes (Hyperband/ASHA) spend compute smartly, and how to parallelize without stragglers.Open full answer →
20Model sharding: tensor vs pipeline parallelism across GPUs.▼hardNVIDIAOpenAIGoogle2 replies○ sign inTensor parallelism splits matrices inside a layer and wants fast intra-node links, pipeline parallelism splits across layers. Matching each to hardware.Open full answer →
