23What is data skew in a distributed job (Spark), and how do you fix it?▼hardDatabricksAmazonMeta1 replies◆ premiumThe top cause of mysteriously slow Spark jobs: one partition does most of the work while the rest idle. The signal is reading the symptom (a few straggler tasks) and reaching for the right fix, salting, broadcast, or AQE.Open full answer →
08How do you run hyperparameter optimization efficiently across a cluster of GPUs?▼mediumGoogleNVIDIADatabricks1 repliesunlockedGrid search is the wrong answer at scale. The signal is knowing why random beats grid, how Bayesian optimization and early-stopping schemes (Hyperband/ASHA) spend compute smartly, and how to parallelize without stragglers.Open full answer →
20What is model sharding, and how do tensor and pipeline parallelism split a model across GPUs?▼hardNVIDIAOpenAIGoogle2 replies○ sign inWhen a model is too big for one GPU you split the model itself, not just the data. The signal is distinguishing tensor parallelism (split within a layer) from pipeline parallelism (split across layers) and matching each to the interconnect. Here is the answer.Open full answer →
