70Deduplicate near-identical documents in a huge corpus. Implement MinHash for fast similarity.▼hardGoogleScale AIDatabricks2 replies◆ premiumComparing every pair of a million documents is a trillion comparisons. MinHash estimates Jaccard similarity from a tiny signature, and LSH turns dedup into a near-linear scan. Here is the implementation.Open full answer →
59Compare similarity/distance metrics: Euclidean, cosine, Manhattan, Jaccard, Mahalanobis.▼mediumAmazonGoogleMeta1 replies◆ premiumPick the wrong distance metric and you quietly break kNN, clustering, and retrieval. The signal is knowing what each metric actually measures and matching it to the data: magnitude vs direction, sets, correlated features.Open full answer →