MLflow Dataset Summary MD5 Collision Vulnerability and Fix

Vulnerability Overview This vulnerability concerns a determinism issue in dataset summary computation, allowing attackers to construct collision datasets. Impact Scope Affected Component: MLflow's dataset summary computation functionality. Specific Impact: Attackers can construct specific datasets to cause collisions in summary computation, thereby compromising data integrity. Remediation 1. Head-Tail Sampling: For dataframes/arrays exceeding 10K elements, sample both the head (5K) and the tail (5K), rather than only the head (10K). This prevents the "keep the first 10K rows identical, modify the rest" attack while maintaining O(1) performance. 2. Include All Column Types: Remove string/numeric filters that silently excluded datetime, boolean, and categorical columns. Now, hashes the entire dataframe. 3. SHA-256 with 32 Hex Characters: Replace MD5[8] (32-bit, 50% collision rate on datasets of 65K) with SHA-256[32] (128-bit, birthday bound ~2^64). 4. Include dtype Information: Add dtype strings to the hash input to prevent type coercion collisions (e.g., int32 vs. int64). 5. Backward Compatibility: Retain as a deprecated wrapper. Performance and Trade-offs Performance: Maintain O(1) sampling without requiring a full dataset scan. Trade-off: The middle section (between the head and tail) is not sampled. This is an acceptable trade-off as the performance is significantly better than the previous deterministic head-only approach. Testing and Verification Existing Unit/Integration Tests: Updated. New Unit/Integration Tests: Added. Manual Testing: Confirmed that datetime/boolean columns now affect the summary (no collisions), tail modifications now affect the summary (head-tail sampling is effective), dtype differences produce distinct summaries, and deterministic behavior remains intact (same data = same summary). Documentation and Release Notes Documentation Updates: Not required. MLflow Skills Repository Updates: Not required. Release Notes: Dataset summary computation now uses head-tail sampling (instead of head-only), includes all column types (not just string/numeric), and uses SHA-256 with a longer output (32 hex characters instead of 8). This is a breaking change: summaries calculated previously will differ from those calculated with the new method. Component Impact Affected Components: MLflow Model Server, Model Deployment Tools, Spark UDFs, MLflow Model Evaluation Functionality, Evaluation Metrics, Evaluation Workflows, MLflow AI Gateway Client API, Server and Third-party Integrations, MLflow Prompt Engineering Features, Prompt Templates and Prompt Management, MLflow Tracking Functionality, Tracking API and LLM Tracking Features, MLproject Format, Project Run Backends, Frontend, UI, Plots, JavaScript Dev Server, Build and Test Infrastructure, MLflow Documentation Pages. Classification and Priority Classification: User-visible bug fix, worthy of mention in release notes. Priority: Non-critical bug or security fix; can wait for the next minor release. Maintainer Feedback Performance Concerns: Performance is a key issue. This PR maintains O(1) sampling (no full dataset scan) while addressing deterministic collision vectors. Head-Tail Sampling Cost: Same cost as head-only (two slices, each O(1)). Include All Column Types: Negligible overhead (datetime/boolean columns are already in memory). SHA-256 vs. MD5: Adds ~2-5% overhead in benchmarks, an acceptable security improvement. Middle Section Blind Spot: An acceptable trade-off—the attacker can no longer easily construct collisions by merely keeping the first 10K rows identical. Code Blocks No POC code or exploit code.

Goal Reached Thanks to every supporter — we hit 100%!

MLflow Dataset Summary MD5 Collision Vulnerability and Fix

More from this source