All projects
Open Food Facts data quality
Porting Open Food Facts' Perl data quality checks to tested Python and DuckDB
Research · Google Summer of Code 2026 proposal (Open Food Facts) · Solo builder · March 2026
Overview
A prototype for my Google Summer of Code 2026 proposal to Open Food Facts. It ports data quality checks from the ProductOpener Perl codebase into tested Python backed by DuckDB.
What I built
I migrated five checks from DataQuality.pm and DataQualityFood.pm, keeping the original thresholds, tolerances, and tag names:
- Completeness: products missing required fields
- Quantity: product and serving quantity ranges
- Nutrition ranges: impossible values, such as more than 105 g per 100 g or negatives
- Nutrition composition: relationships like saturated fat not exceeding total fat
- Ingredient text quality: OCR artifacts, unexpected characters, and unknown ingredients
How it works
Checks register through decorators and run from a small orchestrator over a DuckDB data layer, with a CLI that runs every check or just one. A parity test suite loads real products from the Open Food Facts API and compares the Python output against the Perl generated quality tags stored on each product.
Related contribution
I also opened a Robotoff pull request that adds an OCR regex to tell the new Nutri-Score apart by detecting "Nouveau calcul" text in 11 languages, with 15 new test cases.