Generate and apply insights automatically based on constraints

[raphael0202]

Being able to generate and add automatically information based on available product data would be very valuable.

This new information can be computed using a list of known rules, such as:

• label MSC when MSC packager code (Add MSC or ASC as a parent when a MSC or ASC code is added #143)
• wine category when AOC label is present (Extract AOC from images #78)
• add carrefour store if carrefour brand is present
• add carrefour store and en:organic label and fr:ab-agriculture-biologique label if carrefour-bio is present
• add category frozen ships if has category en:shrimps and if label frozen is present
• add category en:artificially-sweetened-yogurts if has ingredient:sweetener and if category en:yogurts is present
• add category en:sugared-yogurts if has category en:yogurts and has ingredient en:sugar but doesn't have ingredient en:sweetener (yogurts with sugar in the ingredients but without sweeteners)

Each rule has a list of input conditions and an output (addition of one or more information to the product). As a first step, we can create these first input conditions:

• hasCategory, target category value_tag as parameter
• hasIngredient, target ingredient value_tag as parameter
• hasNotIngredient, target ingredient value_tag as parameter
• hasBrand, normalized brand name as parameter
• hasLabel, target label value_tag as parameter
• packagerCodeStartsWith, with string (pattern) as parameter

As output:

• addCategory
• addStore
• addLabel

We can implement this rule engine in Robotoff. Each time a product is updated, Robotoff receives a webhook call. We can check all rules against the product, and generate insights that are automatically applied. We can store the rules in Postgres DB, to allow easy editing/collaboration.

For the rule engine, @alexgarel has suggested to use Lucene query language (see https://github.com/jurismarches/luqum), as we're going to index all data in an Elasticsearch instance. We would search for products that match each rule instead of searching for rules that match the product. Given how simple the rules are (binary AND operator only), I don't believe we need a tool as powerful as Elasticsearch for this.

Not supported yet by Robotoff but to be done:

• add origins of ingredients en:France and labels fr:aoc if category fr:Sancerre is present

[teolemon]

• The Collaborative Regex Engine tries to go a step further by crowdsourcing the creation of rules (for primary and secondary insights) (Create a Collaborative Regex Engine #226)
• There's also a server bug (Tag rules: Tag A + Tab B = Tag C openfoodfacts-server#3426)
• Finally, we have already support in the taxonomy for this, with (for instance) many categories like PGI already mandating an origin 🎉
  • hr:Paška sol
    protected_name_type:en: pdo
    origins:en: en:Croatia