1.56.4 (2024-11-26)
- increase max_requests to 500 (b942c16)
1.56.3 (2024-11-23)
- remove dataset push to HF (#1465) (40ecbb2)
- restart gunicorn workers every 50 requests (#1469) (642583c)
- use Robotoff attributed ports for ML services (#1464) (dd016d9)
- add documentation about the nutrition extraction model (#1468) (26bc881)
- fix markdown syntax (1d68bc9)
- fix syntax error (d46052f)
- remove legacy settings for container deploy (9117d68)
- use port numbers in Robotoff attributed range (#1463) (6a99eba)
1.56.2 (2024-11-07)
1.56.1 (2024-11-05)
1.56.0 (2024-10-29)
- schedule Hugging Face Parquet dataset push every day (7e525c1)
- deps-dev: bump werkzeug from 3.0.4 to 3.0.6 (64f5dbc)
1.55.0 (2024-10-29)
- ⚡ Add CLI command to convert and push JSONL to Huggingface (#1436) (d68a231)
- improve Slack & nutripatrol notifiers (ef0f861)
- add order_by parameter to /api/v1/insights route (2dcf4a7)
- convert image to RGB before nutrition extraction (#1448) (96401ef)
- fix KeyError exception in UPCImageImporter (#1443) (022a788), closes #1442
- fix wrong call to logger.exception (#1441) (750eea9), closes #1440
- remove columns in output parquet files (d8534fe)
- update fields fetched to generate parquet file (4f57419)
- add /api/v1/insights endpoint to documentation (f5b1de4)
- add HF_TOKEN to .env (73019bb)
- reformat test_products.py (11e0199)
1.54.1 (2024-10-25)
- fix issue with max() call on empty sequence (5410b30)
1.54.0 (2024-10-25)
- fix nutrient insight importer (0e4aff3)
- fix NutrientExtractionImporter (b533199)
- fix nutrition extraction insight generation (#1438) (cfdfb07)
1.53.3 (2024-10-16)
- add mg as possible unit for salt (463a1f1)
- bump openfoodfacts python dependency (560c596)
- fix launch normalize barcode job (65bdacb)
- remove incorrect parameter when calling CLIPImageProcessor (340a8e0)
1.53.2 (2024-10-08)
- add script to normalize barcodes in DB (4adeb6b)
- fix normalize script (24db8fb)
- normalize barcode in all API routes (7ca87de)
- use ReditUpdate.product_type (963eabb)
1.53.1 (2024-10-07)
- add new nutrient mention for portuguese (#1425) (49df155)
- remove legacy routes (#1424) (c53617d)
- use images.openfoodfacts.org to fetch OCR files (59fa01f)
- use new barcode normalization (65e16dd)
1.53.0 (2024-10-01)
- don't create index concurrently in migration (96d3374)
- remove old object detection models (#1423) (4d22142)
- remove scripts/ocr/run_ocr.py (2d8fbaf)
1.52.1 (2024-09-18)
- 📝 Batch Job - Spellcheck documentation (#1408) (2748324)
- deps: bump vllm from 0.5.4 to 0.5.5 in /batch/spellcheck (9a09729)
1.52.0 (2024-09-17)
- update ingredient detection model (459f6f4)
- bump ingredient detection model version (dfa2614)
- fix entity aggregation bug for NER detection (5f2b94c)
- use datetime.now(timezone.utc) instead of utcnow() (2203ecf)
- use the same predictor_version for all spellcheck predictions (857fea1)
- wait for Product Opener to generate the dump before downloading it (6eae9d5)
1.51.0 (2024-09-11)
- 🚑 Changes (b8dcd5a)
- 🚑 Credential + Importer (53752d5)
- 🚑 Credentials + Importer + Test (3e2f5e3)
- 🎨 Add key during request by the batch job (fd7c587)
- 🐛 Forgot a return (a1de5d9)
- 🔒 Secure Batch Data Import endpoint with a token key (360f2e4)
- ✨ Change batch job launch from api endpoint to CLI (93c1232)
- ✨ Implemenation reviews (c76df4d)
- ✨ Restructure code (b7a44c4)
- allow to specify parameters when launching spellcheck job (de050fc)
- batch - spellcheck: ⚡ API endpoint batch/launch ok: Batch extraction with DuckDB and launch on GCP (a417fdf)
- batch - spellcheck: ⚡ From predictions to insights (74c3828)
- batch - spellcheck: ⚡ Integrate batch data from job into Robotoff sql tables (bdb8733)
- Batch job - Spellcheck: ⚡ (531f3b5)
- batch-spellcheck: ⚡ Batch extraction from database before Batch processing operational (baf1f1d)
- 🎨 Change predictor version to also track... the predictor version (9800591)
- 🐛 Fixed bug & Better error handling with Falcon (e87857c)
- add batch_dir as param in /batch/import route (36f02fb)
- add log message in import_spellcheck_batch_predictions (2329201)
- batch-spellcheck: 💄 Fix Spellcheck Batch job file name for Dockerfile ENTRYPOINT (7a17edc)
- fix bug in spellcheck insights import (db73aea)
- fix issues with duckdb command + unit test (cbe250f)
- fix mypy and flake8 issues (ac7907d)
- improve naming of batch jobs bucket files (242f950)
- improve spellcheck Dockerfile (235ed4b)
- pass GOOGLE_CREDENTIALS envvar as base64 encoded string (6be2367)
- provide webhook URL as envvar in spellcheck batch job (3897d65)
- update content-type in batch job robotoff webhook call (368ee12)
- update spellcheck batch script (374a829)
- 📝 Add batch/import api endpoint to doc (be563e3)
- 📝 Because perfection (3ea91d2)
- ✨ Black on spellcheck script (8f84146)
- ✨ make lint (2ff7345)
- add GOOGLE_APPLICATION_CREDENTIALS envvar (f93fc9f)
- add GOOGLE_CREDENTIALS during deployment (b18e613)
- batch-spellcheck: 💚 Fix some bugs: batch-extraction & batch-launch (c8758e1)
- remove CodeQL analysis (1c0946d)
- remove Sonar Cloud analysis (4997a01)
1.50.5 (2024-09-04)
- deps-dev: bump cryptography from 43.0.0 to 43.0.1 (0ad0c56)
- update SSH_PROXY_USERNAME for prod deployment (34e6aef)
1.50.4 (2024-09-03)
- fix product weight detection bug (fe7e758)
- only run nutrition table detection for food type (c019df9)
- remove Hacendado store (e946444)
- remove unused class (22fae32)
- add tmp volume for ES (6225bcb)
- add volume for /tmp (15130d2)
- deps: bump sentry-sdk from 1.14.0 to 2.8.0 (f991bf5)
- improve robotoff documentation (b9365a7)
- make api depends on elasticsearch in docker-compose.yml (bf99ca8)
1.50.3 (2024-08-21)
- update proxy_username in container-deploy.yml (d469798)
1.50.2 (2024-08-21)
- add logo_annotation.server_type field (a318737)
- dev_build: ⚡ Add network po-default before build in make dev (c99b2c1)
- display number of logos in index in add-logo-to-ann command (8251082)
- improve performance of random logo search (dec765f)
- improve question count query performance with vote (f19d49a)
- remove some store regex false positive (6ba5f7e)
- use subquery when fetching questions without user votes (15e6cfb)
- add index on logo_annotation.server_type (0da1006)
- bring back robotoff-backups (0f5b8ca)
- fix deploy configuration (e4999ac)
- Update README.md (fc94d96)
1.50.1 (2024-08-13)
1.50.0 (2024-07-30)
- switch from PostgreSQL 11 to PostgreSQL 16 (7cdb0d2)
1.49.0 (2024-07-29)
- add sentry to update-listener daemon (65a1a49)
- upgrade matplotlib (e094e60)
1.48.1 (2024-07-19)
- apply toml lint on pyproject.toml (c316e41)
- avoid failing all metrics for a single failure (86f0cae)
- delete more unused DB indices (4942c44)
- improve scheduler (88501e3)
- switch some info logs into debug (03e8213)
- turn more info message into debug (3dffceb)
- add types-pytz dev dep (6d29674)
- fix backup script (1a03dd8)
- fix errors when building docs (bf26f57)
- upgrade poetry (c742e10)
1.48.0 (2024-07-18)
- use Redis Stream to listen to events (e79e8ec)
- deps-dev: bump black from 22.10.0 to 24.3.0 (#1323) (bc259ec)
- upgrade black (462483d)
- use the common network to access update redis & mongodb (6cf259f)
1.47.0 (2024-07-12)
- decrease minimum score to leverage nutrition table detections (4a26918)
- drop unused indices (#1364) (f0c348f)
- fix get_type call (18181bb)
- fix ImagePrediction.model_name (4a9e1a6)
- fix issue with model loading (1e15d72)
1.46.0 (2024-07-08)
- fix incorrect astype call (#1359) (f3d3ff6)
- make log message less verbose in api.py (#1361) (83e043b)
1.45.2 (2024-07-04)
1.45.1 (2024-06-24)
1.45.0 (2024-05-15)
- deps-dev: bump jinja2 from 3.1.3 to 3.1.4 (#1342) (60e7f13)
- deps-dev: bump werkzeug from 3.0.1 to 3.0.3 (#1341) (a9244f9)
- deps: bump dnspython from 2.4.2 to 2.6.1 (#1335) (084b9d9)
- deps: bump gunicorn from 20.1.0 to 22.0.0 (#1336) (9ec607a)
- deps: bump idna from 3.6 to 3.7 (#1334) (d514f82)
- deps: bump tqdm from 4.66.1 to 4.66.3 (#1340) (beef9a4)
- New Crowdin translations to review and merge (#1324) (5310130)
1.44.0 (2024-04-11)
- fix nutripatrol deployment (#1333) (9e88ac2)
- fix nutripatrol deployment and API call (8455a0e)
- fix unit test (9d339d0)
- tests: fix unit test (0195926)
- deps: bump pillow from 10.2.0 to 10.3.0 (#1328) (a3357dc)
- deps: bump pymongo from 4.5.0 to 4.6.3 (#1330) (5f8b75e)
- deps: bump transformers from 4.36.0 to 4.38.0 (#1331) (4fe0ede)
- docker: remove legacy "version" field in docker-compose configs (1e047aa)
1.43.0 (2024-03-26)
- deps: bump orjson from 3.8.14 to 3.9.15 (#1319) (0f2ee58)
- New Crowdin translations to review and merge (#1312) (6f888aa)
- switch to docker compose v2 (#1325) (5bde479)
- Update docker config (#1321) (09dc8b4)
1.42.1 (2024-02-22)
- deps-dev: bump cryptography from 42.0.0 to 42.0.2 (#1316) (cc3a6bf)
- deps-dev: bump cryptography from 42.0.2 to 42.0.4 (#1318) (b775b8d)
1.42.0 (2024-02-13)
- fix extreme weight detection for multi-packaging (#1298) (d9660c6)
- fix run_logo_detection function (4775ccc)
- fix unit test (cf12304)
- improve docstring and add log messages (#1310) (3e03f83)
- change prod mongodb address (again) (ade67c2)
- Create FEATURES.md to doc features and todos (#1309) (8c1b404)
- deps-dev: bump cryptography from 41.0.7 to 42.0.0 (#1308) (6585390)
- deps-dev: bump jinja2 from 3.1.2 to 3.1.3 (#1305) (e237b6a)
- deps: bump pillow from 10.0.1 to 10.2.0 (#1306) (5ea2ab7)
- deps: bump transformers from 4.30.2 to 4.36.0 (#1301) (e36eb3b)
- New Crowdin translations to review and merge (#1303) (bc00a7c)
1.41.2 (2024-01-08)
- update .gitignore (7b2b849)
1.41.1 (2023-12-18)
- fix image_response function (82a826d)
1.41.0 (2023-12-18)
- add a POST version of /predict/lang endpoint (619c477)
- allow gunicorn auto-reload locally (a1ffa44)
- fix delete_images call (f0ad1e1)
- fix missing SQL join in ANNResource (607f743)
- fix parse_ingredients function (189c5cc)
- deps-dev: bump werkzeug from 3.0.0 to 3.0.1 (#1276) (da54a80)
- increase mem limit of API service (ed32bfb)
- New Crowdin translations to review and merge (#1291) (9b10ae0)
- switch to Falcon 3.X (6362185)
1.40.0 (2023-11-13)
- add basic detection for mention of organic ingredient (0939075)
- add bounding box info to IngredientPredictionAggregatedEntity (f45cd39)
- add ingredient parsing information (e9f2b60)
- save ingredient list detection in DB (8f2a4b4)
- use allergen grammar to postprocess ingredient detections (36eac56)
- add bounding box information to /predict/ingredient_list route (af40b55)
- add missing transformer_pipeline.py file (7543ef0)
- delete logos in elasticsearch when image is deleted (b137866)
- don't include deleted images in searched logos (1fce684)
- don't include logos from deleted images in additional routes (4b7e130)
- improve allergen detection (2a66666)
- improve ingredient detection output saved in DB (6b62ca2)
- increase default logo threshold to 0.2 (from 0.1) (74b48f5)
- remove warning log message (8ea3476)
- add allergen taxonomy (6704189)
- add clean_tests command (a6f8f33)
- add first version of trace grammar (6bafa88)
- fix docstring in models.py (d9edc6f)
- improve documentation on Dockerfile and docker-compose.yml (ab4c4ac)
1.39.0 (2023-11-06)
- save logo text (extracted using OCR in DB) (bf4632f)
- add cache directory to repo (2905741)
- add timeout to robotoff request (healthcheck) (ab17cee)
- fix bounding box field default value (5b78f5d)
- fix incorrect call to cache_asset_from_url (d5562c1)
- use distinct cache for test assets (c52d3f9)
1.38.1 (2023-10-30)
- fix bug in get_image_from_url (ce0c1b2)
1.38.0 (2023-10-30)
- add a disk cache mechanism to cache images (0330216)
- add migrate_peewee library to handle DB migrations (8b77195)
- add product_insight.bounding_box field (1901b15)
- fix Dockerfile related to migrations folder (0fc70ef)
- fix logging issue (5e86341)
- fix migration application during deployment (27ea630)
- fix previously introduced issues (0624efe)
- improve DB migration (e67d271)
- mark images as deleted in DB when deleted on Product Opener (997e989)
- migrate-db when running make dev (0a2171d)
- more stores (3dea05d)
- pull image before launching migration (bd6a15a)
- upgrade peewee (264d9bf)
- add documentation about DB migration (16c865f)
- deps: bump pillow from 9.3.0 to 10.0.1 (e056268)
- remove CachedStore class (5d8c007)
1.37.0 (2023-10-25)
- allow to submit category value_tag in /annotate route (dd6d81e)
- store fingerprint of all images (ed8fd38)
- improve add-logo-to-ann CLI command (c67b8ff)
- set K_NEAREST_NEIGHBORS to 10 (instead of 100) (a177e13)
- use json.dump instead of f.write + json.dumps (60f15e9)
1.36.0 (2023-10-16)
- use openfoodfacts SDK for OCR processing (6344936)
- upgrade to Pydantic V2 (6979cec)
1.35.0 (2023-09-04)
- Crowdin PR title (1adfc96)
- don't initialize unit registry in product_weight.py (468c4f6)
- fix wrong label rouge detection (492fda4), closes #1255
- add docstring (279ced1)
1.34.1 (2023-08-31)
- fix bug in product_weight.py (2e94d11)
- update normalize_weight function after Pint upgrade (80d3a09)
- update packaging denylist (23e6e26)
- update dependencies (5353f1c)
1.34.0 (2023-08-31)
- apply automatically nutrition images insights (6d79855)
- sort label whitelist (7b13090)
- Update packaging material shape map for fr (450409f), closes #1250
- update packaging shape denylist (84256ba)
1.33.0 (2023-08-29)
- disable matcher predictor for category (07ada5b)
- add new exception for packaging (e397e57), closes #1058
- disable temporarily en:eu-non-agriculture and en:eu-agriculture (043b96c)
- don't generate weight insight when value is suspicious (51a19fa), closes #302
- fix bug in refresh_insights (dd19b6e)
- fix init-elasticsearch Makefile command (2d729d4)
- remove pdo and pgi regex (008bd9d)
- update label whitelist (2497642)
- pin openfoodfacts dependency (1f9ac80)
- update branc blacklist and label whitelist (d7948c5)
- upgrade requests (f937442)
1.32.1 (2023-08-28)
- add some items to brand taxonomy blacklist (ef0b626)
- auupdate brand_taxonomy_blacklist.txt (739c748)
- brand blocklist (3055d13)
- fix bug in doctext.py and improve script (b54ef2b)
- fix bug in pprint (c620d59)
- fix call to get_insights_ (5d909b9)
- fix is_data_required method signature (6342832)
- fix wrong call to add_category_insight in tests (69b5f1e)
- improve brand exclusion for 'taxonomy' predictor (7705823)
- improve candidate generation in LabelInsightImporter (51e046a)
- mark use of md5 and sha1 hash function as safe (ca0963f)
- set automatic_processing=None for flashtext label insight (ba6ceb7)
- fix deepsource warning (58ca3e7)
- remove legacy function mark_insights (9460393)
- remove legacy functions in visualization_utils.py (b620062)
- remove pypi.yml action (1773b87)
- switch to f-strings (fdeba36)
- update brand taxonomy blacklist (df19331)
- use consistent logging arg passing (2385889)
1.32.0 (2023-08-21)
- fix re-imports (9a6bcbe)
- fix some deepsource performance warnings (7424587)
- remove legacy code (1aa13ad)
- use prod JSONL dump in staging (07ab8fd)
- add make command to download ingredient detection model (fa305cd)
- delay scheduled jobs relying on JSONL export (d4033f5)
- fix release-please.yml (07a5ff7)
- improve release changelog (57d4bbf)
- rename master to main (update workflows and doc) (b9e1107)
1.31.1 (2023-08-14)
- add robots.txt (10387ff)
1.31.0 (2023-08-11)
- add mdx-truly-sane-list extension back (460dae6)
- barcode should not be an empty str in webhook call (a7b6472)
- fix bug in run_upc_detection (a62319d)
- remove above-threshold-campaign (326626a)
- replace parameter
countrybycountries(56b0804) - send release info to Sentry (58acf39)
- use openfoodfacts-python package for taxonomy processing (eaaeca3)
- update OpenAPI documentation (6da2978)
1.30.1 (2023-06-30)
- allow to provide image embedding as input in /predict/category (655a230)
- make more robust unit tests (bbebf3f)
- remove false positive label for moderation (bfd2b58)
- remove previous categorization model (058d0cc)
- update /predict/category schema to accept images as input (728246e)
- improve OpenAPI documentation (c4b3843)
- improve OpenAPI documentation (2) (db5df96)
- update OpenAPI documentation (07d1136)
1.30.0 (2023-06-22)
- add fasttext langid module (432027b), closes #1122
- add first version of ingredient list NER + API (9083bdb)
- add
insight_typesparameter in /question/{barcode} (fc7b76a), closes #1139 - fix default FASTTEXT_MODEL_DIR value (e4cc4e7)
- improve error handling in /predict/ingredient_list route (7a996cc)
- improve ingredient detection pipeline (9ff8aec)
- remove spellcheck module (916132a)
- sort product questions by priority (5ea69e3), closes #1138
- update apscheduler (dd4eed8)
- update poetry install command in Dockerfile (75f8b34)
1.29.0 (2023-06-01)
- Improve /predict/{nutrient|ocr_prediction} routes (f06a45a)
- improve nutrition image bounding box detection (737b630)
- add CLI command to pretty print OCR result (26f44a4)
- display diffs in /webhook/products route (07255c6)
- fix ENABLE_PRODUCT_CHECK flag (dd1d09f)
- fix NutritionImageImporter.generate_candidate (74e346c)
- fix wrong paragraph offset for OCR (abb54a6)
- improve /predict/nutrition route (95d953b)
- only display most important services in make log command (60b2843)
- rename compute_intersection_bounding_box function (d8d732c)
- support pro platform for MongoDB queries and image/OCR URLs (a3ac603)
- update SSH_PROXY_HOST for deployment (a8b8292)
1.28.1 (2023-05-11)
- add a Make command to init elasticsearch (68e7edf)
- add danish translation for nutrient detection (ba02a63)
- add log messages to init-elasticsearch CLI command (e950cbd)
- add ml-gpu.yml docker-compose file (a005b0f)
- don't update elasticsearch indices in _update_data scheduler job (c12f803)
- don't update product dump at scheduler startup (4bee57c)
- fix dl-models command (ef26311)
- fix launch-burst-worker command (4a49ac3)
- fix logo elasticsearch index mapping (8a5a119)
- fix make dl-models command (d6a3b25)
- fix unit tests (90cce2a)
- init elasticsearch during make dev call (0ac9051)
- remove tf_models/models.config file (eb54e81)
- use explicit model control model on Triton (fb46bf8)
- fix comment in prediction.ocr.dataclass (4bdb768)
- improve documentation about prediction processing (ab75c09)
1.28.0 (2023-05-02)
- add UPC Image detector (f592acd)
- add question formatter for is_upc_image insights (bb6a6da)
- enforce max doc length of 79 with flake8 (d7d58d6)
- improve import-logo-embeddings CLI command (34139d1)
- remove keras category classifier 2.0 model (4038a6e)
- remove legacy model download commands (4a5ef8f)
- remove Tensorflow Serving entirely (f260887)
- set predictor_version and predictor for is_upc_image preds/insights (a9eefc5)
1.27.2 (2023-04-28)
- fix import_insights CLI command (d151b3d)
1.27.1 (2023-04-28)
- make ServerType inherit from str (4f4244c)
1.27.0 (2023-04-27)
- introduce prediction deletion (3dc0a44)
- add missing test JSON data (d885729)
- create robotoff.utils.text module (5d2bc86)
- don't raise error when getting bounding box by default (9208b42)
- fix mypy typing issues (107cc23)
- fix span offset issue when case_sensitive=False (15eb2f8)
- integrate flashtext into robotoff codebase (3ec9979)
- remove debug log message (bc80eee)
1.26.1 (2023-04-26)
- fix NutritionImageImporter.is_conflicting_insight (581376e)
- fix typo in nutrition-table.md (3357adc)
1.26.0 (2023-04-24)
- add nutrition_image insight type (27fd62c)
- add debug log message in importer.py (6c5febc)
- enable again nutrition table object detection (b529769)
- fix error raised when releasing expired lock (0859eef)
- fix Github action (3847226)
- fix issue in unit tests (271b2bb)
- fix issue with crop in
select_rotate_image(b40984d) - fix livecheck script (#1103) (aa1ab8a)
- fix type error in slack.py (d076be7)
- increase lock expire duration during insight import (to 5 min) (fb8b787)
- increase min score for nutrition image detector model (fd29693)
- try to trigger Github actions on push on master (08902b6)
- try to trigger Github actions on push on master (2) (bda2abf)
1.25.4 (2023-04-21)
- always use the same queue for jobs of the same product (e9f066c)
- fix typo in Makefile (6594035)
- improve Robotoff edit message (4b5f230)
- send webhook update jobs for all projects (7949dec)
- update DATASET_CHECK_MIN_PRODUCT_COUNT (e5555cc)
- use md5 hash function in
get_high_queue(29befd6)
- incorrect link in README.md predictions section (bb63afc)
- move a comment in docker-compose.yml (b687265)
- update maintenance.md (842eaf3)
1.25.3 (2023-04-16)
- improve scheduled job refresh_insight (79f68a9)
- remove unused server_domain method (2b43ff6)
- use world subdomain instead of api everywhere (09cb67e)
1.25.2 (2023-04-16)
- fix call to
update_product(16c6453)
1.25.1 (2023-04-16)
- display Product Opener response during OFF product update if an (91d05d4)
1.25.0 (2023-04-16)
- add a function to send image to OFF (f6ac894)
- implement real multi-platform support (OFF, OBF,...) (9464f46)
- support multiple MongoDB DB (multi-project) (2c36b6f)
- add
server_typefield to logo indexed in ES (506ab02) - add DISABLE_PRODUCT_CHECK settings (a201fae)
- fix in insert_images.py script (8dd914f)
- fix issue in settings (DISABLE_PRODUCT_CHECK value) (232e5c6)
- fix refresh-insight scheduled job (d363cf8)
- fix value for ENABLE_PRODUCT_CHECK in local env (20a58d5)
- fix wrong call to run_nutriscore_object_detection (a211249)
- fix wrongly formatted logging message (3ef4a0d)
- rename DISABLE_PRODUCT_CHECK into ENABLE_PRODUCT_CHECK (8c07478)
- rename en:gluten-free into en:no-gluten (e1f6417)
- replace call to lru_cache() by call to cache() (191faab)
- suppress mypy warnings (49a29b9)
- switch log level to DEBUG (d3ebd85)
- add missing docstring parameter descriptions (32cc8f8)
- improve docstrings (d4bb80b)
- improve documentation in add-predictor.md (195f1fd)
- improve documentation in add-predictor.md (90bfda2)
- improve Robotoff API documentation (d930592)
1.24.2 (2023-04-06)
- fix error in product weight insight (6866739)
- load lazily all resources in Robotoff (4dfa93f)
- move LogoLabelType to robotoff.types (0cc7efe)
- add documentation about how to add a predictor (d467a4c)
- add documentation about interaction with Product Opener (05c0781)
- add references to codebase in category-prediction.md (56ccfba)
1.24.1 (2023-04-05)
- fix incorrect offset in get_words_from_indices (547a867)
1.24.0 (2023-04-05)
- add function to get match bounding box (5d7eafa)
- allow to match text on OCRResult (73ae0e6)
- save bounding box information in OCR/flashtext predictions (9d4d432)
- add functions to delete/unselect an image (ade0294)
- allow partial match in get_words_from_indices (34cbd24)
- allow to match across blocks (c073129)
- bug fix in product weight insight generation (031e117)
- cache result word string in Word (30bc347)
- convert absolute coordinates to relative ones (e86e671)
- fix offset bug (a6e1e51)
- make error message easier to understand during HTTP 404 during OCR fetch (b352387)
- remove text_annotations OCRField and use new text field (f0637a4)
- save mapping between position of words and full annotation text (674ad77)
- use new computed text field in regex matching (b29e1db)
- use re.I flag instead of lowercasing string (0ecef5e)
- use strip_accents_v1 when necessary (f2ee677)
1.23.1 (2023-03-16)
- always select deepest categorized nodes in category importer (aba33f8)
- improve healthcheck status check messages (e24e136)
1.23.0 (2023-03-15)
- add missing_category campaign to track products without categories (b103e66)
- cache image embeddings in DB (ImageEmbedding table) (ba25c75)
- store neighbor categories for v3 categorizer models (8d1d727)
- use keras new v3 model as default to predict categories (c0f55cf)
- use keras v3 model as default (a2d23a3)
- use keras_image_embeddings_3_0 by default in categorize CLI (737716e)
- allow to specify LOG_LEVEL in .env file (50a3aa4)
- deprecate campaign parameter in /questions* (82c26fe)
- exclude some categories from predictions (503fa8d)
- fix bug in save_image_embeddings (a600502)
- fix bug that occurs when image are missing in images table (1968f56)
- fix edge-case bug when no image is available (fac1ab1)
- fix newly introduced bug in category importer (a1bb507)
- fix serialization bug in predict (1a69ab2)
- fix SonarCloud-detected bug in BaseURLProvider (290a3fb)
- fix unit tests (61c6177)
- fix unit tests (0a3702f)
- ignore predicted category if it no longer exist in taxonomy (c3c4fbe)
- move save_image function to new robotoff.images module (6efdaf2)
- pass stub as argument in predict for easier testing (807c157)
- relax checks in save_images function (bcf998f)
- remove legacy unit tests (6f3e5e2)
- add comment in build_triton_request function (cf1e8db)
- add documentation about category prediction (0a21476)
1.22.1 (2023-03-13)
- don't generate image embedding/fetch OCR texts if not required (93fc96d)
1.22.0 (2023-03-12)
- add model with image embeddings as input (d79bbc2)
- add authentication for .net Product Opener when fetching products (18cbb20)
- add object detection label assets to repository (c925558)
- add support for git LFS (d6db888)
- disable cat matcher (en) for partial matches (649c016)
- don't keep numpy ndarray in debug.inputs dict (64e7124)
- fix integration test (b2b4f5a)
- increase CLIP max_batch_size to 32 (a39d61a)
- refactor category predictor data structure (812a406)
- update code after code review #1061 (deb20d8)
- update PUT /images/logos/LOGO_ID route to accept null value field (2e488ab)
- add docstring (b4ace04)
- add documentation about install of git lfs (1cd9866)
- add documentation in metrics.py (0ce2b0e)
1.21.0 (2023-02-28)
- add new category classification models (60d167d)
- return debug information in /predict/category route (5b2b392)
- fix category predictions (347c72c)
- remove unused functions (5b0f1ae)
- silence false-positive mypy error (337777d)
- update /predict/category schema (f86c098)
- update poetry lock file (48d921c)
- improve documentation in v3 category predictor code (675e056)
1.20.2 (2023-02-06)
- adapt for autoblack (93e75ca)
- add new question to robotoff.pot (b13d505)
- manually cropping and resizing of logos before CLIP processor (2a660e0)
1.20.1 (2023-01-30)
- add packaging to default types for questions (8c2988f)
1.20.0 (2023-01-29)
- add detailed packaging detection (4916845)
- add packaging formatter and annotator (cd6ec75)
- add PackagingImporter (ce9af41)
- implement new packaging API (3e292f5)
- add brand to taxonomy exclude list (fe39f1d)
- add water brand to taxonomy exclude list (50cb021)
- increase memory limit of robotoff api service to 4G (9cb2454)
- increase MongoDB timeout from 5s to 10s (6fc1a0d)
- remove legacy code (b209707)
- add section about robotoff models (738cc9a)
- adding ann benchmark (ede4330)
- Adding ann-doc in Robotoff Technical References (4090568)
- changing the link of the code (eb7cd85)
- correcting mistakes in the doc (84a6370)
- english trad (2254126)
- fix typo (5bdcb33)
1.19.1 (2023-01-17)
1.19.0 (2023-01-16)
- allow users to filter by confidence in /questions (8af5719)
- fix UnboundLocalError exception (6c3ea9a)
1.18.2 (2023-01-10)
- create and annotate logo insights in job (0e825ed)
- display front image as default for regex-based brand insights (e898b1d)
- fix server_domain bug (80730e4)
- fix unit tests in test_question.py (a05d4ac)
- improve BaseURLProvider class (5178bac)
- improve import-logos CLI command (0e1e92b)
- remove CachedStore use in products.py (57b0970)
- use correct image subdomain for slack notifications (1345e94)
- use image.openfoodfacts.* as default server for serving images (70aceee)
1.18.1 (2022-12-30)
- delete robotoff.utils.types module (e8949da)
- fix /dump route (27dd472)
- fix unit and integration tests (ab17fd2)
- improve /insights/dump route (bc9029c)
- improve annotator classes and logo annotation tests (3a7fc7a)
- improve handling of deleted images in import_image.py (2d24a99)
- return xx name if exists in taxonomy.get_localized_name() (64c9175)
- save confidence score in Prediction.confidence (1aebdc5)
- turn a warning log into an info (f1fa4b9)
- use directly LogoAnnotation.{barcode,source_image} (e31cedb)
1.18.0 (2022-12-29)
- add DB columns (ffc0de8)
- fix condition check for logo processing (ba72538)
1.17.0 (2022-12-28)
- add new endpoint to reset logo annotation (089d289)
- create annotate function to centralize annotation (b55d72c)
- give credit to annotator when annotating logos (02f7e2e)
- honor limit parameter in all cases in run-object-detection CLI (4268939)
- move InsightType to robotoff.types (a5420c8)
- never delete annotated insights (8c67dbe)
- remove legacy log message (20aab5a)
- require auth for logo annotation (4420346)
1.16.7 (2022-12-27)
- add missing field in SQL query (e49af95)
1.16.6 (2022-12-27)
- add CLI to refresh all nearest neighbors (80fa554)
- allow to run object detection models on URL list file (6b04692)
- don't perform full update during insight update (671af07)
- don't refresh nearest neighbors of annotated logos (27fa3de)
- open DB connection when refreshing product insight (f8c9a36)
- use better default for MONGO_URI (5a392db)
1.16.5 (2022-12-27)
- fix /ann/search route (9016968)
1.16.4 (2022-12-26)
- don't open DB connection twice (3319c0b)
- generate insights from annotated logos after PUT logo/{logo_id} (e45bd3a)
- simplify call to /ann/search (1164f65)
1.16.3 (2022-12-26)
- bulk-index logo embeddings (instead of indexing one by one) (4768cfb)
- fix ES product export (ea9cd21)
- fix test_import_image integration test (2031fa4)
- remove automatic processing disabling (081f4a9)
1.16.2 (2022-12-26)
- add result_ttl=0 for upate_insight job (09862f1)
- don't perform image extraction jobs on invalid images (02458c4)
- fix add-logo-to-ann CLI command (8385689)
- remove useless log message (7ded92d)
1.16.1 (2022-12-26)
- check that brand prediction is not in blacklist during import (45bc014)
- improve logging messages for cached resources (844bfd3)
- limit CLIP batch size to 4 (ab0cc99)
- set TTL of 1h for get_logo_annotations function (28e2505)
1.16.0 (2022-12-23)
- Adding ES ANN for logos in robotoff (e4beaad)
- create elasticsearch indices at startup (17c0f17)
- disable temporarily logo processing (10bc089)
- save CLIP embeddings in DB (dbc4d34)
- add
query_logo_idfield in /ann/search response (698025c) - add integration tests to process_created_logos (0e02f7b)
- add robotoff-ann API to robotoff (895264a)
- add robotoff.triton module (5fe1332)
- add updated_at field to logo.nearest_neighbors JSON (db0c44d)
- cast logo_id to int in ES ANN response (abeec15)
- fix add-logo-to-ann CLI command (52ae430)
- fix ES healthcheck (1b54761)
- fix integration tests (316d330)
- fix LogoEmbedding model backref name (b8f2d04)
- fix process_created_logos function (a3aa2c4)
- fix run_object_detection_model CLI command (1480350)
- fix wrong logging level for error messages (9002356)
- improve add-logo-to-ann CLI command (a6c6fc7)
- Improve export-logos CLI command (8441163)
- improve run_object_detection model function (a8040c5)
- move knn_search to distinct function (aa7eff6)
- normalize vector before ANN query (d6f18af)
- pyproject-fix (daaed91)
- remove ANN healthcheck and add an healthcheck for ES (b962659)
- remove legacy export_logo_annotation CLI command (0a3ffdc)
- switch log messages from WARNING to INFO (89c0933)
- use good defaults for ES config in settings.py (71320a9)
1.15.2 (2022-12-21)
- add nutriscore predictor for nutriscore model predictions (3a61b24)
- catch ConnectionError from requests.exceptions and not from stdlib (2357d3e)
- improve request exception handling (36b675e)
- improve request exception handling (b7e3665)
- improve Slack notification request exception handling (41d37f6)
1.15.1 (2022-12-15)
- add better request error handling during annotations (e60a719)
- add better request error handling in process_created_logos (c9b6d9e)
- don't notify on Slack when annotating logos (3503425)
- Don't overwrite annotations in /images/logos/annotate (9d82321)
- fix auth error on OFF due to Smoothie (368b07b)
- fix IndexError when no nutriscore is detected (55d4829)
- improve get_image_from_url exception handling (c5d0b3e)
- improve HTTPError handling during insight annotation (cb9c47a)
- make logging call uniform (390e144)
- remove en:nutriscore detection alert (d201d90)
- remove nutriscore Slack alerts (160c6ad)
- retry 3 times when getting an image/static resource from OFF (901ed97)
- switch to INFO level for most warning log messages in save_images (717471d)
1.15.0 (2022-12-13)
- add a lock during product update jobs (a94632e)
- add more logs during resource loading (7aedd8d)
- add new CLI commands to launch background tasks (6c3184e)
- add redis lock to avoid concurrent insight import for the same product (f596c97)
- add two kind of workers: worker_high and worker_low (f5eddf0)
- create more atomic tasks during image import (69497f1)
- don't process logos when running batch logo detector model (24ed944)
- enable
python -m robotoffto call CLI (7286053) - improve CLI import commands (f5491d1)
- improve generate-ocr-predictions CLI command (15ef7c3)
- improve import-insights CLI command (7766feb)
- make object detection jobs idempotent (a8f002e)
- only accept predictions from one product during insight import (58cf2d4)
- refresh cache when performing worker maintenance tasks (240490b)
- remove orphan containers when doing make up (6452db7)
- save brand prefix as gzipped file (c7c8322)
- simplify image import process (ec272a3)
- update brand_from_taxonomy.gz (1f5cee8)
- update prediction duplicate detection mecanism (ef658ca)
- use batches in refresh_insights job (66fea3b)
- use built-in types instead of typing.* (289dc17)
- use queues for refresh_insight job on all DB (71e5319)
- use robotoff.types.PredictionType enum in Robotoff CLI (032213f)
- use rq to send tasks to workers (7163610)
- add default TF_SERVING_HOST in .env file (21c0760)
- add default timeout during TF serving request (9aaceea)
- add predictor=regex for store predictions (ecac885)
- add with_db decorator to generate_fiber_quality_facet (c62f14c)
- disable deepsource autofix (ace1df3)
- disable INFO log message from selected dependencies (b5eeeab)
- don't refresh ES indexes during scheduler startup (fa72748)
- fix DB connection in workers (2333b41)
- fix flake8 and isort errors (7ed93fc)
- fix integration tests after adding autoconnect=False (0382e35)
- fix integration tests after switch to manual DB connection management (2f3e8f4)
- fix wrong import (15894e5)
- improve DB transaction management during image import (a2e390a)
- move PredictionType to robotoff/types.py (7f374e6)
- move types required by CLI in robotoff/types.py (d66d7e0)
- no need to use db.atomic() when db context manager is used (f24a600)
- open DB connection when needed in scheduler (3e1ec39)
- remove CLI documentation generation (4258748)
- remove legacy script comments (2d96e7a)
- remove legacy test file (21c39c5)
- revert container-deploy-ml.yml update (667d19d)
- switch from redis-stack to classic redis (123fcbf)
- Update Makefile (0c90461)
- update poetry lock file (1cfd0c2)
- updating log level of a log message (ac2a479)
- use autoconnect=False during DB connection (683ba6a)
- use db.connection_context in with_db decorator (bac669c)
- use enqueue_job function everywhere (0dc53c7)
- validate params in GET /images/logos route (b265686)
- add docstrings (4771657)
- improve documentation (7e43b4d)
- update documentation in maintenance.md (64d495d)
1.14.0 (2022-12-06)
- add insights metric (9cc8bb6)
- add misc metrics to InfluxDB (e8e0633)
- add percent field to influx 'insights' measurement (8308027)
- remove update_recycle task (9884ef4)
- add lower value for MongoDB serverSelectionTimeoutMS (8d204c1)
- add mongodb service in webnet network (7714d1b)
- clean obsolete CLI commands (fba622d)
- don't build robotoff image by default (ca8f486)
- fix default value for MONGO_URI in .env (4229c94)
- fix mongoDB healthcheck (c91761a)
- fix parameter typing in one API route (ca19c7c)
- revert deletion of CLI run command (5d0bbeb)
- update barcode range check during insight import (090c746)
- add documentation about predictions and insights (3c43229)
- fix error in dev-install.md (1ef8d33)
- fix typo (aca7a39)
- update make dev installation message (705c580)
1.13.0 (2022-11-25)
- disable extraction of
packagingpredictions (0b68fd2)
- fix mypy and flake8 issues (80fef79)
1.12.0 (2022-11-24)
- switch to InfluxDB v2 (3161015)
- add proper authentication in metrics.py for .net env (5e1ccb9)
- disable grafana annotation until grafana is up and running again (3415c48)
- fix ISO_3166-1 alpha-2 codes used in metrics.py (2224915)
- really catch JSONDecodeError exceptions in metrics.py (5c65ee9)
- use same version for requests and types-requests (b13160e)
1.11.0 (2022-11-15)
- allow to filter insights/question by predictor value (6f840d6)
- consider annotation=-1 as a vote (#908) (18e9552)
- remove dependabot (b8df1aa)
- set value_tag to canonical label value during prediction import (80cf93c)
- switch all object detection models to Triton (#622) (3c786c4)
- add a small fix on dump_ocr.py (450d53f)
- add triton HTTP port in env file (11395c6)
- always serialize insights the same way (4a00f48)
- fix codecov configuration (56ee4b0)
- fix min_confidence parameter in /logos/search route (b00ad84)
- fix refresh_all_insights function (43d6078)
- fix triton model dir volume binding (d8ccffa)
- make API parameters uniform (80ad784)
- make voting mechanism work again (4f9d499)
- remove automatic parameter in InsightImporter (1c11652)
- update brand taxonomy blacklist (4ea5b0c)
- revert codecov config change (850aea1)
1.10.0 (2022-10-25)
- add
data->bounding_boxfield in logo derived insights (1e18b2e) - add build command to Makefile (5c3961d)
- add CLI command export-logos-ann (add6141)
- allow to launch a single service with make (92b01d9)
- improve
generate_predictionfunction (52247f9) - improve apply-insight CLI command (93524c6)
- improve JSON OCR generation script (603e355)
- improve taxonomized value matching (52c99be)
- update category matching algorithm (#952) (d8a04c7)
- update OCR scripts (111ada9)
- add fixes to category matcher (b8c1912)
- don't return auto processable insights in /questions/{barcode} (03376b8)
- filter logos that are almost exactly the same (33cfc88)
- fix /logos/search route (e932407)
- fix
get_tagfunction (cca5592) - fix insight import mecanism (#963) (04412df)
- rename image_url field into ref_image_url (3539d84)
- rename ocr_dump.py script into dump_ocr.py (bcce867)
- require insight_id to be an UUID in /insights/annotate (abb9574)
- simplify filter_logo function (9eb8fdd)
- update .gitignore and .dockerignore (2eb985a)
- update tags of some labels detected with flashtext (f2bd704)
- use canonical value fr:label-rouge everywhere (05197cc)
- use f-string everywhere in robotoff/cli/insights.py (bbd81fa)
1.9.0 (2022-10-17)
- improve request validation during logo annotation (7a1e6d5)
- Request image directly from MongoDB instead of Product Opener (#921) (83f0f09)
- support providing taxonomized value as input during logo annotation (53a9bdf)
- use gzipped version of fallback taxonomy files (73e23ce)
- allow to fetch both annotated/not annotated logos in /images/logos/search (f9e5e96)
- create new /images/logos/search endpoint from /images/logos (20034d8)
- ignore protobuf-generated files during mypy analysis (e3d84ae)
- remove .gz files from .gitignore (0af8a34)
- set higher expiration interval for taxonomy (c15e077)
1.8.0 (2022-10-12)
- Add
agribalyse-categorycampaign to agribalyse category insights (69d0023) - add a
thresholdparameter to /predict/category endpoint (0f68e93) - add campaign filter in question endpoints (91ed2d2)
- add ProductInsight.campaign field (4a54484)
- Improve /predict/category endpoint (0438f4e)
- add fixes to /questions/unanswered endpoint (6ed2d42)
- adding a benchmark done with cosine-distance to the research doc (#945) (f189031)
- adding documentation about the insights/annotate.py file (#944) (18e5c66)
- don't display question about insights that are automatically applicable (5ac392e)
- don't return in /questions/unanswered reserved barcode by default (78af005)
- remove some moderation cloud vision labels (1daa889)
1.7.0 (2022-10-04)
- add to repository latest versions of OCR scripts (#920) (be44e81)
- added country parameter to the API and started with test cases (12b2359)
- added server_domain to the API (#899) (761aa51)
- Adding MongoDB container to Robotoff in dev (#693) (946ce1d)
- expose postgres DB locally (2668f2a)
- Extract USDA packager codes with REGEX and flashtext (aa2b8ec)
- Extract USDA packager codes with REGEX and flashtext (e92163f)
- Robotoff quality monitoring: Saving AnnotationResult (#796) (755d296)
- adapting REGEX to codes like M123 + V123 (470b4d2)
- Adding documentation and reviewing syntaxes (e296f17)
- changing legacy file not to get an error in the typing check mypy (6f1f113)
- changing legacy file not to get an error in the typing check mypy (8b5bcae)
- Changing names from category_from_AOC to category (c490669)
- docstring and file name (3114aea)
- dumping an unused import (5c92e59)
- fix flake8 errors on scrits/ocr/extract_ocr_text.py (aef38df)
- fixed the test case (1b6fb0d)
- Incompatible return value (1b4453b)
- Incompatible return value type (2dc7831)
- incorrect brand taxonomy fallback path + fix tests (13baec4)
- move docker mongodb service to a distinct file in dev mode (#916) (c283499)
- only save annotation_result in ProductInsight when needed (#938) (c14b07f)
- Outdated commentaries (2bdc26f)
- remove trailing slash in all URLs (#915) (5e89e95)
- replace with fr prefix references to en:ab-agriculture-biologique (#919) (ec1500c)
- solving mypy check issue (3a5f9ab)
- store category neural model in ProductInsight.predictor field (#914) (c1c8d8d)
- temporarily disable USDA packager code extraction (#933) (91f65c1)
- typing.dict unused (90112b7)
- unused import (cdab35f)
- Unused import (43930b1)
- Unused variables in a loop (d4a37f1)
- Unused variables in a loop (a30b964)
1.6.0 (2022-09-12)
- add an edit and remove button when nutriscore prediction is posted on Slack channel (#783) (0055ba7)
- add events API requests (#677) (1f212fd)
- Add image moderation service (#889) (40d4ea4)
- Adding a regex for gluten packaging code (#823) (d22ee99)
- api returning predictions (#815) (a9c9be8)
- detect cat images (opff) (#883) (b2b1b00)
- filter images and display in a list (#832) (e9bcb58)
- Filter insights opportunities based on type of tags (#859) (bc5ee1b)
- Get Logo Annotation list (#882) (965b409)
- images predictions collection api (#834) (1477943)
- isolate test network + make single test run (#806) (3618cb1)
- Sort collection API (#888) (275ef95)
- test voting first anonymous then authenticated (#805) (61a24cd), closes #801
- action name (4d88ca1)
- Added server_domain to all "Collection" class (#887) (f3bcd85)
- create docker network in Makefile + docs (#770) (b5ab0e7)
- fix click dependency in autoblack (151bd25)
- fix tests + some i18n utils (#799) (17619ec)
- fix tests date to use utc (#686) (dd76fc0)
- improve the bug report template (72e2073)
- improve the Feature Request template (f7d9a24)
1.5.1 (2022-03-28)
- api: Question API, fix query randomness, and allow to use pagination (#666) (5a7fd71)
- fetch_taxonomy should check response status (26f318e)
- fix log message for sentry grouping (bbf2749)
- wrong error message on exception in fetch taxonomy (5b9252d)
1.5.0 (2022-03-18)
- change error messages to enable sentry grouping (#654) (15abbeb)
- fix timeout issue on fetch taxonomy (#656) (a91ba4e)
1.4.0 (2022-03-16)
- disable auto processing of predicted category (8455868)
- logo annotation propagation to insight (2a57a7d)
- add test on annotation vote cascade (2641e25)
- avoid process_insight failing for all (69aeb63), closes #605
- avoid raising in ObjectDetectionRawResult (c2ab5f6), closes #621
- fix notification message for categories (cf6a675), closes #614
- test logo annotation (e265ce6)
- some typos fixes (6f8ebe0)
1.3.0 (2022-02-22)
- continue structural changes to have insights derived from predictions
1.2.0 (2022-02-07)
- add value_tag in the question formater (#579) (84239f1)
- make neural category apply automatically (#555) (2146f78), closes #552
- ensure influxdb database exists (#559) (1f7a9ee)
- scheduler and tasks in isolated transactions - fixes #608 (#609) (c5d44a4)
- set num workers in gunicorn from env (#563) (0290964)
- set num workers in gunicorn from env (#563) (#564) (89c1c13)
- straight conditions for category insights (#570) (1d44643)