edge-alpine DB fails with an HEIC image for media_entity.metadataSizeWidth constraint #877

malavolti · 2024-04-05T15:00:57Z

Describe the bug

If I try to load this image, I receive the following error on SQLLite DB and no images are loaded on my piGallery2.

Photo/video (optional) that causes the bug

Server logs (optional)

QueryFailedError: SqliteError: NOT NULL constraint failed: media_entity.metadataSizeWidth
    at BetterSqlite3QueryRunner.query (/app/node_modules/typeorm/driver/better-sqlite3/BetterSqlite3QueryRunner.js:103:19)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async InsertQueryBuilder.execute (/app/node_modules/typeorm/query-builder/InsertQueryBuilder.js:106:33)
    at async SubjectExecutor.executeInsertOperations (/app/node_modules/typeorm/persistence/SubjectExecutor.js:260:42)
    at async SubjectExecutor.execute (/app/node_modules/typeorm/persistence/SubjectExecutor.js:92:9)
    at async EntityPersistExecutor.execute (/app/node_modules/typeorm/persistence/EntityPersistExecutor.js:148:21)
    at async IndexingManager.saveChunk (/app/src/backend/model/database/IndexingManager.js:404:32)
    at async IndexingManager.saveMedia (/app/src/backend/model/database/IndexingManager.js:329:9)
    at async IndexingManager.saveToDB (/app/src/backend/model/database/IndexingManager.js:95:13)
    at async IndexingManager.queueForSave (/app/src/backend/model/database/IndexingManager.js:128:17) {
  query: 'INSERT INTO "media_entity"("id", "name", "type", "directoryId", "metadataCaption", "metadataTitle", "metadataCreationdate", "metadataCreationdateoffset", "metadataFilesize", "metadataKeywords", "metadataRating", "metadataFaces", "metadataPersons", "metadataPersonslength", "metadataBitrate", "metadataDuration", "metadataSizeWidth", "metadataSizeHeight", "metadataCameraDataIso", "metadataCameraDataModel", "metadataCameraDataMake", "metadataCameraDataFstop", "metadataCameraDataExposure", "metadataCameraDataFocallength", "metadataCameraDataLens", "metadataPositionDataCountry", "metadataPositionDataState", "metadataPositionDataCity", "metadataPositionDataGPSDataLatitude", "metadataPositionDataGPSDataLongitude") VALUES (NULL, ?, ?, 344, NULL, NULL, 1706438594000, 180, 2158564, NULL, NULL, NULL, NULL, 0, NULL, NULL, ?, ?, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, 9.061331, 38.761711)',
  parameters: [ '2024-01-28_13.43.14.heic', 'PhotoEntity', null, null ],
  driverError: SqliteError: NOT NULL constraint failed: media_entity.metadataSizeWidth
      at BetterSqlite3QueryRunner.query (/app/node_modules/typeorm/driver/better-sqlite3/BetterSqlite3QueryRunner.js:85:38)
      at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
      at async InsertQueryBuilder.execute (/app/node_modules/typeorm/query-builder/InsertQueryBuilder.js:106:33)
      at async SubjectExecutor.executeInsertOperations (/app/node_modules/typeorm/persistence/SubjectExecutor.js:260:42)
      at async SubjectExecutor.execute (/app/node_modules/typeorm/persistence/SubjectExecutor.js:92:9)
      at async EntityPersistExecutor.execute (/app/node_modules/typeorm/persistence/EntityPersistExecutor.js:148:21)
      at async IndexingManager.saveChunk (/app/src/backend/model/database/IndexingManager.js:404:32)
      at async IndexingManager.saveMedia (/app/src/backend/model/database/IndexingManager.js:329:9)
      at async IndexingManager.saveToDB (/app/src/backend/model/database/IndexingManager.js:95:13)
      at async IndexingManager.queueForSave (/app/src/backend/model/database/IndexingManager.js:128:17) {
    code: 'SQLITE_CONSTRAINT_NOTNULL'
  },
  code: 'SQLITE_CONSTRAINT_NOTNULL'
}

Environment (please complete the following information):

OS: Server runs on Linux Debian 12 with Docker, FireFox browser on Linux Debian 12
Browser: FireFox 122.0.1

Used app version:

docker-edge (alpine)

The text was updated successfully, but these errors were encountered:

grasdk · 2024-04-05T22:14:35Z

@malavolti I found the problem and a solution. May I use the picture you provided as test-material in the project (adding it to /test/backend/assets/)?

malavolti · 2024-04-05T22:16:26Z

@malavolti I found the problem and a solution. May I use the picture you provided as test-material in the project (adding it to /test/backend/assets/)?

Yes, sure! 😊

grasdk · 2024-04-06T08:44:35Z

@malavolti Hey again

I upgraded the image-size version in #879, but I actually found out that it incorrectly reports the dimensions of your photo as 512x512. It fixes the problem with null in pigallery, but it's still not correct.

Would you mind me submitting the photo in a bug-report over at https://github.com/image-size/ ?

malavolti · 2024-04-06T09:43:51Z

@malavolti Hey again

I upgraded the image-size version in #879, but I actually found out that it incorrectly reports the dimensions of your photo as 512x512. It fixes the problem with null in pigallery, but it's still not correct.

Would you mind me submitting the photo in a bug-report over at https://github.com/image-size/ ?

Yes, no problem. Thank you! 😊

grasdk · 2024-04-13T10:02:05Z

@malavolti If you have the opportunity to install the latest edge-version. The issue should be resolved by that. Otherwise, please let me know ;)

malavolti · 2024-04-13T10:03:19Z

I'll try as soon as possible! 😊 Thank you for all!

bpatrik · 2024-04-14T13:13:57Z

NOTE: builds are failing in the last few days(weeks):

So edge might not have the new fix-feature.

grasdk · 2024-05-12T21:53:39Z

@bpatrik I think this is merged, so should probably be added to the "Next (probably v2.5)" milestone?

bpatrik · 2024-05-15T21:40:15Z

Can someone confirm that the original issue was fixed?

grasdk · 2024-05-16T12:59:34Z

Maybe @malavolti is still around to test latest edge build?

malavolti · 2024-05-16T22:25:19Z

Dear @grasdk I've test the only the image that I shared with you in the past and the latest edge build fails with the same error of the issue I have opened in April.

2024-01-28_13.43.14.heic.txt

5/16/2024, 10:19:52 PM[INFO_][server] running diagnostics...
5/16/2024, 10:19:52 PM[INFO_][SQLConnection] Updating database scheme
5/16/2024, 10:19:52 PM[WARN_][ConfigDiagnostics] Jobs error. Resetting to default for now to let the app start up. Please adjust the config properly. Error: Unknown Job :Thumbnail Generation
5/16/2024, 10:19:52 PM[INFO_][JobManager] Running job schedules
5/16/2024, 10:19:52 PM[INFO_][server] Listening on port 80
QueryFailedError: SqliteError: NOT NULL constraint failed: media_entity.metadataSizeWidth
    at BetterSqlite3QueryRunner.query (/app/node_modules/typeorm/driver/better-sqlite3/BetterSqlite3QueryRunner.js:103:19)
    at async InsertQueryBuilder.execute (/app/node_modules/typeorm/query-builder/InsertQueryBuilder.js:106:33)
    at async SubjectExecutor.executeInsertOperations (/app/node_modules/typeorm/persistence/SubjectExecutor.js:260:42)
    at async SubjectExecutor.execute (/app/node_modules/typeorm/persistence/SubjectExecutor.js:92:9)
    at async EntityPersistExecutor.execute (/app/node_modules/typeorm/persistence/EntityPersistExecutor.js:148:21)
    at async IndexingManager.saveChunk (/app/src/backend/model/database/IndexingManager.js:400:20)
    at async IndexingManager.saveMedia (/app/src/backend/model/database/IndexingManager.js:329:9)
    at async IndexingManager.saveToDB (/app/src/backend/model/database/IndexingManager.js:95:13)
    at async IndexingManager.queueForSave (/app/src/backend/model/database/IndexingManager.js:128:17) {
  query: 'INSERT INTO "media_entity"("id", "name", "type", "directoryId", "metadataCaption", "metadataTitle", "metadataCreationdate", "metadataCreationdateoffset", "metadataFilesize", "metadataKeywords", "metadataRating", "metadataFaces", "metadataPersons", "metadataPersonslength", "metadataBitrate", "metadataDuration", "metadataSizeWidth", "metadataSizeHeight", "metadataCameraDataIso", "metadataCameraDataModel", "metadataCameraDataMake", "metadataCameraDataFstop", "metadataCameraDataExposure", "metadataCameraDataFocallength", "metadataCameraDataLens", "metadataPositionDataCountry", "metadataPositionDataState", "metadataPositionDataCity", "metadataPositionDataGPSDataLatitude", "metadataPositionDataGPSDataLongitude") VALUES (NULL, ?, ?, 1, NULL, NULL, 1706438594000, 180, 2158564, NULL, NULL, NULL, NULL, 0, NULL, NULL, ?, ?, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, 9.061331, 38.761711)',
  parameters: [ '2024-01-28_13.43.14.heic', 'PhotoEntity', null, null ],
  driverError: SqliteError: NOT NULL constraint failed: media_entity.metadataSizeWidth
      at BetterSqlite3QueryRunner.query (/app/node_modules/typeorm/driver/better-sqlite3/BetterSqlite3QueryRunner.js:85:38)
      at async InsertQueryBuilder.execute (/app/node_modules/typeorm/query-builder/InsertQueryBuilder.js:106:33)
      at async SubjectExecutor.executeInsertOperations (/app/node_modules/typeorm/persistence/SubjectExecutor.js:260:42)
      at async SubjectExecutor.execute (/app/node_modules/typeorm/persistence/SubjectExecutor.js:92:9)
      at async EntityPersistExecutor.execute (/app/node_modules/typeorm/persistence/EntityPersistExecutor.js:148:21)
      at async IndexingManager.saveChunk (/app/src/backend/model/database/IndexingManager.js:400:20)
      at async IndexingManager.saveMedia (/app/src/backend/model/database/IndexingManager.js:329:9)
      at async IndexingManager.saveToDB (/app/src/backend/model/database/IndexingManager.js:95:13)
      at async IndexingManager.queueForSave (/app/src/backend/model/database/IndexingManager.js:128:17) {
    code: 'SQLITE_CONSTRAINT_NOTNULL'
  },
  code: 'SQLITE_CONSTRAINT_NOTNULL'
}

Rename the file to get the image.heic

I've used the following docker-compose.yml and simply added heic and heif to supportedFormats into config.json file:

services:
  pigallery2:
    image: bpatrik/pigallery2:edge-alpine
    container_name: pigallery2
    environment:
      - NODE_ENV=production # set to 'debug' for full debug logging
      # - NODE_OPTIONS=--enable-source-maps # enable source map support on the backend for development
    volumes:
      - "./config:/app/data/config"
      - "./db-data:/app/data/db"
      - "./images:/app/data/images:rw"
      - "./tmp:/app/data/tmp"
    ports:
      - "85:80"
    restart: always

Thank you so much!
Marco

grasdk · 2024-05-17T09:01:51Z

Hey @malavolti

Thank you for the input. Can you show the version of pigallery (click settings, hover over the app-version):

mine is in Danish locale.

malavolti · 2024-05-17T12:09:22Z

Hey @malavolti

Thank you for the input. Can you show the version of pigallery (click settings, hover over the app-version):

mine is in Danish locale.

grasdk · 2024-05-17T13:56:24Z

I see the build date is 5th of april. Before the fix. You may need to clear the edge build image from your docker installation to force an update.

malavolti · 2024-05-24T12:33:39Z

I can say that with this version I can see the HEIC images, but all of them are compressed to the resolution of 512x512. :(

grasdk · 2024-05-25T13:56:13Z

@malavolti thanks. So the issue here is resolved.

But there is a new issue. However, it's about the image-size library. I reported the bug over there: image-size/image-size#403
@bpatrik should we raise an issue here for pigallery "HEIC image size reported wrongly"? with a description linking to the above bug-report?

bpatrik · 2024-06-28T20:53:13Z

Yeah we should track the HEIC image size issue somewhere.

grasdk · 2024-06-29T06:32:11Z

@bpatrik ok, done: #918 . I'll make a more comprehensive description if you would like. For now I just link to the issue with the image-size library.

@malavolti do you agree that we can close this issue, given that there is another issue now for tracking the 512x512-problem?

malavolti · 2024-06-29T07:21:36Z

@bpatrik ok, done: #918 . I'll make a more comprehensive description if you would like. For now I just link to the issue with the image-size library.

@malavolti do you agree that we can close this issue, given that there is another issue now for tracking the 512x512-problem?

Yes, sure. I agree. Thank you for all! 😊

grasdk · 2024-06-29T08:36:29Z

@malavolti thank you too. To think out in open issues, I would suggest closing the issue if you have access to that. Otherwise maybe @bpatrik can do it ;)

grasdk mentioned this issue Apr 5, 2024

updated image-size dependency to read dimensions from heic file #879

Merged

bpatrik added the bug label May 15, 2024

bpatrik added this to the Next (probably v2.5) milestone May 15, 2024

grasdk mentioned this issue Jun 29, 2024

HEIC image dimensions reported wrong for HEIC files with multiple images embedded #918

Open

malavolti closed this as completed Jun 29, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

edge-alpine DB fails with an HEIC image for media_entity.metadataSizeWidth constraint #877

edge-alpine DB fails with an HEIC image for media_entity.metadataSizeWidth constraint #877

malavolti commented Apr 5, 2024

grasdk commented Apr 5, 2024

malavolti commented Apr 5, 2024

grasdk commented Apr 6, 2024 •

edited

Loading

malavolti commented Apr 6, 2024

grasdk commented Apr 13, 2024

malavolti commented Apr 13, 2024

bpatrik commented Apr 14, 2024

grasdk commented May 12, 2024

bpatrik commented May 15, 2024

grasdk commented May 16, 2024

malavolti commented May 16, 2024 •

edited

Loading

grasdk commented May 17, 2024

malavolti commented May 17, 2024

grasdk commented May 17, 2024

malavolti commented May 24, 2024

grasdk commented May 25, 2024 •

edited

Loading

bpatrik commented Jun 28, 2024

grasdk commented Jun 29, 2024 •

edited

Loading

malavolti commented Jun 29, 2024

grasdk commented Jun 29, 2024

edge-alpine DB fails with an HEIC image for media_entity.metadataSizeWidth constraint #877

edge-alpine DB fails with an HEIC image for media_entity.metadataSizeWidth constraint #877

Comments

malavolti commented Apr 5, 2024

grasdk commented Apr 5, 2024

malavolti commented Apr 5, 2024

grasdk commented Apr 6, 2024 • edited Loading

malavolti commented Apr 6, 2024

grasdk commented Apr 13, 2024

malavolti commented Apr 13, 2024

bpatrik commented Apr 14, 2024

grasdk commented May 12, 2024

bpatrik commented May 15, 2024

grasdk commented May 16, 2024

malavolti commented May 16, 2024 • edited Loading

grasdk commented May 17, 2024

malavolti commented May 17, 2024

grasdk commented May 17, 2024

malavolti commented May 24, 2024

grasdk commented May 25, 2024 • edited Loading

bpatrik commented Jun 28, 2024

grasdk commented Jun 29, 2024 • edited Loading

malavolti commented Jun 29, 2024

grasdk commented Jun 29, 2024

grasdk commented Apr 6, 2024 •

edited

Loading

malavolti commented May 16, 2024 •

edited

Loading

grasdk commented May 25, 2024 •

edited

Loading

grasdk commented Jun 29, 2024 •

edited

Loading