Unable to initialize reverse geocoding: Error: timeout exceeded when trying to connect

[michalkrol]

The bug

Hi,

After the update to 1.123.0, the server app crashes soon after the start. the process is killed with the error trace.
Fix: Rollback to v1.122.2

Setup:
X86_64, 12GB RAM
Debian 6.1.119-1 (2024-11-22) x86_64
Docker version 27.4.0, build bde2b89
Docker Swarm Stack,
Access through Cloudflare Tunnel, with a docker network.
Storage for uploads UPLOAD_LOCATION and EXTERNAL_LIBRARY: NFS mounted on the Host level
Storage for DB_DATA_LOCATION and config: NFS mounted on the Host level

The OS that Immich Server is running on

Debian 6.1.119-1 (2024-11-22) x86_64 + Docker version 27.4.0

Version of Immich Server

v1.123.0

Version of Immich Mobile App

v1.120

Platform with the issue

• Server
• Web
• Mobile

Your docker-compose.yml content

version: "3"

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - ${EXTERNAL_LIBRARY}:/immich_external_library
      - /etc/localtime:/etc/localtime:ro
    ports:
      - '2283:2283'
    depends_on:
      - redis
      - database
    restart: always
    healthcheck:
      disable: false

  immich-machine-learning:
    container_name: immich_machine_learning
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    volumes:
      - model-cache:/cache
    ports:
      - 3003:3003
    restart: always
    healthcheck:
      disable: false
    deploy:
      replicas: 2

  redis:
    container_name: immich_redis
    image: docker.io/redis:6.2-alpine@sha256:eaba718fecd1196d88533de7ba49bf903ad33664a92debb24660a922ecd9cac8
    healthcheck:
      test: redis-cli ping || exit 1
    restart: always
    
  database:
    container_name: immich_postgres
    image: docker.io/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      POSTGRES_INITDB_ARGS: '--data-checksums'
    volumes:

      - ${DB_DATA_LOCATION}:/var/lib/postgresql/data
    healthcheck:
      test: >-
        pg_isready --dbname="$${POSTGRES_DB}" --username="$${POSTGRES_USER}" || exit 1;
        Chksum="$$(psql --dbname="$${POSTGRES_DB}" --username="$${POSTGRES_USER}" --tuples-only --no-align
        --command='SELECT COALESCE(SUM(checksum_failures), 0) FROM pg_stat_database')";
        echo "checksum failure count is $$Chksum";
        [ "$$Chksum" = '0' ] || exit 1
      interval: 5m

    command: >-
      postgres
      -c shared_preload_libraries=vectors.so
      -c 'search_path="$$user", public, vectors'
      -c logging_collector=on
      -c max_wal_size=2GB
      -c shared_buffers=512MB
      -c wal_compression=on
    restart: always
    
volumes:
  model-cache:

Your .env content

IMMICH_VERSION=v1.123.0

Reproduction steps

Update the release version to v1.123.0
Start the Stack
App crashes soon after start

Relevant log output

Initializing Immich v1.123.0
Detected CPU Cores: 4
Starting api worker
Starting microservices worker
(node:7) [DEP0060] DeprecationWarning: The `util._extend` API is deprecated. Please use Object.assign() instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
(node:7) [DEP0060] DeprecationWarning: The `util._extend` API is deprecated. Please use Object.assign() instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
(node:17) [DEP0060] DeprecationWarning: The `util._extend` API is deprecated. Please use Object.assign() instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
[Nest] 7  - 12/19/2024, 10:22:10 AM     LOG [Microservices:EventRepository] Initialized websocket server
[Nest] 7  - 12/19/2024, 10:22:11 AM     LOG [Microservices:MetadataService] Bootstrapping metadata service
[Nest] 7  - 12/19/2024, 10:22:11 AM     LOG [Microservices:MetadataService] Initializing metadata service
[Nest] 7  - 12/19/2024, 10:22:11 AM     LOG [Microservices:MapRepository] Initializing metadata repository
[Nest] 17  - 12/19/2024, 10:22:11 AM     LOG [Api:EventRepository] Initialized websocket server
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:ServerService] Feature Flags: {
  "smartSearch": true,
  "facialRecognition": true,
  "duplicateDetection": true,
  "map": true,
  "reverseGeocoding": true,
  "importFaces": false,
  "sidecar": true,
  "search": true,
  "trash": true,
  "oauth": false,
  "oauthAutoLaunch": false,
  "passwordLogin": true,
  "configFile": false,
  "email": false
}
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:StorageService] Verifying system mount folder checks, current state: {"mountFiles":true,"mountChecks":{"thumbs":true,"upload":true,"backups":true,"library":true,"profile":true,"encoded-video":true}}
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:StorageService] Successfully verified system mount folder checks
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:VersionService] Version has changed, adding 1.123.0 to history
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:SystemConfigService] LogLevel=debug (set via system config)
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:NestFactory] Starting Nest application...
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:InstanceLoader] BullModule dependencies initialized
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:InstanceLoader] OpenTelemetryModule dependencies initialized
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:InstanceLoader] TypeOrmModule dependencies initialized
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:InstanceLoader] OpenTelemetryCoreModule dependencies initialized
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:InstanceLoader] DiscoveryModule dependencies initialized
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:InstanceLoader] ClsModule dependencies initialized
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:InstanceLoader] ScheduleModule dependencies initialized
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:InstanceLoader] BullModule dependencies initialized
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:InstanceLoader] BullModule dependencies initialized
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:InstanceLoader] TypeOrmCoreModule dependencies initialized
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:InstanceLoader] TypeOrmModule dependencies initialized
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:InstanceLoader] ApiModule dependencies initialized
[Nest] 17  - 12/19/2024, 10:22:12 AM   DEBUG [Api:ClsModule] Mounting ClsMiddleware to *
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RoutesResolver] APIKeyController {/api/api-keys}:
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/api-keys, POST} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/api-keys, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/api-keys/:id, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/api-keys/:id, PUT} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/api-keys/:id, DELETE} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RoutesResolver] ActivityController {/api/activities}:
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/activities, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/activities, POST} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/activities/statistics, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/activities/:id, DELETE} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RoutesResolver] AlbumController {/api/albums}:
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/albums, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/albums, POST} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/albums/statistics, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/albums/:id, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/albums/:id, PATCH} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/albums/:id, DELETE} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/albums/:id/assets, PUT} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/albums/:id/assets, DELETE} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/albums/:id/users, PUT} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/albums/:id/user/:userId, PUT} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/albums/:id/user/:userId, DELETE} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RoutesResolver] AppController {/api}:
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/.well-known/immich, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/custom.css, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RoutesResolver] AssetController {/api/assets}:
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/assets/memory-lane, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/assets/random, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/assets/device/:deviceId, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/assets/statistics, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/assets/jobs, POST} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/assets, PUT} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/assets, DELETE} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/assets/:id, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/assets/:id, PUT} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RoutesResolver] AssetMediaController {/api/assets}:
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/assets, POST} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/assets/:id/original, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/assets/:id/original, PUT} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/assets/:id/thumbnail, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/assets/:id/video/playback, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/assets/exist, POST} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/assets/bulk-upload-check, POST} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RoutesResolver] AuditController {/api/audit}:
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/audit/deletes, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RoutesResolver] AuthController {/api/auth}:
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/auth/login, POST} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/auth/admin-sign-up, POST} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/auth/validateToken, POST} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/auth/change-password, POST} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/auth/logout, POST} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RoutesResolver] DownloadController {/api/download}:
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/download/info, POST} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/download/archive, POST} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RoutesResolver] DuplicateController {/api/duplicates}:
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/duplicates, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RoutesResolver] FaceController {/api/faces}:
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/faces, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/faces/:id, PUT} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RoutesResolver] JobController {/api/jobs}:
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/jobs, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/jobs, POST} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/jobs/:id, PUT} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RoutesResolver] LibraryController {/api/libraries}:
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/libraries, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/libraries, POST} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/libraries/:id, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/libraries/:id, PUT} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/libraries/:id, DELETE} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/libraries/:id/validate, POST} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/libraries/:id/statistics, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/libraries/:id/scan, POST} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RoutesResolver] MapController {/api/map}:
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/map/markers, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/map/reverse-geocode, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RoutesResolver] MemoryController {/api/memories}:
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/memories, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/memories, POST} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/memories/:id, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/memories/:id, PUT} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/memories/:id, DELETE} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/memories/:id/assets, PUT} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/memories/:id/assets, DELETE} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RoutesResolver] NotificationController {/api/notifications}:
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/notifications/test-email, POST} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/notifications/templates/:name, POST} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RoutesResolver] OAuthController {/api/oauth}:
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/oauth/mobile-redirect, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/oauth/authorize, POST} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/oauth/callback, POST} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/oauth/link, POST} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/oauth/unlink, POST} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RoutesResolver] PartnerController {/api/partners}:
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/partners, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/partners/:id, POST} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/partners/:id, PUT} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/partners/:id, DELETE} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RoutesResolver] PersonController {/api/people}:
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/people, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/people, POST} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/people, PUT} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/people/:id, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/people/:id, PUT} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/people/:id/statistics, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/people/:id/thumbnail, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/people/:id/reassign, PUT} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/people/:id/merge, POST} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RoutesResolver] ReportController {/api/reports}:
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/reports, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/reports/checksum, POST} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/reports/fix, POST} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RoutesResolver] SearchController {/api/search}:
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/search/metadata, POST} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/search/random, POST} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/search/smart, POST} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/search/explore, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/search/person, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/search/places, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/search/cities, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/search/suggestions, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RoutesResolver] ServerController {/api/server}:
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/server/about, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/server/storage, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/server/ping, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/server/version, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/server/version-history, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/server/features, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/server/theme, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/server/config, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/server/statistics, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/server/media-types, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/server/license, PUT} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/server/license, DELETE} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/server/license, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RoutesResolver] SessionController {/api/sessions}:
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/sessions, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/sessions, DELETE} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/sessions/:id, DELETE} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RoutesResolver] SharedLinkController {/api/shared-links}:
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/shared-links, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/shared-links/me, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/shared-links/:id, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/shared-links, POST} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/shared-links/:id, PATCH} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/shared-links/:id, DELETE} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/shared-links/:id/assets, PUT} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/shared-links/:id/assets, DELETE} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RoutesResolver] StackController {/api/stacks}:
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/stacks, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/stacks, POST} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/stacks, DELETE} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/stacks/:id, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/stacks/:id, PUT} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/stacks/:id, DELETE} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RoutesResolver] SyncController {/api/sync}:
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/sync/full-sync, POST} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/sync/delta-sync, POST} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RoutesResolver] SystemConfigController {/api/system-config}:
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/system-config, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/system-config/defaults, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/system-config, PUT} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/system-config/storage-template-options, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RoutesResolver] SystemMetadataController {/api/system-metadata}:
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/system-metadata/admin-onboarding, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/system-metadata/admin-onboarding, POST} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/system-metadata/reverse-geocoding-state, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RoutesResolver] TagController {/api/tags}:
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/tags, POST} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/tags, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/tags, PUT} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/tags/assets, PUT} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/tags/:id, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/tags/:id, PUT} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/tags/:id, DELETE} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/tags/:id/assets, PUT} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/tags/:id/assets, DELETE} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RoutesResolver] TimelineController {/api/timeline}:
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/timeline/buckets, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/timeline/bucket, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RoutesResolver] TrashController {/api/trash}:
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/trash/empty, POST} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/trash/restore, POST} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/trash/restore/assets, POST} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RoutesResolver] UserAdminController {/api/admin/users}:
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/admin/users, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/admin/users, POST} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/admin/users/:id, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/admin/users/:id, PUT} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/admin/users/:id, DELETE} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/admin/users/:id/preferences, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/admin/users/:id/preferences, PUT} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/admin/users/:id/restore, POST} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RoutesResolver] UserController {/api/users}:
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/users, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/users/me, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/users/me, PUT} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/users/me/preferences, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/users/me/preferences, PUT} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/users/me/license, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/users/me/license, PUT} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/users/me/license, DELETE} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/users/:id, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/users/profile-image, POST} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/users/profile-image, DELETE} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/users/:id/profile-image, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RoutesResolver] ViewController {/api/view}:
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/view/folder/unique-paths, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:RouterExplorer] Mapped {/api/view/folder, GET} route
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:NestApplication] Nest application successfully started
[Nest] 17  - 12/19/2024, 10:22:12 AM     LOG [Api:Bootstrap] Immich Server is listening on http://[::1]:2283 [v1.123.0] [production] 
[Nest] 17  - 12/19/2024, 10:22:21 AM   DEBUG [Api:LoggingInterceptor~0cj0j1gc] GET /api/server/ping 200 5.49ms ::ffff:127.0.0.1
[Nest] 17  - 12/19/2024, 10:22:26 AM     LOG [Api:EventRepository] Websocket Connect:    0FJiRfSoKUu1X1zLAAAB
[Nest] 7  - 12/19/2024, 10:22:35 AM   ERROR [Microservices:MetadataService] Unable to initialize reverse geocoding: Error: timeout exceeded when trying to connect
Error: timeout exceeded when trying to connect
    at Timeout._onTimeout (/usr/src/app/node_modules/pg-pool/index.js:205:27)
    at listOnTimeout (node:internal/timers:594:17)
    at process.processTimers (node:internal/timers:529:7)
[Nest] 7  - 12/19/2024, 10:22:36 AM     LOG [Microservices:ServerService] Feature Flags: {
  "smartSearch": true,
  "facialRecognition": true,
  "duplicateDetection": true,
  "map": true,
  "reverseGeocoding": true,
  "importFaces": false,
  "sidecar": true,
  "search": true,
  "trash": true,
  "oauth": false,
  "oauthAutoLaunch": false,
  "passwordLogin": true,
  "configFile": false,
  "email": false
}
Error: timeout exceeded when trying to connect
    at Timeout._onTimeout (/usr/src/app/node_modules/pg-pool/index.js:205:27)
    at listOnTimeout (node:internal/timers:594:17)
    at process.processTimers (node:internal/timers:529:7)
microservices worker error: Error: timeout exceeded when trying to connect
microservices worker exited with code 1
Killing api process

Additional information

No response

[mmomjian]

The error indicates an issue connecting to the database. This could be related to the use of a network share for the database, which is strongly discouraged.

[michalkrol]

The database was not restarted during the service update, as only services affected by the release version change were updated, and services like DB stayed untouched up & running.
The rollback of the server service to 1.122.2 without DB restart made the service work stable.

After shutting down the whole stack, pruning all service leftovers, and starting it again (still with the network share as the DB location), the stack works stable. Apparently, it was due to the connection with the DB, that after the restart allowed for successful connection.

[bo0tzz]

Nonetheless, you should move away from having the database on a network share. It will cause problems.