'invoke restore' relation does not exist

[ursincantieni]

Please verify that this bug has NOT been raised before.

• I checked and didn't find a similar issue

Describe the bug*

Similar issue #8605, but it's solution doesn't fully solve my problem.

I need to properly backup all data and be able to restore it on a new system. This should not just include parts etc, but also plugins, api keys, etc.

With invoke import-records I can successfully restore part data etc, but I lose users and so on. Invoke restore doesn't crashes with

Steps to Reproduce

Create backup from 'old' system

Before backup, run

sudo docker compose run --rm inventree-server invoke update

Create backup with

sudo docker compose run --rm inventree-server invoke backup

and export records with

sudo docker compose run --rm inventree-server invoke export-records -f "data/data.json"

Setup new system

Copy .env, Caddyfile and docker-compose.yml from old system to new one.
Initialize database with

sudo docker compose run --rm inventree-server invoke update --skip-backup

and copy data backup default-Z.psql.bin.gz to inventree-data/backup.
Running

sudo docker compose run --rm inventree-server invoke restore -d default-<NUMBERS>.psql.bin.gz

fails with the logs attached.

If instead of running invoke update, I run

sudo docker compose run --rm inventree-server invoke import-records -c -f 'data/data.json'

it successfully completes and restores all the parts, but is missing plugins, API keys, etc.

Any ideas what's going wrong? Any more generally, is this the recommended approach to backing up data? Would simply archiving the inventree-data directory make sense?

Expected behaviour

Successfully restore all data, including parts, api keys, plugins, aso

Deployment Method

• Docker
• Package
• Bare metal
• Other - added info in Steps to Reproduce

Version Information

0.17.0

Please verify if you can reproduce this bug on the demo site.

• I can reproduce this bug on the demo site.

Relevant log output

### Logs of `invoke restore`

[+] Creating 2/0
 ✔ Container inventree-cache  Running                                                                                 0.0s 
 ✔ Container inventree-db     Running                                                                                 0.0s 
Loading config file : /home/inventree/data/config.yaml
Restoring InvenTree database
Python version 3.11.9 - /usr/local/bin/python3
/root/.local/lib/python3.11/site-packages/allauth/exceptions.py:9: UserWarning: allauth.exceptions is deprecated, use allauth.core.exceptions
  warnings.warn("allauth.exceptions is deprecated, use allauth.core.exceptions")
2025-01-20 08:17:15,105 INFO Restoring backup for database 'default' and server 'None'
2025-01-20 08:17:15,106 INFO Restoring: default-8ebe3e3ea933-2025-01-20-074704.psql.bin.gz
2025-01-20 08:17:15,109 INFO Restore tempfile created: 473.4 KiB
2025-01-20 08:17:15,110 DEBUG  pg_restore --dbname=postgresql://user:user@inventree-db:5432/inventree --single-transaction --clean 
Traceback (most recent call last):
  File "/home/inventree/src/backend/InvenTree/manage.py", line 24, in <module>
    main()
  File "/home/inventree/src/backend/InvenTree/manage.py", line 20, in main
    execute_from_command_line(sys.argv)
  File "/root/.local/lib/python3.11/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/root/.local/lib/python3.11/site-packages/django/core/management/__init__.py", line 436, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/root/.local/lib/python3.11/site-packages/django/core/management/base.py", line 412, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/root/.local/lib/python3.11/site-packages/django/core/management/base.py", line 458, in execute
    output = self.handle(*args, **options)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/lib/python3.11/site-packages/dbbackup/management/commands/dbrestore.py", line 87, in handle
    self._restore_backup()
  File "/root/.local/lib/python3.11/site-packages/dbbackup/management/commands/dbrestore.py", line 143, in _restore_backup
    self.connector.restore_dump(input_file)
  File "/root/.local/lib/python3.11/site-packages/dbbackup/db/base.py", line 106, in restore_dump
    return self._restore_dump(dump)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/lib/python3.11/site-packages/dbbackup/db/postgresql.py", line 145, in _restore_dump
    stdout, stderr = self.run_command(cmd, stdin=dump, env=self.restore_env)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.local/lib/python3.11/site-packages/dbbackup/db/base.py", line 172, in run_command
    raise exceptions.CommandConnectorError(
dbbackup.db.exceptions.CommandConnectorError: Error running:  pg_restore --dbname=postgresql://user:user@inventree-db:5432/inventree --single-transaction --clean 
pg_restore: while PROCESSING TOC:
pg_restore: from TOC entry 4205; 2606 42786 FK CONSTRAINT inventree_kicad_selectedcategory inventree_kicad_sele_footprint_parameter__747c08c6_fk_part_part user
pg_restore: error: could not execute query: ERROR:  relation "public.inventree_kicad_selectedcategory" does not exist
Command was: ALTER TABLE ONLY public.inventree_kicad_selectedcategory DROP CONSTRAINT inventree_kicad_sele_footprint_parameter__747c08c6_fk_part_part;

ERROR: InvenTree command failed: 'python3 manage.py dbrestore --noinput --uncompress -v 2 -i default-8ebe3e3ea933-2025-01-20-074704.psql.bin.gz'
- Refer to the error messages in the log above for more information

[SchrodingersGat]

Ah ok, that is an interesting failure mode.

So, you have plugin(s) installed which make changes to the database - in this case the inventree-kicad plugin (thanks @afkiwers ;) )

When you create the new database schema (via invoke update) the core database tables are constructed - but not the tables associated with the plugins.

What if you try the following sequence (where O = Old database, N = new database)...

1. O: invoke update
2. O: invoke backup
3. N: invoke install
4. N: invoke restore -d <backup>

Here, you are explicitly skipping the update step for the new database. Looking at the django-dbbackup documentation what you may want to do is import the records and scheme, not just the records...

[matmair]

@SchrodingersGat we probably are missing the required settings to activate the plugins so that the schema is missing. To address this we probably would need a small file that contains info regarding versions and plugins source to ensure the environments match. That would also help with general backup / restore tasks as it would enable better error messages.
Could be a bunch of work though