Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[ADD] Odev framework and default scripts
[REM] *: remove erroneous symbolic link [IMP] clean: force password = 'odoo' for all users except admin [FIX] listing: remove typo in column titles [REF] cleanup setup file - Add utils.mkdir method - Refactor src/setup.py to use mkdir instead of rpeating the same actions four times [REF] refactor scripts - Rename to - New methods to ensure a database is running or stopped - Save config to file while running commands - Small fixes and improvements [ADD] new scripts - [#3] Dump a SaaS database - Initialize an empty database - Kill the process of a running database - [#5] Quickstart the generation of a new database (create, init/dump/restore, clean) - Small improvements in scripts [IMP] update documentation [FIX] subprocess command for restoring dumpfiles [FIX] default permissions for created directories [FIX #1] *: standard python libraries in requirements [FIX] reference errors in setup [IMP] automatically create configuration file if not existing [IMP] create: duplicate database from template [IMP] init: prevent loading demo data [IMP] setup: move settings to user config by default [FIX] scripts: path to local config [IMP] run: add support for virtualenv [FIX] run: typo in message to user [IMP] utils: print carets in bold [IMP] utils: code formatting [IMP] scripts: remove support for virtualenv as it is user and system-specific [IMP] *: code formatting [IMP] dump: download dumps of SH databases [ADD] task: get the link to a task [FIX] quickstart: init empty database with version number [FIX] readme: remove sudo during install [IMP] scripts: add virtualenv support - Create virtualenv linked to odoo version - Switch to correct virtualenv when running odoo-bin - Automatically install requirements in the correct virtualenv - improve checking for changes in Odoo's GitHub repositories [ADD] help: display help about subcommands [README] update setup after repository transfer [IMP] add sql queries to clean script [IMP] uncomment query to disable oauth providers [PERF] faster db list [IMP] always add current directory if addon directory [ADD] cli: imported module from `psbe-custom-upgrade` PR #17 [IMP] cli: refactored and added sub-subcommands feature [REF] global refactoring, adapted scripts to `cli` interface [IMP] allowing databases with slightly more exotic names [FIX] fixed multiline argparse formatting [FIX] `cli` refactoring testing fixes [IMP] run: made addons argument optional [FIX] restore: fixed and refactored pg import / zip handling code [FIX] quickstart: fixed refactoring bugs, sql dumps, url regex [IMP] cli: short help feature in CliCommand, filled in for commands [FIX] cli: fixed root / subroots not running __init_subclass__ [FIX] __main__: relative imports when run from symlink [FIX] quickstart: fixed url regex [FIX] __main__: exceptions TODO, check ret code vs. log level on exit [FIX] kill: removed duplicate/unused `pid` re-assignment [REF] *: removed NO_DB, refactored `template1` methods in base class [FIX] *: AVS PR review, TODOs, typos, small tweaks and fixes [REF] *: refactored sql classes code and interface [REF] utils: clearer `curl` function signature for response headers [FIX] *: clear database cache when altering databases [FIX] create: adapted to new `dbname_validate` approach for template [IMP] added requirements.txt file [REF] *: substituted `clint` lib with `blessed`, logging with stdlib [REF] *: renamed package `src` to `odev`, removed `odev.py` odev can now be run with the installed ~/.local/bin/odev symlink, that points to `__main__.py`, or by doing `python3 -m odev` from the repo root. [ADD] utils: imported NDA shconnector script from `connect-plus` repo [ADD] odoosh: ported over from `psbe-custom-upgrade` PR #17 as `sh` [IMP] cli: print args before doing anything else, including __init__ [ADD] utils: added secret storage with agentcrypt (also, github API) [REF] odoosh: split upgrade class into Base and -manual, small fixes [ADD] odoosh: upgrade-merge method for upgrading with a PR merge [FIX] sh upgrade-*: ensure copied upgrade paths are added for cleanup (although in practice this was already taken care of at L150, it's better if `_prepare_upgrade_path_files()` does it too for some futureproofing) [REF] sh upgrade-*: moved build code into own `wait_for_build` method [FIX] sh upgrade-*: fixed rsync paths bug [FIX] sh upgrade-merge: longer wait time for sh build to appear [IMP] sh upgrade-merge: flashy warning to confirm before merging PR [IMP] sh upgrade-*: small fixes, tweaks, TODOs [ADD] sh upload: first implementation of SH database upload command [REF] odoosh: renamed `project` argument to `repo` [ADD] sh rebuild: first implementation of SH branch rebuild command [IMP] cli: less verbose with parsed args [IMP] odoosh: fancy progress bar while waiting SH build [IMP] sh rebuild: flashy warning to confirm before staging rebuild [FIX] __main__: relative imports when called from symlink [IMP] odoosh: added show progress option to rsync upload [FIX] quickstart: fixed url regex [IMP] secrets: implemented secret deletion functionality [IMP] shconnector: valid session tests, expired credentials renewal [IMP] odoosh: handle missing build cases [IMP] odoosh: better handling of SH build warnings [ADD] sh upgrade-wait: command for generic SH builds (redeliver/push) also upgrade code refactoring/DRY and bugfixes [IMP] sh upgrade-wait: long 10min wait for build to appear [FIX] sh upgrade-*: cleanup upgrade_path config before removing files [ADD] *: added `noconfirm`/`-y` cli option to skip prompts (+TODOs) [REF] Global framework refactor - Automatic command loading - Automatic arguments parsing - Improved command registering and dispatching - Added overall typings - Improved exception handling in main process + added custom exceptions Overall reorganization of the project structure and split accross directories for clarity. Commands will now extend predefined structures with helper methods and class attributes. Commands will be automatically loaded from the 'odev/commands' directory and sub-directories without requiring any init file or export. Added a directory to store commonly used constants. [IMP] Add configuration manager class - Load and write config files from/to a standardized location using aliases - Set and get values from configuration at anytime - Automatically save current configuration to file with no explicitly accessing the os [REF] Gloabl refactoring of utils Misc improvements to utils and split of methods in different files, regrouped by scope or features. [REF] Logging and user interraction Global refactor of the logging mechanism to extend standard logging capabilities. Added theming support to the logger to allow for minimalistic (original odev style) or extended (logging library) styling. Interractivity is assured through the logger instead of external utility methods thanks to the addition of custom 'logging levels'helpers. Also reintroduced requirements.txt file. [REF] Main process refactor Add support for few custom exceptions. Improve signal handling. Use new logging mechanism. Improve project structure. [REF] Setup process refactoring Improve project structure. Make setup more robust, default to user-specific directories. Improve configuration handling. Find previous configuration if re-running setup. Prevent breaking user's system with incorrect path handling. Do not ask for root access anymore and actually advise against it. [REF] Original commands Rework original commands to work with the new framework and make use of the new logging capabilities. Reorganize the project directories to auto-load commands through the registry. [IMP] Add automatic help command Add support for 'odev help' to display hints about all or specific commands. Automatically loading description, usage and arguments from commands loaded into the registry. [IMP] Add support for odoo-bin commands Add support for various odoo-bin commands, including a refactor of 'odev run'. Now providing specific commands and support for odoo-bin shell and cloc. Also providing a command wrapper for enabling tests when running a database. [REF] Odoo SH commands Refactor of Odoo Sh specific commands to work with the new framework, logging and utils. [ADD] Config commands Add commands for easily setting or reading odev configuration. [IMP] Auto-update Automatically check for updates in the current odev branch and pull changes if necessary (and approved by the user). If updated, restart the process immediately to benefit from the new update(s). [FIX] SQL error handling and escaping [IMP] Capture SIGINT and SIGNUM in subprocesses [IMP] Capture SIGINT and SIGNUM in curl subprocess [FIX] Write symlink to /usr/local/bin for system-wide access [FIX] Update psycopg2 python dependencies [FIX] Add decorator dependency to requirements.txt [DOC] Refactor README.md [FIX] Addons-path argument parsing in cloc command [IMP] Print newline before cloc report [FIX] Setup: Symlink creation and permissions management [IMP] Doc and help [FIX] Restart current process after code upgrade [IMP] Force fetch before update check [IMP/FIX] Allow dumping dev/staging databases from Odoo SH + fix regular expressions for finding tokens [FIX] README: reorder installation steps Do not try to install requirements before clone the repository as requirements.txt does not exists yet. [IMP] Git error handling Detect errors in git subprocesses (git fetch, git pull) and print a useful message. [FIX] Sanitize URL when dumping databases [IMP] Allow saving and reusing arguments in run commands Applies to the following commands: - odev run - odev test - odev shell - odev cloc [FIX] Traceback in list with new databases [FIX] Allow to run cmd on self imported db (#43) * [FIX] Allow to run cmd on self imported db * [FIX] Fix review [IMP] init: Create SQL unaccent extension on database creation The 'unaccent' SQL extension is used on Odoo database to perform in-text search while stripping accented characters (i.e. 'e' matches 'é'). Odoo SaaS and SH databases have this extension enabled and available, but databases initialized locally with 'odoo-bin' do not activate it when installing the base module. This commit provides this extension as defined in Odoo SaaS/SH SQL dumps on empty databases initialized locally. [IMP] `README.md`: track all `odev*` branches (#39) [IMP] logging: set log-level at script's start Set log-level before loading the registry if provided in command's arguments. This enables support for custom and debug log-levels outside of the commands scope, as otherwise the log-levels where set at the initialization of the registry. [FIX] Remove unused imports [DOC] REAMDE.md: improve contributing section Add text paragraph to present issues and new feature requests. [FIX] odoo sh command line args The merged odev-abt* branch was using slightly different command line args. [IMP] upgrade: add support for database upgrades (#49) Closes #44 Upgrade a local Odoo database, running migration scripts between each major versions. [FIX] upgrade: remove unused argument parsing Argument `--silent` was used for debugging purposes but removed after validation of the features. Though parsing of that argument was not correctly removed. Also moved the `upgrade` command to the `odoo_bin` directory. [IMP] restore: add `--no-clean` argument (#53) Closes #51 Allows running the `restore` command without cleaning the database, useful for upgrades. [IMP] run: install `requirements.txt` for custom addons (#54) Closes #52 Previously to this commit, only the requirements from the `odoo/odoo` repository were installed. This automatically finds new `requirements.txt` files in the provided addons and attemps to install them if necessary. [ADD] Support`restore``sql.gz` dumps (SH backups) [FIX] dump: using incorrect support url on saas instances [FIX] clean: set expiration date as datetime for versions >= 15.0 (#56) Closes #55 The database expiration date became a datetime object in version 15.0, as opposed to a date object in previous versions. [FIX] `quickstart`: pass `no_clean` argument to `restore` [FIX] shconnector: infinite recursion bug on expired SH session [REF] odoosh: re-rename attributes `*repository` to `*repo` [IMP] odoosh: use branch history (trackings) to follow new build [IMP] odoosh: more robust build following (use tracking id) [IMP] sh upload/rebuild: adapt code for branch history tracking [REF] config: refactored and improved ConfigManager, added autosave [IMP] shconnector: odoo impersonation for login (instead of github) - shconnector now requires repository + GH username for session setup - refactored shconnector methods to remove repo argument - implemented util.credentials to provide helpers for getting credentials - removed secret_storage helper and supporting classes - small OdooComCliMixin for providing odoo credentials cli options - refactored and improvedConfigManager, added autosave [FIX] sh upgrade: contextmanager import from wrong library [ADD] sh upload: implement keep filestore feature [IMP] Add section about `ssh-agent` to `README.md` (#29) [ADD] odev sh upload: dump zipfile sanity check (#32) * typo in typehint * [ADD] odev sh upload: dump zipfile sanity check * PR comments ABT [FIX] commands: fix broken commands arguments inheritance [FIX] sh upgrade: fix broken bash paths escaping using backticks see https://unix.stackexchange.com/a/27432 [FIX] adjust files mode to 644 [FIX] config: fix MutableMapping import conflict w/ typing-abc (#80) [IMP] Auto install pudb and ipdb to each venv (#68) [FIX] Fix debug log level (#69) [IMP] `restore`: pipe extract The temp partitions size can be limited. The original solution to extract zipped dumps was using pythons `tempfile` lib which by standard writes to the temp partition. Especially big databases are able to fill the temp partition on many systems. Why use the filesystem in the first place? `dump.sql` is directly piped to `psql`. The filestore is handled according to the users choice, also w/o `tempfile`. [FIX] Fix requirement install (odev run) (#82) * [FIX] Fix requirement install (odev run) * [FIX] Fix review * [FIX] Fiw comment 2 * [FIX] remobe blank line [IMP] Use worktree instead of cloning (#67) * [IMP] Use worktree instead of cloning * [FIX] Fix review comment * [FIX] Fix comment review * [FIX] Fix review , remove unused import * [FIX] Fix review , .git is a DIR or a FILE [FIX] Remove useless logger oopsy [FIX] Fix requirement.txt by adding packaging (#83) [FIX] Replace match statements (#84) Ensure compatibility with python 3.8 because the `match` statement was only introduced in version 3.10. [FIX] github: remote default fallback Remote branch may not always have the name `origin`. Accessing that name fails for some repos (e.g. odoo enterprise). Resolved by adding a method calling the 1st remote branch as fallback if there is no `origin`. [IMP] Add no-root scenario to setup [REF] github: refactor module, improve flexibility, add documentation * moved "check if cloned and updated" code from `odoo` module and split into multiple functions * refactored, DRYed and renamed `pre_run` fn from `utils.odoo` (now `prepare_odoobin`) * adapted usages of `prepare_odoobin` for the changed signature * added head == branch sanity check to `git_pull` * DRYed `self_update` function by improving and calling `git_pull` * small refactorings here and there * added docstrings and type annotations * Black formatted * merged with `use worktree instead of cloning (#67)` * merged with #63 [IMP] odoo: improved version handling and parsing - improved odoo-to-python versions mapping - added `saas~` versions support - added function to get a comparable `StrictVersion` object - added function to get odoo branch name from version - DRYed versioned repos folder path generation - renamed `LocalDatabaseCommand.db_version` to more appropriate `db_base_version` - added `LocalDatabaseCommand.db_version_parsed` to get a comparable `StrictVersion` for a db - removed `RE_VERSION` constant as not used anymore - added custom `InvalidVersion` exception - refactored other version handling code with new utils functions [IMP] exceptions: added base classes for all custom exceptions - added modules docstrings. - removed unnecessary `pass` after classes docstring. - Black-formatted code. - moved exceptions defined in `utils/github` to `exceptions/git` - merges with #67 [IMP] upgrade: rewrite feature leveraging upgrade-platform code - merged with #68 - merged with #82 - merged with #67 [IMP] *: use `Version` from `packaging` instead of `StrictVersion` [FIX] sh upgrade: fix broken SQL backticks vs single quotes (#88) [ADD] odev: offline support (#72) This commit adds support for running `odev` in an environment that is not connected to the interwebs. An error is still indicating that the connection to the remote git repo was not possible. [FIX] comparing version with str during db cleaning (#89) [IMP] commands: fix arguments merging across classes hierarchy (#87) - also improve check for already-defined commands when the class is the same (due to implicit + explicit module imports) [IMP] Implemant a new odev clone command (#92) * [IMP] Split shConnector into two method login / impersonate * [IMP] Make ask() more smart by checking if answer is in the list * [ADD] Add method to check if an Odoo db is on saas * [ADD] Add clone command to odev * [IMP] Invoke odev clone at the end of quickstart cmd * [FIX] Fix after review [IMP] Automatically install new requirements during self update (#85) * [IMP] Automatically install new requirements during self update * [REF] python: refactor install requirements/packages to new util Co-authored-by: abk16 <[email protected]> [FIX] Add missing requirement after rebase (#95) [IMP] Use CredentialsHelper in dump cmd (#96) [FIX] Fix pg_restore that failed with wrong args (#97) [FIX] Fix clone call from QS (#99) * [FIX] Fix clone call from QS * [FIX] Fix after review [FIX] Fix do_raise method after review + odev clone (#101) [IMP] Print confirm message even if -y/-n (#100) [FIX] test: correct arguments order with inherited `args` (#106) [FIX] clone: dev path in config is without 's' (#109) Issue: TypeError: expected str, bytes or os.PathLike object, not NoneType [IMP] run,cloc: running command as info, not debug (#103) [WIP] Add pull command and ask on run after x days (#110) * [WIP] Add pull command and ask on run after x days * [IMP] Add clone command to the readme * [FIX] Fix after review * [FIX] Fix review [IMP] Add color bar in the log (#94) * [WIP] Add color bar in the log * [IMP] Put config inside theme [FIX] Fix empty answer on confirm triggering Index out of range (#113) [WIP] Fix pull command (#114) * [WIP] Fix pull command * [FIX] Fix after review [IMP] Use a globals_context var to share data between successive calls to run_with() (#102) * [IMP] Allow to passe a dict to a command to get back some data * [FIX] Fix review * [FIX] Fix review 2 [FIX] Fix worktree list creation and kwargs (#117) * [FIX] Fix worktree list creation and kwargs * [FIX] Fix after review [IMP] git_pull: fetch once per run per repo, and merge vs pull (#111) [FIX] git_pull: fetch-once feature merge fix with #110 (#122) [IMP] Allow to pass --without-demo to init and qs (#120) [FIX] Remove command name printed twice in help (#119) [IMP] Add -t --template argument to run/remove and qs (#112) * [IMP] Add -t --template argument to run/remove and qs * [FIX] Fix after review [ADD] Add Odev scaffold and pleasedo method (#93) * [ADD] Add scaffold method * [IMP] Improve separation between class, qs project * [FIX] Re-apply commit from ACR and fix some bug * [IMP] Use new ask method, change help text * [IMP] Fix template, fix texte, update staging db name * [ADD] Add pleasedo command * [FIX] Only show Assume Yes/No once * [WIP] Fix string * [FIX] Remove the second call to clone (already present in quickstart * [IMP] Get path from clone command to scaffold directly in the repo * [WIP] Fix git clone cmd * [WIP] Fix path from clone * [FIX] Fix error on backend view if field is empty * [FIX] Fix after review * [FIX] Remove import test from init file * [FIX] Fix staging url * [IMP] Auto install pre-commit if git_repo * [IMP] use path from clone if exist * [FIX] Fix review second pass * [FIX] Remove useless comment and rename task_id * [IMP] Add unit test file if needed, split method * [IMP] Handle asset for v15 * [IMP] Handle depends on business flow * [IMP] Fix calling super() for method and compute code * [IMP] Add support for mail and activity mixin * [FIX] Fix init for tests folder * [FIX] Fix constraint not scaffolded * [FIX] Fix requirements.txt config * [FIX] Fix xml_id for saas and add index field param * [FIX] Fix constraint string * [IMP] Fix string, remove todo * [FIX] Fix template error and string * [FIX] Move default space value for data into the template * [FIX] Fix detection of saas database * [IMP] Continue pleasedo even if dump failed * [IMP] Retun Sh repo if prefect match * [FIX] Fix fix fix after review [FIX] Fix wrong depends for Unicode + improve requirement based on ABT proposal (#123) [FIX] Fix get_worktree_list (#124) [IMP] Only check for odev update once a day (#125) * [IMP] Only check for odev update once a day * [EOF] Fix * [FIX] Fix after review [IMP] Only try to install requirement if install or update [ADD] VSCode debug config generator (#121) Create a vscode config file to debug odoo projects [ADD] Add the exporter to odev (#127) * [ADD] Add the exporter to odev * [FIX] Fix class name for color logging * [FIX] Fix broken init_config in scaffold * [FIX] Remove usless import * [FIX] Remove wrongly added newline * [FIX] Fix after review [ADD] unix-style help for all commands (`odev cmd --help`) (#128) Improve backup dump + `requests` > `curl` (#126) * [IMP] dump: refactor connection flow and error handling Replace usage of `curl` subprocess calls with python `requests` package. Improve HTTP requests status check using HTTP response codes instead of parsing text headers. Improve error handling on large databases and provide sensible messages to the user. * [FIX] commands: add `do_raise` arg if not used through `run_with` The Quickstart command uses `do_raise` when called by PleaseDo, but that argument was not set when running Quickstart manually, resulting in a traceback during controlled error handling. * [IMP] dump: fallback to daily backup When the backup to download is too large to be fetched through the support page of SH projects, fallback to downloading the last daily backup through SCP. [FIX] help: executable path in global help message (#129) [IMP] deploy: add support for `odoo-bin deploy` (#48) * [IMP] deploy: add support for `odoo-bin deploy` Deploy a local module to a SaaS database, or a local database if no URL is provided. [FIX] Add custom repo for dubai,hong kong and usa [ADD] pre-commit: configuration & adaptations (#132) * [ADD] pre-commit: configuration & adaptations * [REM] pre-commit: project.toml config for black * [IMP] pre-commit: config for black * [IMP] pre-commit: MyPy configuration and initial corrections * [FIX] odev: make it green again Fix all remaining linting errors in VSCode. ... at least on my machine. * [REF] odev: `_logger` code coherence Switch from `logger` to `_logger` variable name accross the whole codebase. * [IMP] pre-commit: isort lines after import * [IMP] pre-commit: autoflake remove unused variables * [FIX] shconnector: regular expression format * [DOC] readme: update worktree vs single-branch * [DOC] readme: update required python version to 3.8+ * [REF] odev: code coherence `%s`->`f-string` * [FIX] pleasedo: support for psql templates * [FIX] pleasedo: quickstart `do_raise` * [FIX] scaffold: variable naming `data`->`views` * [FIX] commands: conflicting arguments * [VER] 2.1.0 * [REF] scaffold: cast `self.version` to `Version` instead of `str` * [FIX] util.py: restore previous version & ignore pre-commit [IMP] Standardize database creation and check (#138) [FIX] Remove default .Sh value for export type (#136) [IMP] Remove pleasedo and use quickstart instead (#137) * [IMP] Remove pleasedo and use quickstart instead * [FIX] Fix after code review [FIX] Quickfix missing inherinting for qs (#139) [FIX] python install command [FIX] pull command [IMP] Clean Saas module, install req on first run, avoid sudo (#144) * [FIX] Fix cleaning by removeing saas_module * [FIX] Install requirements.txt on first run * [FIX] Prevent to run setup or odev as root user * [FIX] Dump version number for bugfixes * [FIX] Fix dependency problem with Jinja by warning the user * [FIX] Fix after review * [FIX] Fix help [IMP] Create a sandbox venv linked to the database (#140) * [WIP] Create a sandbox venv linked to the database * [FIX] Fix after review * [FIX] Reuse venv if already existing, improve logger, bug folder doesn't exist * [FIX] Rename alternative venv argument [IMP] Try to generate a repo path if clone failed (#147) [IMP] Allow odev dump to dump a locate database (#149) * [IMP] Allow odev dump to dump a locate database * [FIX] Fix after review [IMP] Allow user to add queries to the clean command #blackmode (#148) * [IMP] Allow user to add queris to the clean command #blackmode * [FIX] Fix after review * [FIX] Fix after review [FIX] Fix DBExistsCommandMixin help text (#150) [FIX] Fix pre-commit install when scaffolding (#146) * [FIX] Fix pre-commit install when scaffolding * [FIX] Fix following review [FIX] Surround template name with quote (#145) [FIX] unknow log level when '-v' in sys.argv (i.e. --database psbe-vsomething) (#153) [FIX] fix saas dump error [FIX] fix SH dump error (#156) When running `odev dump` for an online db, `AttributeError: 'DumpCommand' object has no attribute 'url'` [FIX] Fix branch_name when scaffold without clone (#158) [FIX] Fix regression when using dump with qs (#157) [FIX] Fix get_odoo_version for saas (#159) [FIX] Disable 2FA on admin account and 50 firsts local accounts (#160) [FIX] sh: increase poll interval, default retry sh connections (#161) [FIX] Disable 2FA on admin account and 50 firsts local accounts (#163) * [FIX] Fix totp_secret query , execute only if exist * [FIX] Fix totp_secret query , execute only if exist * [FIX] Fix log info for the first 50 users Odev precommit sh monitoring (#133) * [ADD] sh-monitor: monitor test on SH databases [IMP] monitor: parse a csv files with odoo sh projects and run tests * [FIX] github: `get_worktree_list` installed versions was always empty * [IMP] monitor: checkout production branch * [IMP] monitor: tests runtime * [IMP] monitor: save info to ps-tools database * [IMP] monitor: cloc + pre-commit + credentials + logs + handle single url * [IMP] utils: improve pip packages install + handle pip errors * [IMP] logging: sh_monitor command tree color * [FIX] create: create database with lc_collate and encoding * [IMP] monitor: results fallback + build errors + prepare venv * [IMP] monitor: build results + fetch submodules + pre-commit config update + repositories cloning * [IMP] monitor: include submodules * [REM] pleasedo: delete file (reset correct state) * [FIX] exporter: C416 Unnecessary list comprehension * [IMP] monitor: create project before tests + ps-tools database in config * [FIX] @sea-odoo review comments * [FIX] monitor: no pre-commit config found for version * [IMP] monitor: filter assigned/unassigned projects [FIX] upgrade: fix stdin in subprocess to enable debuggers (#164) [FIX] Fix field_name changed on ps-tools (#165) [FIX] Fix url-detection in scaffold command (#166) * [FIX] Fix url-detection in scaffold command * [FIX] use None instead of empty string [ADD] cloc: output csv formatted data (#169) [FIX] pin MarkupSafe==2.0.0 for copier/jinja incompatibility (#167) [IMP] github: patch GitPython for bug using relative worktrees (#168) [FIX] Fix call to clone only if we have a repo (#170) [FIX] odev: allow inserting blank between imports The error > Cannot parse: 3:27: from . import modelsimport logging The solution: remove the - to allow blanks [IMP] odev: set https website, as it should be Set url to https, as in pre-commit conf https://github.com/odoo-ps/psbe-ps-tech-tools/blob/14.0-pre-commit-config/.pre-commit-config.yaml#L42 [REF] commands: move odev runtime-specific code to `Command` [ADD] basecommand_test: new `~._get_merged_arguments()` method tests [FIX] code: fix `render_debug_template` signature and `upgrade-path`s - remove `database` arg from `render_debug_template()` - fix `upgrade` repo paths, non-included `migrations` folder due to wrong kwargs in render fnc. [ADD] code_test: new `odev code` command tests - also test for `upgrade` repo paths [ADD] .github: PR checks workflow - only run actions on pull requests targeting `odev` - 2 jobs: - pre-commit: run pre-commit and check that it was correctly applied on code - tests: run tests suite with pytest and check coverage (nonblocking) Co-authored-by: Marten (msc) <[email protected]> Co-authored-by: abk16 <[email protected]> [REF] shconnector: remove duplicate `get_project_info` method [REF] commands: rename `OdooSHDatabaseCommand` `repo` to `project` - also `OdooUpgradeRepoMixin`: added `--custom-util-repo-path` as args alias for existing `--psbe-upgrade-repo-path`, the latter preserved for backwards compatibility (but `custom_util` was removed from that repo) - small TODOs [ADD] submodule: new command to add a submodule on odoo.sh [ADD] prepare-util: new command to prepare util(s) for odoo.sh [FIX] upgrades/3.2.0: do proper `girurlparse` packages cleanup [FIX] rename `psbe_upgrade_repo_path` to `custom_util_repo_path` #183 [IMP] allow to run same db in web server and shell [IMP] avoid force validating upgrade paths Currently when running odev code [dbname] [path] will generate error when trying to validate the upgrade paths . [IMP] create: also copy template filestore (#182) - added optional cli switch `--no-filestore` to disable this behavior [IMP] remove: add new option `--keep-filestore` (#182) - also refactored `keep_template` attribute to an actual cli argument [IMP] commands: add arguments default in `run_with()` Add defaults for missing arguments when calling a command with `run_with`. The priority of the arguments is the same as `ArgumentParser`, specifically (from most important): - `do_raise`, `capture_output` - passed `kwargs` - parser actions defaults - parser arguments defaults Also removed useless `*args`, since it would have never been used (Namespace accepts only kwargs). [IMP] submodule: add `--update` switch to pull submodule (#191) - adapted `prepare-util` to properly work with the new feature - also refactored `_git_commit_context` contextmanager method to `GitCommitContext` class in `github.py` - misc small methods refactorings and changes [FIX] Fix lastrun key to avoid unnecessary pip req check [FIX] clean: clean totp_secret on res_users (set to NULL instead of 'false') (#197) [ADD] `export`: support local exports Local use of `export` was not possible due to: - hardcoded standard port (443) - hardcoded jsonrpc via SSL Port is now detected in url cmdline parmeter. A basic detection of local urls deactivates SSL. [FIX] Fix read_safe method after migration to odev (#194) * [FIX] Fix read_safe method after migration to odev * [FIX] Fix after review [FIX] github: fix worktree list regex (#200) [^] will match any character that is not in the set while we want to match the whole string master. This is causing issue for saas worktrees as the letter "s" and "a" are in "master" and in "saas" This causes the run command on a saas-15.X database to pull everytime you run a database which is slow [IMP] clean: add a localhost outgoing mail server [IMP] clean: add versionised clean_queries [FIX] Secure path deletion and improve version detection (#181) * [FIX] Secure path deletion and improve version detection * [FIX] Fix after review * [FIX] Fix after review [ADD] allow no extract of dump.sql from sql.gz (#162) It is not necessary to extract dump.sql from gzip file in all situations. Current default is to extract so preserve this behaviour unless `-x,--no-extract` is provided. (Similar to RD `upgrade` script). [FIX] create,rename: update databases config (#176) `odev create new_db template_db` does not add the `new_db` to the databases config, even if the `template_db` is in the config. `odev rename ...` does not save the changes to the databases config. [IMP] utils.odoo: improve version parsing, conversions, edge cases - properly handle all known saas version schemes, including pre-v11 ones - fix pre-v11 saas branch names from version - normalize any version with minor != 0 to saas versions (except 6.1) [ADD] odoo_test: new `utils.odoo` version handling functions tests - also refactored tests directory structure [IMP] enable running databases with odoo version <= 9.2 - use lessc v3 for ver < 10.0 databases - add version-specific node_modules to PATH Co-authored-by: Maurin3 <[email protected]> Co-authored-by: abk16 <[email protected]> [REF] refactor "run odoo-bin" code to `utils.odoo.run_odoo` function - small common run/init/cloc commands code refactor into `OdooBinMixin` - added db-specific venv removal in `remove` command + argument to optionally skip it (+ small refactor) - pull up args saving code from `RunCommand` to `OdooBinMixin`, refactored to separate methods - split saved arguments from addons in config (they wouldn't appear like that in odoo cli anyways) - this check keeps the existing code but saves to the `addons` key, so no upgrade script is necessary - small misc code simplification - rename `--env` CLI arg for separate db venv to `--venv` (and remove `-e`) [IMP] odev: remove a db already drop with dropdb If we create/run a database with the `odev run` command, the following 3 parts are created: - A database in SQL - A filestore - An entry in .config/odev/database.cfg If the user use the command `dropdb`, only the database in SQL will be dropped. We find ourselves in a hybrid state, where it is no longer possible to create a db with the same name or to use the `odev remove` command to delete it properly. With this dev, a user can use odev remove to delete the filestore + the config entry even if the SQL database no longer exists [IMP] disable TOTP policy on neutering (#201) [ADD] bash completion script (#172) See `./complete_odev.sh` for usage. [ADD] odev code: templates for empty db / mailcatcher (#212) * [ADD] code: configuration for empty database, `tasks.json`; adapt/extend tests (WIP) * [ADD] tests for filestructure and `tasks.json` assertion * add template args for mailcatcher; debug command path validator * adapt order of depends in template for (python) json decoding * fixup! adapt order of depends in template for (python) json decoding * boolean logic is hard * make tests pass for empty `upgrade_repo_path` * mock `ConfigManager` for environment-independent test results * [ADD] odev code: mailcatcher preLaunch task to db launch configs * [ADD] odev code: exclude `util_package` from custom module list [FIX] pre-commit: bump pre-commit-hooks-safety to 1.3.0 for bugfix [IMP] restore: progress bar for dump file import - use enlighten to display a progress bar of the dump file restoring process - use python to open the file to pass as stdin to the restore subprocess command - for zipfiles, use `ZipFile.open` to stream uncompressed data directly to subprocess pipe - renamed `pg_subprocess` to `restore_subprocess`, refactored signature, renamed decorated functions [ADD] restore_test: new tests add tests monkey patch call to psycopg more monkey patching of psycopg [FIX][Scaffold] Fix ondelete args name (#202) [IMP] sh_monitor: add multi company support [IMP] Check requirements.txt timestamp to avoid useless call (#217) * wq[IMP] Check requirements.txt timestamp to avoid useless call * [FIX] Fix after review + add mig script * [REF] refactor "lastrun" code to OdooBinMixin * fixup! [FIX] Fix after review + add mig script * [REF] rename `lastrun` to `last_run` + adapt mig script Co-authored-by: abk16 <[email protected]> [FIX] Fix error when models is missing (#219) [FIX] run: venv name is not set correctly [REF] Use api key to download saas db Refactore the download of SAAS dumps to use api key since 2FA Remove the deletion of tmp dumps after quickstart and allow to reuse dumps downloaded the same day Use stream to download dumps and log progress with enlighten [FIX] restore: restoring pg_dump formated sql Both psql and pg_dump can generate a .sql file but the format is not the same. It needs to be restored with the right tool. [FIX] scaffold: fix saas version issue [IMP] scaffold: add a config to not use pre-commit [REF] utils, dump, deploy: sanitize url Fix the is_saas_db method to work with urls without https Move the duplicated code of url sanitization in utils [IMP] utils: update python version list [IMP] Bump python version to 3.8 for Odoo 14.0 (#232) * [IMP] Bump python version to 3.8 for Odoo 14.0 * [FIX] test_get_python_version: align data with expected behaviour [IMP]init: initialize db from Odoo repository [ADD] run_standard: add a new command to run database as standard (#227) The command is just a scaffold to clone R&D repo and run the clean_database command https://github.com/odoo/support-tools#clean_database [FIX] clean: MailHog server for Odoo 14 (#235) As SMTP Authentication was backported to Odoo version 14.0, the cleaning queries had to be adapted. The `smtp_authentication` was only filled in for versions greater than 15.0 but now includes versions from 14.0 upward. Additionally, refactoring the query's contruction to avoid duplicate code. [ADD] Add timesheet command (#90) [FIX] clean: Reset Mailhog queries for Odoo 14 SMTP Authentication wasn't backported to Odoo version 14.0 so 4b8a5d6d5555c37f1e1959b1cbd84c17f6b778e4 introduced an issue. This commit fixes it. [ADD] restore/clean: set website domains to NULL (#243) * [ADD] restore/clean: set website domains to NULL to make testing multi website locally possible * SEA: fix query for dbs with website module not installed Co-authored-by: sea-odoo <[email protected]> * ABT: add log info informing user about websites/domains Co-authored-by: sea-odoo <[email protected]> [FIX] upgrade: support odoo version 16.0 target, venvs fixes (#248) [IMP] clean: find most sane admin user record for creds reset (#249) [IMP] shconnector: make bad credentials error more readable (#254) [IMP] utils.github: properly parse git worktrees list info [FIX] utils.github: resolve worktrees paths to check identity The current logic in `git_worktree_create()` for ensuring a worktree path is checked out works by: attempting to add the worktree, and if that fails, assuming that if it appears in `git worktree list` it means it exists already. When comparing paths, we need to make sure all symlinks are resolved first, both in the worktree path we're trying to create and the paths returned by git. [FIX] fix flake8 url in pre-commit config [FIX] .github: enforce python3.8 version [IMP] run: allow force-version argument to force an upgrade Makes testing upgrades much easier with odev. [IMP] upgrade: deprecate command (#263) - suggest test.upgrade.odoo.com server instead - see discussion at https://discord.com/channels/678381219515465750/1034431597308555355/1039479207740313670 [IMP] psql: access the cursor with `.cr` instead of `.cursor` It's more cozy (coming from Odoo where we access it with `env.cr` or just `cr` in upgrade scripts). [IMP] remove: allow removing non Odoo databases When creating a database with the create command, the user might want to remove it without initializing it. Currently that fails, raising an Exception when getting the version with `version = self.db_version_clean()`. This commit fixes that and adds a warning message to the prompt when deleting a non Odoo database. [BUMP] Odev version [FIX] Fix branch name after odev migration to his own repo (#13) [FIX] Fix template restore command that failed if db removed from config (#7) [FIX] Fix cleaning query for fetchmail_server (#10) [FIX] Fix submodule slowness detection (#8) * [FIX] Fix submodule slowness detection * [FIX] Fix after review [FIX] Fix dump command for local database (#9) * [FIX] Fix filestore folder name * [FIX] Fix dump command for local database - Database arg should not be present with database_required=False - Create folder if doesn't exist yet Odev fix scaffold sea (#12) * [FIX] Fix make column1 and column2 string args * [FIX] Put fields and models in lowercase * [FIX] Fix empty model name if named 'x_' * [FIX] Don't add mail.* inherit if base model * [FIX] Fix selection field string for integration (edi) * [FIX] Fix missing compute if no description * [FIX] Allow per version folder, add bootstrap5 for Odoo > 16 * [FIX] Fix trailing whitespace before depends * [FIX] Fix ir_act_window_view template * [FIX] Remove CDATA for type=hml xml node * [FIX] Fix ir_action_windows_view template and domain * [FIX] Fix model_name in ir.ui.view template * [FIX] Fix ir_model_access model name when exported * [FIX] Translate field into module field in the context * [FIX] Fix base_automation export that doesn't generate anything * [FIX][Scaffold] Fix view name to follow standard * [ADD] Add autoflake to remove unused import * [FIX] improve ir_actions_act_window.jinja template * [FIX] exporter: do not export ir.filters by default Co-authored-by: nda-odoo <[email protected]> [FIX] Last run not working with venv (#15) First issue is that by default if the last_run is not in the config it is set to now. This causes an issue with venv where it compares the date of edition of the file with the last run in order to trigger or not the installation of the requiremements. If by default it is now then it will never trigger the installation. The second issue is that the last_run is not updated when the command run is executed as you have to ctrl-c to stop the execution of Odoo it kills odev before it has time to update the last_run. I added an upgrade script to reset all last_run date as to ensure all venv are up to date. Third issue is that when comparin the last_run date with the last edition date of the requirement files it requires that all files have last edition date greater than the last_run. This is ok for the first run but most likely after that only one requirement file will be updated at a time and odev will never pickup those changes. [FIX] Scaffold missing database argument (#18) [FIX] Scaffold path missing database database arg does not exist, so in case of none url_info it can scaffold to args path [FIX] quickstart: restore dump downloaded online (#17) This PR introduced the issue: https://github.com/odoo-ps/ps-tech-odev/pull/9 The name of the dump was previously the name of the database. Since that fix it is the name of the database in the url. [IMP] python.py: log errors if pip install fails and exit [FIX] python.py: ignore special case for util package in requirements `util_package` is added to odoo.sh projects as a submodule with the help of `requirements.txt`, where a line is added that points to a filepath specific for odoo.sh containers. This makes the `pip install` command return a non-zero exit code because of the none existant path on the local machine. Add a special case to check for this until this process changes. [IMP] Enable GitHub workflows and add coverage report [FIX] commands.dump: Namespace has no 'database' parameter (#23) * [FIX] LocalDatabaseCommand: database not required if argument no added Makes sure the `database_required` attribute is set to `False` if the `add_database_argument` attribute is falsy. Introduced by #9 Fixes #22 [FIX] Traceback when user refuses download of daily ackup [IMP] pre-commit config: Set `xmlWhitespaceSensitivity` to "strict" Lines in XML files that exceed 120 characters are formatted by pre-commit by breaking the lines with newline characters where possible. This is generally safe because HTML ignores these newline characters, but it's not safe for inline elements, having the following (known) side effects: - `mail.template`s are broken if the subject for ex. contains a line-break - translation issues depending on the version of Odoo used This can be fixed though by using `xmlWhitespaceSensitivity: "strict"` in the prettier config, see https://prettier.io/blog/2018/11/07/1.15.0.html#whitespace-sensitive-formatting. [IMP] make odev compatible with ubuntu 22.04 (#27) 2 issues so far with odev and ubuntu 22.04: python 3.6 is not available through deadsnakes ppa anymore, see https://launchpad.net/~deadsnakes/+archive/ubuntu/ppa Solution: use python 3.7 instead python3.X-distutils is not installed by default anymore handle setuptools 58 not being compatible with python version lower than 3.8 [IMP] pre-commit config: use the standard .flake8 config odev pre-commit was ignoring unused imports because of the flake8 config, which is fixed now to use the standard version we have, plus set flake8 to ignore F403 in `__init__` files only. [IMP] *: remove unused imports [IMP] odoo.py: more robust url parsing Currently 'get_database_name_from_url' fails if the url is for ex `https://test.odoo.com/` because of the trailing slash. Fixed by using urllib which allows to refactor `sanitize_url` too. [FIX] Fix wrong type rendering (Xml in py files) [FIX] Don't print warning if there is no integration line [ODEV][ADD] shortest_path: shortest path between two models (#16) * [SHELL][TOOL] shell command: --script args + ShortestPathCommand * [SHELL][TOOL] shortest_path: check if model defined in DB --------- Co-authored-by: Rodesc <[email protected]> [IMP] dump: add exception if no access to the dump route (#35) For both SH and SAAS add a clear message about why the dump is not downloading [IMP] Add error message when virtualenv is missing (#37) [IMP] Dispaly proper message when virtualenv creation fails [IMP] restore: add bzip support (#36) [ADD] Ask to Auto clean db older than 10 days (#11) * [ADD] Ask to Auto clean db older than 10 days * [FIX] Add option to whitelist from cleaning * [IMP] Only ask once every 5 days * [REVIEW] Change default value logic * [TYPO] --------- Co-authored-by: ros-odoo <[email protected]> [IMP] odoo.py: handle missing scheme during urlparse Use `tldextract` package to parse a URL instead of urlparse, which fixes any issue when the scheme is not present in the URL or it's a multi-level subdomain. [ADD] pre_commit.py: add pre-commit command This command can be used to install/update the pre-commit configuration to the target version, supplied as an existing Odoo database name or a valid Odoo version. It uses the git repo in the current working directory and automatically creates a commit for the downloaded configuration. Other details: - refactored the fetching of pre-commit configs - not using `GitCommitContext` for better control of exceptions [IMP] odoo.py: fix sanitize url and refactor tldextract Fix `sanitize_url` to work for multi-level subdomains too. [FIX] Fix wrong ir.actions.act_window template (#45) [FIX] .pre-commit-config: bump isort to fix broken dependency Bumping isort to 5.12.0 to fix pre-commit failing to setup env due to dependency incompatibility. See https://github.com/PyCQA/isort/issues/2083 [IMP] quickstart: allow to qs a database with `--no-clean` [IMP] clean: make sure `database.is_neutralized` is set in db - make sure when cleaning a database that in `ir.config.parameter` the parameter with key `database.is_neutralized` is set to `true` (this is used in standard v16 and support team neutralize script). This is in general a stronger indicator that a database was cleaned, vs. other more heuristic checks. - add a `db_is_neutralized` helper method in `LocalDatabaseCommand` to (semi-heuristically) determine if a database was cleaned or not. - bump version to 3.3.3 + add an upgrade script to make sure all dbs that (seem) cleaned have the `database.is_neutralized` set. - fix an issue with `PSQL.query()` that would return always `True` on multiline statements starting with a newline (i.e. strip + dedent as a pre-pass). [IMP] run: add ominous warning + delay when running unclean db [IMP] disallow whitelist option in `_clean_old_db` for unclean dbs [IMP] list: display warning for unclean dbs after the name [FIX] upgrades/3.3.3: ignore errors on non-existing databases (#48) [IMP] Add version increment instructions [ADD] GitHub templates - Multiple templates for issues (bugs and improvements) - Template for pull requests [REM] Pull request template directory: only keep one default [ADD] Generic issue template As issue forms are not supported on private repos, fallback to a generic markdown issue template. [DOC] Add contribution guidelines and simplify the README - Renamed `doc/` to `docs/` - Added `CONTRIBUTING.md` file - Simplified `README.md` file and removed long sections [DOC] Update links to Odev repository Links pointing to the GitHub repository `odoo-ps/odev` have been updated to point to the new repository `odoo-ps/ps-tech-odev`. Also, a quick run of pre-commit was done to fix some formatting issues. [ADD] Setup helper methods (recallable) [IMP] Utilities and refactor of logging [ADD] Temp test entry-point [BUMP] Version: 4.0.0 [REF] main & setup [REF] Refactor self-update Refactor the self-update mechanism to improve reliability and performance while keeping the code readable. The new mechanism is based on the following principles: 1. When odev is run, check its local repository for changes against the remote tracked branch (not targeting a specific branch anymore), but only if the check interval has passed since the last check. 2. Add the possibility for the user to choose whether to download updates automatically, never download updates or ask before downloading. 3. If an update is available and changes are pulled, check whether the requirements.txt file inside of the repo was modified. If so, install the new dependencies. In order to save execution time, changes are not fetched in the current process anymore but in a separate - detached - subprocess. This implies that updates can be delayed until the next time odev is run but it has the benefit of not slowing down the current execution. [REF] Refactor update and upgrade mechanisms Include update and upgrade of odev in a single framework class. [IMP] Silence logging for external libraries in debug mode [REF] Refactor commands loading and dispatching Replace the registry with the more generic odev framework. Move all existing commands to an `old` directory and create a new `utilities` directory for the new help command (simplified for testing). [REM] Remove manual test files Odev can now be tested manually using the system's `odev` command. [IMP] setup.directories: consistent ask order for paths Retry asking for the same path until a valid value is inputted instead of asking all paths and validating them at the end. [IMP] setup.update: Default to values already in config file Use values from the config file as default instead of hardcoded values. [REF] Various changes and improvements - [IMP] Styling for `prompt` and `logging` with shared colors - [REF] Replace occurrences of `str(Path)` with `Path.as_posix()` - [IMP] Add support for various prompt types (text, checkboxes, etc.) - [FIX] pre-commit flagged errors [IMP] common.prompt: Remove unused __all__ attribute [FIX] pre-commit flagged errors [REF] Restore old_commands to commands [REM] constant.arguments: help argument aliases Moved to commands.utilities.help [REF] commands.help [IMP] common.odev: include assignation of framework in preparation [IMP] common.commands.base: catch errors when parsing arguments [IMP] commands.utilities.help: refactor string manipulation Add new methods under common.string to manage indents and dedents without requiring to import textwrap at all occasions. [ADD] commands.utilities.setup: re-run odev's setup after install [IMP] Add handling of command runtime errors Add a new CommandError class that is caught when a command is run. This allows to handle errors in a more generic way, and to display an error log instead of a stack trace. Add a new method `error()` to the BaseCommand class to generate a CommandError without having the need to import the class. [MOV] Ignore commands that still need to be refactored [IMP] common.commands.base: page long text when printing to terminal When trying to print a long text to the terminal, use a pager (usually `less` on unix systems) if the text is bigger than the space available in the terminal. [REF] Export HelpCommand.__min_indent() to common.string [FIX] Handle log level correctly in arguments Remove the log level from command arguments after setting the global level across the application. [REF] Remove commented code [FIX] Reset log level when restarting odev [REF] Rename BaseCommand to Command [ADD] Unit tests for odev.comm and odev.commands [REF] common.commands.base: Refactor _is_abstract attribute Replace the `Command._is_abstract` attribute with a class method. [ADD] Connectors and connector mixins Add a PostgreSQL connector to interact with PSQL and add a mixin to apply this connector to commands instances. [REF] Command class import Make it less verbose. [ADD] .coveragerc Exclusion rules for coverage. [IMP] Dynamic actions mapping for arguments parsing Remove OptionalStringAction and a new RegexAction, as well as generating a name for each custom action and mapping it to the action class. This allows to use actions the same way as argparse standard actions, and removes the need from importing the custom actions. [FIX] ignore F405 to generated connectors.pyi definitions [IMP] common.connectors: add SQL query execution to Postgres connector Add a decorator to the connectors mixins to ensure that the connector is connected before executing queries and/or requests. [ADD] Odoo-specific version parsing [ADD] common.databases: classes for handling local databases Providing a name during instancing will open a connector to a local running service of PostgreSQL. [IMP] common.commands: print tables to the console Add a method to the base Command class to print tables from a list of column titles and rows. [ADD] common.commands: Database command with additional database arg [ADD] commands.local.list: simple version of the list command [IMP] Improve existing tests for python and setup commands [ADD] Unit tests fixtures and new tests [IMP] commands.local.list: Add information and filter non-odoo databases [MOV] command.utilities.list -> commands.local.list [IMP] Repr for commands and databases [IMP] main: measure process execution time [FIX] mixins.connectors: raise if not connected [FIX] Set current log level in commands arguments [IMP] test.common.odev: test upgrade features [ADD] commands.kill: kill a running odoo-bin process Add a new class for representing an odoo-bin process and interacting with it. [ADD] commands.run: draft for run command starting an odoo-bin process [IMP] flake8 and pre-commit config Ignore unused imports in `__init__.py` files and type check methods with untyped definition. [IMP] upgrades 4.0.0: combine directory paths Merge standard and custom directories together under a single path. [IMP] setup.directories: merge directories into one [IMP] databases.postgres: Add helper methods for common SQL operations And other misc. changes. [IMP] connectors.postgres: add helper methods and improve caching LRU caching from functools was used to cache all SQL queries. This was not a good idea because it was not possible to invalidate the cache and it was not efficient because of context managers which made the `self` object to be generated each time a new connection was open. The commit adds a simple cache system which can be bypassed easily by setting the `nocache` attribute to `True`. Only `SELECT` statements are cached to let alteration statements to be executed at all times. [IMP] mixins.connectors: Split connector mixins into their own files Remove the dynamic generation of custom attributes typing, let the developers handle it. [IMP] common.config: Add repr and global config object [IMP] common.logging: Context manager to silence loggers temporarily [IMP] common: Add repr to misc classes [IMP] main and setup: use the global config instance [ADD] Github connector and secrets vault New connector to the Github API and to manage local Git repositories. Handles the clone and pull of repositories, and the creation of worktrees. The secrets vault is a new module to manage credentials and secrets in a secure way. It uses the the local SSH agent to encrypt secrets and stores them in a local PostgreSQL database. [IMP] common.bash: new method to stream output Add a new `bash::stream` method to capture output of a subprocess line by line, allowing to manipulate data as it is being generated. [IMP] common.python: improvements to requirements installation Allow to find out if requirements are missing or already installed. Display progress while installing packages from a `requirements.txt` file. [ADD] odev.common.commands.odoobin: Add odoobin command class New class to interact with local databases through odoo-bin. Add new helper methods in `common.odoo` to setup virtual environments and Git worktrees automatically before running odoo-bin. [ADD] commands.run: Run a local odoo database through odoo-bin [IMP] commands.kill: adapt kill to use OdoobinCommand [FIX] Tests [IMP] Synchronize repositories and worktrees if deleted from elsewhere When a repository is removed from the filesystem `odev` will attempt to clone it anew. This might lead to errors when creating worktrees if the old worktree directory is still present. This commit checks for existing worktrees and removes them before cloning the repository. When a worktree is removed from the filesystem but is still referenced in Git, `odev` will attempt to load and use it. This might lead to errors where we are loading a non-existing path. This commit checks for existing worktrees references and removes them before re-creating the worktree. Because fetching changes in repositories is done in a detached process, `odev` does not wait for the process to end before running the next operation. If a call to :func:`fetch` or :func:`fetch_worktrees` is followed by a call to :func:`pull` or :func:`pull_worktrees`, the fetch subprocess might still be running when the pull is executed. This leads to situations where the pull is executed on a repository that is not entirely fetched yet and Git raises an error stating that rebase cannot be performed on two different branches at once. This commit inverts calls to fetch and pull so that the fetch is always executed *after* the call to git pull. Changes pulling might be delayed until the next time `odev` is run. [ADD] command.shell: support for `odoo-bin shell` Allow passing `odoo-bin` subcommands to :func:`OdooBinProcess.run()`. [ADD] commands.cloc: Count custom lines of code in Odoo databases Adapt the old `odev cloc` command to the new framework with display improvements and spinner during loading of information. Add spinners on several time-consuming operations like preparation of virtual environments before running `odoo-bin`. [IMP] Add checks on existing and running odoo-bin databases Prevents calling commands if conditions on existing and running databases are not met. [IMP] Requirements and forcing enterprise edition on OdooBinProcess Improve `requirements.txt` installation in python virtual environments and add default packages to each environment to avoid manual intervention on each first run in the environment. Also, force the use of enterprise edition when using OdooBinProcess if a command is run with the `--enterprise` flag. fixup [ADD] Context manager for spinner animations on long running tasks [IMP] SQL queries context Run SQL queries in transactions and rollback if necessary Make sure SQL queries are surrounded by `BEGIN` and `COMMIT` statements (or `ROLLBACK` if an error ocurred). Queries are now run in a new thread to allow cancelling long running operations (i.e. creating a database from a large template). This is an improvement for security and user experience, not performance hence the thread is called in a blocking way and no code is executed in parallel in the parent thread. [ADD] commands.create: Create and initialize databases [IMP] Allow forcing Odoo version to run odoo-bin A version can be provided to odoo-bin commands to force the version used which also allows to run odoo-bin with the master version. [ADD] Commands history Keep an history of the commands executed and there last execution date. This enables knowing when a command was last run on a database and provides more accurate "last access" information for the `list` command. [IMP] Framework improvements: Split `PostgresDatabase` into `LocalDatabase` and `PostgresDatabase` to allow reusability of the `PostgresDatabase` class for other non-Odoo databases. Make the Odev framework global and accessible from anywhere through the `OdevFrameworkMixin` mixin class. This adds easy to access properties for the configuration and the datastore. Move datastore models to a new store instance with a more consistent way of creating and accessing tables. Harmonize the use of properties instead of methods when possible. [FIX] Check if database exists before checking if it is an Odoo database [FIX] Remove `nargs` key in `create` command arguments [FIX] Handle partially initialized databases Databases that have been killed before the base Odoo module was properly installed have no version number while the row exists in Postgres. This commit handles this case by forcing the version number to `None` if a result was returned with no value. [IMP] Double confirmation before removing whitelisted databases [IMP] Harmonize logger and include loading time in performance log Use `logger` instead of `_logger` in `__main__.py` and `__setup__.py` to keep consistent with other files in the framework. Also, include the loading time in the performance log and dynamically import the framework to measure the time it takes. [ADD] commands.database.test: Run tests on an empty copy of the database Copy the selected database configuration and initialize a new database with no data. Install the selected modules and run unit tests. Capture the output and display it in the terminal and keep track of failing tests to display a summary at the end of the command. Delete the test database after the tests have been run. Odoo-bin and other python scripts can be run with a progress callback that is called on every line outputted to STDOUT or STDERR by the script. [ADD] commands.database.rename: Add command to rename a local database Alter the name of the database in PostgreSQL and in the data store. Move the filestore to its new logical location. [IMP] Improve progress display and avoid multiple live-display errors Override Rich's Status class to avoid multiple live-display errors by stacking spinners and pausing them when a new one is started and resuming them when the last one is finished. Also, add a new Progress class to display a progress bar with default styling applied. The progress also automatically pauses running spinners and resumes them after completion. [FIX] store.secrets is a table and not a database Call queries on the `odev` database instead of the old `secrets` database that does not exist anymore. [REM] Remove the old `rename` command [IMP] databases: uniformize database classes (prepare remote connectors) Add a `platform` attribute on database classes. Uniformize database classes methods by adding default methods and properties to the base `Database` class. [IMP] databases: unaccented schema in database class Move the creation of the `unaccent` schema to the database class instead of the `create` command. [IMP] commands.list: Display totals under size columns Count the number of databases and add a total row under the database list table. [IMP] commands: Move database existence check to DatabaseCommand [IMP] databases: dump a local database Allow creation of a dump of a local database on the filesystem. [FIX] Gracefully handle errors during arguments parsing [IMP] Do not update secrets when not changed [FIX] Do not fetch database info for non-existing local databases [ADD] databases.saas: Support for SaaS databases Add a SaaS connector to interact with `_odoo/support` pages using stored credentials. Add a SaaS database class with access to the connector. Infer the database type from the name passed to `DatabaseCommand` instances. [ADD] com…
- Loading branch information
0 parents
commit 1efb8c3