Merge master into PytestBenchmarkTesting #582
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Remove unused pyopenssl Signed-off-by: Lukas Puehringer <[email protected]>
Implements basic primitives, defined by the python-tuf Repository abstraction, to read and edit metadata on disk, handling version and expiry bumps, and signature creation, and facilitating snapshot and timestamp creation. And adds exemplary API methods that use these primitives while preserving consistent repo states: - create - add_target_files - add_keys Can be tested with: ``` PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 pytest --noconftest taf/tests/tuf/ ``` More detailed usage docs + migration path TBD... Signed-off-by: Lukas Puehringer <[email protected]>
The original design aimed at separating the concepts of delegation (adding public keys) and signing (using private keys). Since the MetadataRepository assumes that metadata can be signed rightaway after edit (e.g. after having added a delegation), which in turn requires private keys to be available, we might as well conflate these two concepts. The advantage is that the signer cache does not have to be managed independently and is more likely to stay in sync with the delegations. Signed-off-by: Lukas Puehringer <[email protected]>
Signed-off-by: Lukas Puehringer <[email protected]>
This should really happen upstream (see linked issue) Signed-off-by: Lukas Puehringer <[email protected]>
YkSigner provides a minimal compatibility layer over `taf.yubikey`
module functions for use with MetadataRepository.
Even though a yubikey signer implementation (HSMSigner) based on
pykcs11 is available in securesystemslib, YkSigner was added for the
following reasons:
- TAF requires rsa support for yubikeys, but HSMSigner only supports
ecdsa. Adding rsa support to HSMSigner, or providing a custom
pykcs11-based RSAHSMSigner is feasible, and seems desirable, but
requires more effort than this YkSigner did.
- TAF provides a few additional features, like setting up a Yubikey,
changing pins, etc., which will not be added to securesystemslib.
This means the current Yubikey infrastructure based on yubikey-manager
needs to be preserved for the time being. Thus it made sense to
re-use the existing implementation for YkSigner.
- YkSigner show-cases the new Signer API and might be used as blue print
for future Signer implementations in TAF.
This commit adds basic tests with fake and real Yubikey:
```
REAL_YK=1 PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 \
pytest --noconftest taf/tests/tuf/ taf/tests/tuf/test_yk.py -s
```
Signed-off-by: Lukas Puehringer <[email protected]>
This allows running previously added YkSigner tests, but breaks other things, which need change anyway in the course of upgrading to latest tuf/securesystemslib. Signed-off-by: Lukas Puehringer <[email protected]>
Add alternative TUF metadata repo implementation (WIP)
…gated role lookup
…n dates to the new repository class
…ort for delegations
…ass and added tests
The idea being we cover all the key cli commands with tests. - `click` supports CLI tests [1]. To get started, we initialize a `CliRunner` and invoke the taf command that we want to test. Thankfully, testing is relatively straightforward. In cases where the CLI expects a user input, such as pressing ENTER or a "[y/N]" answer, `runner` supports an `input` param that gets passed into the subprocess stdin. Moreover, we can take the output of the cli test and assert the print/logging statements that occurred, which is really cool. This should make adding new cli tests relatively easy. - When asserting CLI output, such as logging statements (when the command began and when it finished executing), with the `caplog` built-in pytest fixture, things get funky since we use `loguru` instead of built-in python logging module. To resolve, we patch the `caplog` fixture in `conftest.py` to point to the `loguru` module. Added a docstring explaining it in more detail in `conftest.py` - Added ~14 cli tests that should cover all the important flows that we use. I managed to get most of them working, but a couple of them seem to be having slight issues with asserts and expected states. I've added comments to those to debug easier. - Since cli tests share a lot of the fixtures that `test_api.conftest` has, slightly re-organized the `test_api` module to avoid duplicating code/functions. The existing tests are now in their own subdirectory (e.g. `test_api\dependencies\api\` `test_api\roles\api`, etc.), while the newly added tests are in the sibling `cli` directory (e.g. `test_api\dependencies\cli\`...). The nice thing is is that this is complementary to the api functions, so when adding a new api test, we can easily add a cli test. [1] - https://click.palletsprojects.com/en/stable/testing/
It was a convenience method used to figure out which paths from repositories.json match which role. Early exit function if repo is bare (to signal that it's currently not supported)
This is because file hashes get calculated differently on Windows vs Linux if line endings are first saved as LF then get converted to CRLF on Windows. To resolve, normalize all line endings to LF.
…get files, even if no targets were modified
…y auth repo updated (#577) * Fix: Fix setting last updated commit when pushing and update when only auth repo updated If only the auth repo was updated, commits from which the update of target repos should start were not being determined correctly. Additionally, checking if repo is a test repo did not work for bare repos. Only update last validated commit if pushing to the default branch. * chore: update changelog * chore: update log message
fix: run validation with --no-deps when pushing
Transition to the newest version of TUF
Sometimes we don't want to run pre-push hook validations when calling out to git push from our programs. To avoid running any git hooks, we can use --no-verify.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description (e.g. "Related to ...", etc.)
Please replace this description with a concise description of this Pull Request.
Code review checklist (for code reviewer to complete)