extra_env_vars support fnmatch globs #21781
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
lilatomic
force-pushed
the
feature/envvar_globs
branch
from
2327eef to
47dc38c
Compare
cognifloyd
approved these changes
Dec 19, 2024
huonw
reviewed
Dec 19, 2024
There was a problem hiding this comment.
Nice, I like it.
Can we expand the docs of the relevant options/fields to mention they support globs? Otherwise people coming in new (and/or not reading release notes in detail) won't be aware of this handy functionality.
(And/or, maybe there's a relevant .mdx doc file, although I couldn't find one from a quick skim just now.)
| @@ -62,6 +64,12 @@ def check_and_set(name: str, value: Optional[str]): | |||
|
|
|||
| return FrozenDict(env_var_subset) | |||
|
|
|||
| def get_or_match(self, name_or_pattern: str) -> dict[str, str]: | |||
There was a problem hiding this comment.
Totally minor, but given this is just used as a .items(), maybe it could return Iterator[tuple[str, str]], with the body like
if value := self.get(name_or_pattern):
yield name_or_pattern, value
for k, v in self.items():
if fnmatch.fnmatch(k, name_or_pattern):
yield k, v| @@ -37,3 +37,20 @@ def test_invalid_variable() -> None: | |||
| "An invalid variable was requested via the --test-extra-env-var mechanism: 3INVALID" | |||
| in str(exc) | |||
| ) | |||
|
|
|||
|
|
|||
| def test_envvar_fnmatch() -> None: | |||
There was a problem hiding this comment.
Could we add a test of the exact-matching behaviour? E.g. include an env variable LETTER_P* and call get_subset with that exact name, to validate that LETTER_PI isn't included?
| @@ -21,6 +21,7 @@ Thank you to [Klayvio](https://www.klaviyo.com/) and [Normal Computing](https:// | |||
| - Fixed a longstanding bug in the processing of [synthetic targets](https://www.pantsbuild.org/2.24/docs/writing-plugins/the-target-api/concepts#synthetic-targets-api). This fix has the side-effect of requiring immutability and hashability of scalar values in BUILD files, which was always assumed but not enforced. This may cause BUILD file parsing errors, if you have custom field types involving custom mutable data structures. See ([#21725](https://github.com/pantsbuild/pants/pull/21725)) for more. | |||
| - [Fixed](https://github.com/pantsbuild/pants/pull/21665) bug where `pants --export-resolve=<resolve> --export-py-generated-sources-in-resolve=<resolve>` fails (see [#21659](https://github.com/pantsbuild/pants/issues/21659) for more info). | |||
| - [Fixed](https://github.com/pantsbuild/pants/pull/21694) bug where an `archive` target is unable to produce a ZIP file with no extension (see [#21693](https://github.com/pantsbuild/pants/issues/21693) for more info). | |||
| - `[subprocess-environment].env_vars` and `extra_env_vars` (on many subsystems) now supports a generalised glob syntax using Python [fnmatch](https://docs.python.org/3/library/fnmatch.html) to construct patterns like `AWS_*`, `TF_*`, and `S2TESTS_*`. | |||
There was a problem hiding this comment.
I think many targets have extra_env_vars fields too, so maybe:
Suggested change
| - `[subprocess-environment].env_vars` and `extra_env_vars` (on many subsystems) now supports a generalised glob syntax using Python [fnmatch](https://docs.python.org/3/library/fnmatch.html) to construct patterns like `AWS_*`, `TF_*`, and `S2TESTS_*`. | |
| - `[subprocess-environment].env_vars` and `extra_env_vars` (on many subsystems and targets) now supports a generalised glob syntax using Python [fnmatch](https://docs.python.org/3/library/fnmatch.html) to construct patterns like `AWS_*`, `TF_*`, and `S2TESTS_*`. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
It first checks for exact matches, then tries an fnmatch.
closes #20291