Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: Enable use of latest: in .tool-versions files #1793

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

feat: Enable use of latest: in .tool-versions files #1793

wants to merge 1 commit into from

Conversation

rtyley
Copy link

@rtyley rtyley commented Oct 17, 2024 •
edited
Loading

Summary

Implements #1012 , and addresses some of the underlying needs in #1235, #1342, #1736, etc

This change enables asdf's existing latest-version-resolution functionality within the .tool-versions file itself. Rather than having to specify a full version number in the file:

java corretto-21.0.5.11.1

...you can now use the same latest: syntax that is already available in the local & global commands, ie:

java latest:corretto-21

Use case

For many tool/runtime ecosystems (eg Java), if a program runs correctly under a specific version of that runtime, it can generally be relied on to run correctly under any later version of that runtime with the same major version number (eg if a project runs under Corretto Java 21.0.5.11.1, it will run on any later version of Corretto Java 21).

This means that for projects in those ecosystems, there is little incentive to pin to fully-specified versions like 21.0.5.11.1, and in fact there are downsides - over time, developers will default to using older, unpatched versions of Java, unless they are assiduous in continually updating the contents of the .tool-versions file, or have tooling devoted to doing so.

At the Guardian we have several hundred projects that run on the Java platform, and due to our security obligations we generally want to be running under the latest security-patched version of the Java runtime that matches our major-version requirement. We love asdf as a tool, and like that the .tool-versions file can become a source-of-truth documenting which version of Java a project uses, but we don't want to have to commit fully-specified version numbers like 21.0.5.11.1 to source control, or set up tooling to increment those version numbers across those hundreds of repositories.

Allowing the use of latest: in the .tool-versions file means that we don't need to continually update those .tool-versions files. It also partially addresses some of the needs raised by #1736, though this solution uses the existing asdf version-resolution functionality, rather than adopting nodejs version syntax (^1.1.0, etc).

Determinism & variability of version - could we let the user decide?

In the past, the maintainers of asdf have argued against this kind of change:

Special versions like latest are resolved at the time the asdf command specifying them is run and are not permitted to in .tool-versions because of the nondeterminism they would introduce. We don't want to allow any variability in versions specified in any of the version files that asdf supports because we want asdf to be deterministic. We don't want a new version release to accidentally pull the rug out from under someone because they had specified a shorthand version or version range.

...and also:

asdf is designed to ensure everyone on a team is using the exact same version. If I am using [email protected] and you are using [email protected] those two versions are not anywhere close to being the same. They have different V8 engines, many different features and numerous bug fixes etc. How can we determine the root cause with this variability? Solution, remove the variability. Yes, this makes asdf slightly less convenient to use, but the purpose of locking a team to a version is to lock them to a version. Version ranges introduce variability.

I think there are at least some users who would accept some variability in exchange for staying up-to-date, and it's worth considering whether it's reasonable to allow them to select that. In terms of the existing latest: syntax already used by asdf, the constraint can be made quite tight (eg. java latest:corretto-21.0.5) or looser (eg. java latest:corretto-21) - so the choice of how much variation they want to accept can be expressed by the user.

In some ecosystems - like Java - if the version number is constrained to particular LTS major version, the only expected updates are security updates. At the Guardian, we do want to be using those latest security updates, and so would use a constraint like latest:corretto-21.

Implementation

A new resolve_version_spec() function has been extracted from the existing version_command() function. This takes a version-spec string, like latest:corretto-11 or corretto-21.0.5.11.1, and resolves it to a precise version number.

To support latest: syntax in .tool-versions, the new resolve_version_spec() function is now called in select_version(), used by with_shim_executable(), meaning that any execution of the asdf shim (eg, executing java) will now resolve any version specifications found in the .tool-versions file - if .tool-versions contains java latest:corretto-21, this will be resolved and the latest version of Java 21 used.

Testing

Tests have been added for these asdf commands to ensure they can handle version resolution in the .tool-versions file:

  • exec
  • install
  • current

Other Information

Previous asdf PRs relating to latest:

Previous asdf issues relating to keeping up with latest-and-greatest versions:

A couple of Guardian systems attempting to standardise on using .tool-versions as a source of truth:

cc @adamnfish @akash1810 @tjsilver

@rtyley rtyley changed the title feat: use of latest: in .tool-versions files feat: Enable use of latest: in .tool-versions files Oct 17, 2024
@rtyley rtyley force-pushed the support-latest-in-tool-versions branch 10 times, most recently from d767058 to 8b175d3 Compare October 18, 2024 16:47
rtyley
rtyley commented Oct 18, 2024
View reviewed changes
test/test_helpers.bash
Comment on lines +7 to +8
# shellcheck source=lib/functions/versions.bash
. "$(dirname "$BATS_TEST_DIRNAME")"/lib/functions/versions.bash
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is required because utils.bats has a test that invokes with_shim_executable() (which itself now calls resolve_version_spec() in versions.bash).

Without this change, the test fails:

 ✗ with_shim_executable doesn't crash when executable names contain dashes
   (in test file test/utils.bats, line 460)
     `[ "$status" -eq 0 ]' failed

@rtyley rtyley force-pushed the support-latest-in-tool-versions branch from 8b175d3 to 92a7bf2 Compare October 18, 2024 17:19
@rtyley rtyley marked this pull request as ready for review October 18, 2024 17:25
@rtyley rtyley requested a review from a team as a code owner October 18, 2024 17:25
@rtyley
feat: Enable use of latest: in .tool-versions files
bc95d68 
This change enables `asdf`'s existing latest-version-resolution
functionality within the `.tool-versions` file itself. Rather than
having to have a `.tool-versions` file that contains a full version
number:

```
java corretto-21.0.5.11.1
```

...you can now use the same `latest:` syntax that is already available
in the `local` & `global` commands, ie:

```
java latest:corretto-21
```

### Use case

For many tool/runtime ecosystems (eg Java), if a program runs correctly under
a specific version of that runtime, it can generally be relied on to run
correctly under any _later_ version of that runtime with the same major version
number (eg if a project runs under Corretto Java 21.0.5.11.1, it will run on
any _later_ version of Corretto Java 21).

This means that for projects in those ecosystems, there is little incentive
to pin to fully-specified versions like `21.0.5.11.1`, and in fact there are
downsides - over time, developers will default to using older, unpatched versions
of Java, unless they are assiduous in continually updating the contents of
the `.tool-versions` file, or have tooling devoted to doing so.

At the Guardian we have several hundred projects that run on the Java platform,
and due to our security obligations we generally want to be running under the
_latest_ security-patched version of the Java runtime that matches our
major-version requirement. We love `asdf` as a tool, and like that the
`.tool-versions` file can become a source-of-truth documenting which version
of Java a project uses, but we don't want to have to commit fully-specified
version numbers like `21.0.5.11.1` to source control, or set up tooling to
increment those version numbers across those hundreds of repositories.

Allowing the use of `latest:` in the `.tool-versions` file means that we
don't need to continually update those `.tool-versions` files. It also
partially addresses some of the needs raised by asdf-vm#1736,
though this solution uses the existing `asdf` version-resolution functionality,
rather than adopting the version requirements system used in nodejs.

### Implementation

A new `resolve_version_spec()` function has been extracted from the
existing `version_command()` function. This takes a version-spec string,
like `latest:corretto-11` or `corretto-21.0.5.11.1`, and resolves it to
a precise version number.

This new `resolve_version_spec()` function is now also called in
`select_version()`, used by `with_shim_executable()`, meaning that any
execution of the `asdf` shim (eg, executing `java`) will now resolve
any version specifications found in the `.tool-versions` file - if
`.tool-versions` contains `java latest:corretto-21`, this will be
resolved and the latest version of Java 21 used.

## Other Information

Previous `asdf` PRs relating to `latest`:

* asdf-vm#575 in November 2019: added the `latest`
  command, eg `asdf latest python 3.6` reports the latest version of Python 3.6.
* asdf-vm#633 in July 2021: made it possible
  to specify `latest` when using the `local` & `global` commands, eg:
  `asdf local python latest:3.7` - this would save a precise version number
  to `.tools-versions`, which is undesired behaviour for us at the Guardian.

A couple of Guardian systems attempting to standardise on using `.tool-versions`
as a source of truth:

* guardian/gha-scala-library-release-workflow#36
* https://github.com/guardian/setup-scala
@rtyley rtyley force-pushed the support-latest-in-tool-versions branch from 92a7bf2 to bc95d68 Compare October 22, 2024 09:39
rtyley
rtyley commented Oct 22, 2024
View reviewed changes
lib/utils.bash
Comment on lines -739 to +740
for plugin_version in "${usable_plugin_versions[@]}"; do
for version_spec in "${usable_plugin_versions[@]}"; do
plugin_version="$(resolve_version_spec "$version_spec")"
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the key change to enable the desired feature of this PR - now resolving each version spec in the .tool-versions file to a precise version number, rather than it having to already be a precise version number.

@rtyley rtyley mentioned this pull request Oct 24, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@rtyley