Use bundled releases for v2

[MetricMike]

Is your feature request related to a problem? Please describe.

asdf-awscli currently builds from source and requires an external Python.

awscli v2 bundles all dependencies, and provides packages for linux, macos, and windows. This should be the preferred method.

Describe the solution you'd like

asdf install awscli 2.x.y uses the prebuilt releases, and does not require an external Python for:

• linux (thanks @iamhsa!)
• macos (thanks @elementalvoid!)
• windows

See discussion at asdf-vm/asdf-plugins#222

[jthegedus]

I did't mean to badger you into this feature 😅 I am just anticiapting pushback from the restrictions v1 would impose on v2 usage (I know some people who would not use this if v2 didn't use the pre-built libraries)

I like this proposal of handling v2 differently 💯

[elementalvoid]

@MetricMike 👋

You mentioned elsewhere that

Unfortunately, I don't have a mac environment to work with

I'm absolutely happy to help in any way I can. I run Mac for work and would love love love to stop using brew to manage my awscli! I just installed your plugin and will report anything I find in your repository.

---

I did some quick poking around and found the following (crude method) to work for using the pre-packaged Mac bundle.

# hack over the existing asdf install ; }
$ cd ~/.asdf/installs/awscli/2.1.29

# grab the package
$ wget https://awscli.amazonaws.com/AWSCLIV2-2.1.29.pkg

# extract it
$ xar -x -f AWSCLIV2.pkg

# extract the "Payload" -- the prebuilt binary distribution
$ gunzip -dc aws-cli.pkg/Payload | cpio -i

# It works?
$ ./aws-cli/aws --version
aws-cli/2.1.29 Python/3.8.8 Darwin/20.3.0 exe/x86_64 prompt/off

# the aws_completer is also available
$ file ./aws-cli/aws_completer
./aws-cli/aws_completer: Mach-O 64-bit executable x86_64

# update the 'list-bin-paths'
$ sed -i -e 's:venv/bin:aws-cli:' ~/.asdf/plugins/awscli/bin/list-bin-paths

# reshim
$ asdf reshim awscli

# Success!
$ asdf which aws
/Users/user.name/.asdf/installs/awscli/2.1.29/aws-cli/aws

$ aws --version
aws-cli/2.1.29 Python/3.8.8 Darwin/20.3.0 exe/x86_64 prompt/off

[elementalvoid]

If that seems like a reasonable mac-specific, v2-specific approach I would be willing to create a PR for it. I've not experimented at all with Linux or Windows variants so can't speak to what a similar bundle-based approach might look like there.

[MetricMike]

Hello, @elementalvoid! 👋👋👋

Thank you so much for your workflow! I started down that approach originally and found https://stackoverflow.com/questions/41166805/how-to-extract-contents-from-payload-file-in-a-apple-macos-update-package which indicated that the PKG format has changed a bit so xar/gunzip is not quite the solution. Which macos version did you execute your workflow on?

That does confirm that what I was doing in b077b47 to avoid needing sudo isn't totally wild and I think the latest commit on the main branch should be fine for most modern (10.10+) macos systems.

Changes to your workflow as posted:

1. I switched from wget to curl because it's my understanding that wget is not standard on macos systems
2. Pushed the bin path update over to ./list-bin-paths
3. I don't think you explicitly need to call reshim? That could be my individual setup though

GitHub Actions says everything is working the way it should for macos/v2, and not affecting ubuntu/v1, ubuntu/v2, or macos/v1 installations (via pip). Could you try that out and see if anything looks odd? I'm a little concerned that I'm not actually cleaning up all the downloaded files I should be at https://github.com/MetricMike/asdf-awscli/blob/main/lib/utils.bash#L68.

When testing, if you haven't updated to the latest commit on asdf, you'll need to explicitly update to the main branch by using asdf plugin update awscli main. ASDF recently added support for main branches in asdf-vm/asdf#800 but a new release hasn't been cut yet for folks who aren't using asdf update --head.

I also updated the github actions on this repo so v1 and v2 are tested separately, so you should be able to fork and tinker to your heart's content :)

[elementalvoid]

This looks awesome! I'll test in the morning and let you know how it goes -- though from my read of code it should .. just work.

I guess I totally missed the expand-full option when I was playing with pkgutil because it's where I went looking first. I only went with xar and friends later. Oh well! Glad you got that cleaned up.

I am running Big Sur with the gnu tools from brew. They are prepended into my $PATH.

The wget, sed on list-bin-path, and reshim were all side effects of me mucking about inside your plugin's install path. So are definitely not to be considered for a real implementation. 😬

Also, thanks for the pointer to the work on master vs main. I knew it would hit but hadn't noticed it yet!

[elementalvoid]

^^ I did find a couple small breaking issues.

[jthegedus]

Interesting, I would have expected the method for macos and linux to be the same. I might take a stab at linux soon and merge the code paths if sufficiently similar.

[iamhsa]

Thanks for your work !
In case you are interested, I made some modifications to use the bundled version on Linux.
Let me know if you are interested in a PR.

See : https://github.com/iamhsa/asdf-awscli/tree/linux-v2-bundled-release

[MetricMike]

@iamhsa that'd be great!

Taking a quick look at your branch, I'd leave the list-bin-paths alone (cause v1 and v2 windows still need venv/bin), but otherwise it should be a simple and clean merge!

[iamhsa]

@MetricMike you're right ! I forgot Windows OS 😕
I look for this.

[iamhsa]

@MetricMike : what is the use case in windows?

As I understand it, asdf-vm is installable on Linux and MacOs.
So if I understand correctly, on Windows it is supposed to be used with WSL: which is a Linux (uname -s = linux).

For me, I think v2 is installable with bundle versions, and v1 with python venv and pip install.

Is there something that I did not understand?

[MetricMike]

ASDF got windows support in asdf-vm/asdf#451 and awscli-v2 has official support for windows in both v1 and v2, so this plugin should support windows as well.

Current support is provided with venv/pip, but v2 should use the bundled release too eventually. I'm happy working on that as a separate/future issue if you just want to open a PR for linux though!

[iamhsa]

@MetricMike
Thank you for clarification !
👌

[jthegedus]

I do not beliebve we fully support Windows on asdf. WSL on Windows works and is the recommended usage. The linked PR was to add some tests, which there are many broken and we have yet to fix. Sorry for the confusion.

[bdellegrazie]

@MetricMike Any chance of having #4 merged?

[MetricMike]

@bdellegrazie done! Thanks for the reminder and PR!

[bdellegrazie]

@MetricMike just for clarification, it was @iamhsa who implemented the PR, not me - so thanks go to @iamhsa
I'm just a grateful consumer in this case - thank you both! :)