Docker Hub automation automation

[weinholt]

We will have a large number of automated builds on Docker Hub, because I think we will want to keep using Docker's public build infrastructure. Using their infrastructure is some kind of guarantee that we haven't slipped anything unexpected into the builds ourselves.

But it is cumbersome to set up Docker Hub automation by hand for over 40 images. Luckily, someone has made a Node.js library that can do the job via an API:

https://github.com/RyanTheAllmighty/Docker-Hub-API

Because we will have a tool to generate Dockerfiles, I think we can also use the input to that tool to automate creation of automated builds. The tool would need to know which repositories and branches there are with Dockerfiles and which build tags it should create.

@lassik How goes the work on the tool to generate Dockerfiles, is it available somewhere?

@guenchi I think you know Node.js, is this issue something you would be interested in working on?

[guenchi]

Yes, I’m interested. I think I can port your work to github actions. Tell me what can I do?

…

2019-12-24，09:25，Göran Weinholt ***@***.***> ： We will have a large number of automated builds on Docker Hub, because I think we will want to keep using Docker's public build infrastructure. Using their infrastructure is some kind of guarantee that we haven't slipped anything unexpected into the builds ourselves. But it is cumbersome to set up Docker Hub automation by hand for over 40 images. Luckily, someone has made a Node.js library that can do the job via an API: https://github.com/RyanTheAllmighty/Docker-Hub-API Because we will have a tool to generate Dockerfiles, I think we can also use the input to that tool to automate creation of automated builds. The tool would need to know which repositories and branches there are with Dockerfiles and which build tags it should create. @lassik How goes the work on the tool to generate Dockerfiles, is it available somewhere? @guenchi I think you know Node.js, is this issue something you would be interested in working on? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[weinholt]

Let's see what @lassik has been working on.

Meanwhile, you could try to get GitHub actions working with the images at https://hub.docker.com/u/weinholt

The R6RS images are set up so that Scheme programs with the line #!/usr/bin/env scheme-script work out of the box. This sort of thing is important so that test suites can run without special adaptation to each Scheme implementation (but we should be better about it for the non-R6RS Schemes). One of the goals for this project has been to make it easier to write portable Scheme code that works across multiple implementations. I have only tried to do this with R6RS code until now. I think this affects how users would perceive the GitHub Actions support for Scheme.

The plan going forward is to move everything to https://hub.docker.com/orgs/schemers and keep the Dockerfiles somewhere in this scheme-containers project. I created another issue for discussing how to arrange the branches or projects. Maybe there are some considerations with the Docker Hub API that makes one approach easier than another approach?

[lassik]

@weinholt Once again, sorry about the delay with this. I've been procrastinating on it, no excuse. The R7RS code is in a shape that I've already built some containers with it, just need to finish the Akku wrapper. I'll try to get to it ASAP.

The plan going forward is to move everything to https://hub.docker.com/orgs/schemers and keep the Dockerfiles somewhere in this scheme-containers project.

👍

It might be easiest to keep the generator script in implementation-metadata so it can co-evolve with the files describing implementations. We could also move the script (and/or the Docker-related metadata) under the scheme-containers GitHub organization, but I don't see any particular upside to it. If it's a matter of permissions, I can simply grant write access to implementation-metadata for anyone who needs it. The GitHub orgs don't have any formal chain of command or anything, they're just informal hubs for getting things done.

As for separating the Docker-related metadata from other metadata, I would prefer to keep all metadata in the same file since there's quite a lot of overlap, and if there's only one file per implementation it's easiest to remember to keep everything in sync and debug any problems when there are not possibly divergent versions of related files.

[lassik]

If Docker Hub needs a particular repo/branch layout in order to build things, it's probably best to establish those repos under the scheme-containers org and automate commits/triggers as you say.

[weinholt]

More delays happened, but now I have set up some kind of Docker hub integration for us and tried it out with Loko Scheme. I tried to follow Docker's instructions for doing this and they just don't work. (Halfway through they start talking about things that don't exist). But I got it working.

The situation today:

• A scheme-docker-bot user on GitHub that has read/write access to the scheme-containers projects. Docker Hub wants the read/write access to set up CI.
• The scheme-docker-bot user is easily replaceable, but I have added @lassik on the email alias just in case.
• I've made the scheme-containers/loko repo where I just pushed the previous loko branch as master.
• There is a "schemers" team on Docker Hub (feel free to request access so you can manage builds, right now it is me and @lassik).
• The name "scheme" on Docker Hub is still hogged, it would be nice to get the support to hand it over to us. :)
• An automated build for the Docker Hub repo schemers/loko as a proof of concept.

The remaining work is to create GitHub repos for the old branches on weinholt/scheme-docker and then to set up Docker Hub automation for each repo.

[lassik]

• The name "scheme" on Docker Hub is still hogged, it would be nice to get the support to hand it over to us. :)

One thing we could try is to pay for a month or two of service in case they respond to support requests from paying customers. (Unpaid support requests seem to go straight into the round file, which I can't really complain about, though it would be nice if they don't give false hope by giving unpaid users a feedback box :) But before paying, it would be nice to get confirmation from someone who did actually get an answer on a particular pricing tier. It could be that requests on the cheapest tiers are ignored as well.

[lassik]

Here are their pricing tiers: https://hub.docker.com/pricing From "large" upwards it's really expensive (upwards of 50 USD/month). Not fun to try as an experiment. I get the impression they heavily sell to "enterprise" customers now, so I wonder if they give proper and timely responses to support tickets on the lower tiers.

[lassik]

There is information at https://hub.docker.com/support/ conflicting the squatting policy I saw in their terms of service:

How can I claim ownership of an existing Docker ID?

All Docker IDs are first come, first serve except for companies that have a US Trademark on a username. If you have a US Trademark claim on a name, open a support ticket and include:

• The username you wish to claim
• Proof of US Trademark on the username

[lassik]

(My comments about the scheme username should be moved into a separate issue, but I can't find a move command.)

[weinholt]

@lassik It looks like Docker Hub has disabled automated builds for this project by now, since they started charging for the feature.

What are the hopes for getting the project approved under their open source support program?

If that does not work out, then I suspect we can use GitHub Actions to build and push Docker images.