start of work to add singularity build from Dockerfile support!

[vsoch]

Description of the Pull Request (PR):

This is a prototype or proof of concept that we can build from Dockerfile. The work here currently includes support for FROM/ADD/COPY/ENV/LABEL/CMD/ENTRYPOINT and is setup to also include support for multistage builds (although this needs to be finished and I need to look into how singularity handles this with respect to copying files between stages. There is no expectation of merging this, it was a lot of fun to work on / figure out and I learned a lot!

Signed-off-by: vsoch [email protected]

This fixes or addresses the following GitHub issues:

• Fixes Build From Dockerfile (native mode non-OCI-SIF) #84

[dtrudg]

Hi @vsoch - as there's not an expectation of merging this (at least yet) - I'll mark this as a draft for now.

Code looks really interesting 👍 . I'd say it's in roughly the right place for this kind of approach, and it's a great starting point for further consideration of #84.

With respect to what's supported, COPY and ADD aren't quite the same. ADD can do files (like COPY), but also two things that Singularity's %file section won't:

• Download from a source URL into a destination.
• Extract a source tar file into a destination.

CMD / ENTRYPOINT is also a whole can of worms... but we do have:

singularity/internal/pkg/build/sources/conveyorPacker_oci.go

Line 48 in fd7a3f2

const ociRunscript = `

... which handles that in OCI image -> SIF translations.

Anyway, that's just a quick look. Cheers.

[vsoch]

With respect to what's supported, COPY and ADD aren't quite the same. ADD can do files (like COPY), but also two things that Singularity's %file section won't:

Yep this I'm aware (I'm not a container noob @dtrudg!) I have a Singularity <-> Docker converter that I made years ago and for the time being, we treat COPY and ADD as the same, assuming that users don't know this distinction and use them to add files to the container (not with the special listing of actions). If someone goes outside that use case I consider it not supported as Singularity doesn't support it.

[dtrudg]

With respect to what's supported, COPY and ADD aren't quite the same. ADD can do files (like COPY), but also two things that Singularity's %file section won't:

Yep this I'm aware (I'm not a container noob @dtrudg!)

Yup, indeed, I know. I was not even a container noob when you were writing Singularity docker support things @vsoch 😄

Please bear with me ... I'm going to be writing things out in detail here, even if they might be known behaviours / limitations, so that we can gather thoughts on how the limitations could/would be accepted/presented. Also, so others who might not be aware of some of these issues can chime in around the topic of compatibility.

I have a Singularity <-> Docker converter that I made years ago and for the time being, we treat COPY and ADD as the same, assuming that users don't know this distinction and use them to add files to the container (not with the special listing of actions). If someone goes outside that use case I consider it not supported as Singularity doesn't support it.

Right.. I just think we'd need to be very careful around things like the implicit tarball extraction. If someone writes or obtains a Dockerfile relying on it then we could end up copying the tarball into the rootfs verbatim (rather than extracting it), and the build failing further down when some of the extracted content is expected to be used. That would be confusing for people, who'd have to know that certain ADD isn't supported, to understand what had happened.

It might seem minor, and generally your assumption probably does hold, but there's a lot of stuff when you run a Docker container in Singularity that you need to be aware already of e.g. the $HOME bringing in python modules that mess up behavior.... so my preference is to really explicitly list, and consider limitations when any new OCI/Docker related handling is added.

[vsoch]

Gotcha, works for me!