Feedback needed for automatic overlay partition creation

[iammattcoleman]

I made a Dracut module that creates an overlay partition for dmsquash-live and would like to contribute it to dracut.

It creates the overlay partition if the rd.live.overlay kernel parameter references a partition LABEL. The user can optionally specify which filesystem to use for the overlay partition. Currently, ext4, xfs, and btrfs are supported.

A Kiwi image description that builds an ISO image containing the module is available here:
https://github.com/iammattcoleman/overlay-test

The dracut module is located here:
https://github.com/iammattcoleman/overlay-test/tree/main/image-description/root/usr/lib/dracut/modules.d/85create-overlay

This is my first time writing a dracut module, so please go over it with a fine-toothed comb.

In addition to an initial code review, I'm curious what you think about these remaining concerns that I've got:

• When booting on read-only media (E.G. the ISO image attached to a VM's virtual CD-ROM drive) with the rd.live.overlay parameter set, the boot process halts with an error about not being able to locate the persistence partition.
  
  I anticipate that this would be a common scenario. For instance, distributions' live/installer ISO images would want to cleanly boot on both read-only and read-write media.

  In order to facilitate that behavior, which of the following approaches would be preferable?

  1. Alter dmsquash-live to skip that warning if it's booting on read-only media.
  2. Add a kernel parameter that prevents that warning from appearing in dmsquash-live.

• This should work as-is if I just open a pull request to add my 85create-overlay module to dracut. However, it's very tightly coupled to the dmsquash-live module.

  Should this functionality be directly included in the dmsquash-live module instead of a separate module?

[Conan-Kudo]

It creates the overlay partition if the rd.live.overlay kernel parameter references a partition LABEL. The user can optionally specify which filesystem to use for the overlay partition. Currently, ext4 and xfs are supported, in order to match Kiwi's hybridpersistent_filesystem setting.

How hard would it be to add Btrfs? I could see some interesting use-cases around being able to use btrfs send | btrfs receive for replicating user data stuff...

I anticipate that this would be a common scenario. For instance, distributions' live/installer ISO images would want to cleanly boot on both read-only and read-write media.

In order to facilitate that behavior, which of the following approaches would be preferable?

1. Alter dmsquash-live to skip that warning if it's booting on read-only media.
2. Add a kernel parameter that prevents that warning from appearing in dmsquash-live.

From the Linux distribution perspective, I would prefer things to work automatically. So I would go with option 1 rather than option 2. The problem with option 2 is that it makes the warning not show up at all with ISOs and that would make it difficult for when an ISO is burned to a USB stick unless we modified the image as part of flashing it (which people generally don't do anymore).

This should work as-is if I just open a pull request to add my 85create-overlay module to dracut. However, it's very tightly coupled to the dmsquash-live module.

Should this functionality be directly included in the dmsquash-live module instead of a separate module?

The tight coupling is pretty convincing to me that we should just merge it into the main module.

[cmurf]

I've jerry rigged a Fedora ISO to use a Btrfs root, and the existing rd.live.overlay.overlayfs worked without modifications.

[iammattcoleman]

It was very easy to add btrfs support, so I've gone ahead and done that. This does add about half a megabyte to the initramfs image size, due to the inclusion of mkfs.btrfs. I'm not sure that's a concern in 2022, though. Also, it'll only be included if btrfs-tools is installed on your system.

[Conan-Kudo]

That's awesome! Thanks!

[iammattcoleman]

I've discovered a potential edge case bug: the overlay partition is created based on percentages to allow parted to handle optimal alignment. As it is currently, if you created an image that consumed 99.5% of the disk you write it to, it would attempt to create the partition starting at 100% and ending at 100%. That obviously wouldn't work.

A simple fix is to check if the partition start is 100% before attempting to create the partition and cleanly exit, similarly to how it already behaves if there's absolutely no free space on the device. I've just pushed this change to my overlay-test project.

However, should it go one step further and enforce a certain minimum amount of free space?

That could prevent confusion on the user's part. For example, if you wrote a 3.9G ISO to a 4G USB stick, it'd only have 100MB of space for overlay contents, which could lead to system instability pretty quickly.

Should the initramfs really make that type of decision, though? It's unlikely that we could come up with a truly universal minimum free space percentage. It seems like individual projects that use this module should just document the minimum supported USB disk size in their system requirements.

[Conan-Kudo]

I would probably say yeah, it should. If there's not enough useful space, the fallback case is to go with the current dmsquash-live behavior of running in RAM, which is fine.

[iammattcoleman]

I've been taking a look at merging my create-overlay module into dracut's dmsquash-live module, today. I was leaning towards that approach due to the tight coupling between the two modules. However, I'm second-guessing that now because of the number of extra binaries required to create the overlay.

Currently, dmsquash-live only installs a few binaries into the initramfs:

inst_multiple umount dmsetup blkid dd losetup blockdev find rmdir

The dependencies for create-overlay are much more extensive:

inst_multiple awk basename blkid cat grep mkdir mkfs.btrfs mkfs.ext4 mkfs.xfs mount parted readlink rmdir tr umount

Should I keep create-overlay as a separate module in order to reduce initramfs sizes for users of dmsquash-live who may not want overlay creation functionality?

[Conan-Kudo]

Wouldn't it make sense to leverage a config knob to enable/disable the extra dependencies instead? But if you feel like you want to do it differently, then dmsquash-live-autooverlay would probably make sense.

[iammattcoleman]

I've addressed all the concerns brought up in this discussion and opened a pull request:
#1991

[iammattcoleman]

The pull request has been merged and the new module is available in the master branch:
a3c67d2

[Conan-Kudo]

In absence of a better way to deal with this, I'm going to lock this discussion....