firmware.patch handles FAT resources

[jeanparpaillon]

No description provided.

[jeanparpaillon]

fwup part implemented in fwup-home/fwup#202

[fhunleth]

I think the trick here will be figuring out which files in the archive support deltas and which do not. Unfortunately, this is tricky to figure out since the information is in the meta.conf and it's figured out indirectly. I.e., you have to look at the tasks and see which ones use which resources and whether the source data variables are set. The good news is that if you can do this, if a file isn't used in the task (e.g., the upgrade tasks`), it can be removed completely from the archive.

The other consideration is that zip archive order needs to be preserved for everything in data/ as well.

My other thought is that the patching is complicated enough that I wouldn't mind maintaining it in the fwup repository. I'm not super excited about writing it in C (or shell or python), though. How about let's work on it here and I'll worry about moving it over in the future.

[jeanparpaillon]

@fhunleth I definitly agree on moving the code to a single place. However, can we split it into:

• patch creation: elixir code, used by nerves_hub_www, nerves (mix firmware.patch task)
• patch application: C code, used by fwup

If you agree with this split, I can start an elixir application (nerves_delta ?), that could be used as a dep by nerves and nerves_hub _www for consistency.

I don't really understand why order matters in data as well but I can update the code to tackle with that also.

[jeanparpaillon]

@fhunleth I think part of the complexity lies in matching patch creation with capabilities of fwup. Should we couple this discussion with nerves-hub/nerves_hub_web#556 and imagine the following architecture for OTA:

• NH pushes meta.conf to device
• for each resource, device request full data or delta based on its capability, with content-type like negotiation, for instance:
  • GET ...zImage (accept: application/raw)
  • GET ...rootfs (accept: application/raw+xdelta3)

Instead of having NervesHub behaviour depending on device's fwup version, we would just require the device to handle a protocol.

[fhunleth]

NervesHub already knows the version of fwup running on the device so it can create and send down the proper update. There's no need to create a program or library on the device that does this for NervesHub. A side benefit of this is that firmware updates that don't use the fwup "unsafe" commands can be tested offline (on NervesHub, for example) to verify that they work without needing to send a byte down to a device. I don't know whether we'll get to implementing that right away, but I really like the idea of being able to debug firmware updates without worrying about runtime behavior other than transferring the one firmware update file down (and that's unavoidable).

I added this NervesHub issue to track making different deltas based on the device's fwup version.

I had a brief conversation with @oestrich and @jjcarstens on this. It sounds like there might or might not be concerns on the NervesHub side. I'll try to keep things posted to keep you on the same page.