Generated user-data is failing cloud-config schema validation

[afbjorklund]

Description

There are some reports in the log, about the user-data failing validation.

[WARNING]: Invalid cloud-config provided: Please run 'sudo cloud-init schema --system' to see the schema errors.

We should probably run yaml and jsonschema validation, on the user-data?

But at least fix the syntax errors, and try to conform to the cloud-init schema...

[afbjorklund]

Note that changing to "ssh_authorized_keys" will actually fail some alternative cloud-init implementations...

Such as lima-init, or tinycloudinit. They have "ssh-authorized-keys" hardcoded, in their alternate parsers (awk)

Seems to be a (python) bug in the cloud-init documentation:

• https://bugs.launchpad.net/cloud-init/+bug/1420018

[jandubois]

Such as lima-init

We can easily fix lima-init, but don't know about tinycloudinit. I did the same already for ca-certs and ca_certs: https://github.com/lima-vm/alpine-lima/blob/5a09f68c35124ba6049ec15fd972991c445856f7/lima-init.sh#L129-L130

awk -f- "${LIMA_CIDATA_MNT}"/user-data <<'EOF' > ${LIMA_CA_CERTS}
# Lima currently uses "ca-certs", which is deprecated and should be "ca_certs"
/^ca.certs:/ {
…

[afbjorklund]

The tinycloudinit was just a hack that I used when playing with tinycorelinux, it is a similar awk script (with similar fix)

https://github.com/spinto/tinycloudinit

---

EDIT: fixed on master now (by checking for both dashes and underscores...), along with some other bugs and features

afbjorklund/tinycloudinit@e44d4e0

[afbjorklund]

Checking for deprecated and changed fields actually requires both enabling "annotations" and patching jsonschema...

        "uid": {
          "description": "The user's ID. Default value [system default]",
          "oneOf": [
            {
              "type": "integer"
            },
            {
              "type": "string",
              "changed": true,
              "changed_description": "The use of ``string`` type is deprecated. Use an ``integer`` instead.",
              "changed_version": "22.3"
            }
          ]
        }

The "changed" feature seems to have been invented by cloud-init, since I can't find it on https://json-schema.org/

There is a very similar "deprecated" feature that handles removal, but doesn't cover haphazard renaming of fields.

[afbjorklund]

As mentioned in the PRs, the revision v1 (or main) of this jsonschema is actually quite a lot of versions.

And there are even more variants of cloud-config, if you count the years before the jsonschema existed...

So we should not change the type of these, just because they are deprecated (or outright missing):

users.uid, users.ssh-authorized-keys - since it breaks stuff, and some lima distributions stop working

[afbjorklund]

Related issue:

• Add templating support for cloud-init in lima.yaml #2128

[afbjorklund]

The lima generated cloud-config.yaml now passes validation with cloud-init schema,
after the ssh-authorized keys was deprecated in schema (24.2~1g370e680c-0ubuntu1)

check-jsonschema --schemafile schema-cloud-config-v1.json cloud-config.yaml
https://github.com/canonical/cloud-init/blob/370e680c/cloudinit/config/schemas/schema-cloud-config-v1.json

• Generate separate cloud-config.yaml for when not using cidata.iso #2268