"mgr podman" commands fail with common error

[digdilem-work]

Problem description

I'm aware of several other issues raised reporting similar issues, but I've not found a resolution in those.

Server was migrated from RPM to Podman about 3 weeks ago. Most things are fine, but I'm concerned that this will prevent us upgrading to any future versions. We have a lot of changes invested since the upgrade, so starting the migration over is very much something we don't want to do.

During the upgrade I created a new vm, named it uyuni02 and as per the guide, turned off the original and then changed the ip and hostname of the new vm to that of the original. The clients are checking in, we've worked out scheduled events and things seem good, apart from this.

I have tried "mgradm uninstall --force" and "mgadm install podman" as advised in #9023 but this did not resolve it for me.

(I've manually changes the fqdn for privacy, but the two match)

ata-oxy-uyuni01:/home/simon # mgradm inspect
2:29PM INF Welcome to mgradm
2:29PM INF Executing command: inspect
2:29PM INF Computed image name is registry.opensuse.org/uyuni/server:latest
2:29PM INF Pull Policy is always. Presence of RPM image will be checked and if it's not present it will be pulled from registry
2:29PM INF Cannot find RPM image for registry.opensuse.org/uyuni/server:latest
2:29PM INF Running podman pull registry.opensuse.org/uyuni/server:latest
Trying to pull registry.opensuse.org/uyuni/server:latest...
Getting image source signatures
Copying blob 0a5f8240e04b skipped: already exists
Copying blob 4dcdc701cb4a skipped: already exists
Copying blob 3e004aa9a8af skipped: already exists
Copying config 1b794a1eb8 done   |
Writing manifest to image destination
1b794a1eb81cf14fe337f9bf243e0cd56f81c4af9bc3e1bb259396f702a6f49e
Error: inspect command failed: cannot inspect data: cannot read config: While parsing config: line `ata-oxy-uyuni01.company.com` doesn't match format

Uyuni host vm (Leap micro 5.5)

ata-oxy-uyuni01:/home/simon # hostname -f
ata-oxy-uyuni01.company.com

In container, via "mgrctl term"

uyuni-server:/ # hostname -f
uyuni-server.mgr.internal

DNS lookup is resolved correctly inside and outside of the container

uyuni-server:/ # nslookup ata-oxy-uyuni01.company.com
Server:         10.89.0.1
Address:        10.89.0.1#53

Name:   ata-oxy-uyuni01.company.com
Address: 172.30.101.199

"mgradm upgrade podman" also fails with the same error.

Please advise. Thanks.

Steps to reproduce

As above

Uyuni version

2024-08 Podman

Uyuni proxy version (if used)

No response

Useful logs

No response

Additional information

No response

[aric89]

I'm having the same issue as well. After the uninstall and reinstall, I'm getting a blank green page for the web gui.

[aric89]

Crazy thing just happened around 4 am CST. My uyuni server updated itself but spacewalk-service won't start properly.

Oct 24 03:31:45 uyuni systemd[1]: Starting Uyuni server image container service... Oct 24 03:31:45 uyuni podman[1156]: 2024-10-24 03:31:45.572529075 -0500 CDT m=+0.168989932 system refresh Oct 24 03:31:45 uyuni podman[1171]: 2024-10-24 03:31:45.674225914 -0500 CDT m=+0.079691911 container create 1408afdf8b995f277b1ab90321b8cba07be2cdfabac1f98d86527bdf151d0174 (image=registry.opensuse.org/uyu> Oct 24 03:31:45 uyuni podman[1171]: 2024-10-24 03:31:45.628366517 -0500 CDT m=+0.033832471 image pull d78bb09cff782bfdae5724c727fac4393f5560b21cc437a26c30357cc5fb3e03 registry.opensuse.org/uyuni/server:lat> Oct 24 03:31:46 uyuni podman[1171]: 2024-10-24 03:31:46.406638808 -0500 CDT m=+0.812104765 container init 1408afdf8b995f277b1ab90321b8cba07be2cdfabac1f98d86527bdf151d0174 (image=registry.opensuse.org/uyuni> Oct 24 03:31:46 uyuni podman[1171]: 2024-10-24 03:31:46.429036515 -0500 CDT m=+0.834502504 container start 1408afdf8b995f277b1ab90321b8cba07be2cdfabac1f98d86527bdf151d0174 (image=registry.opensuse.org/uyun> Oct 24 03:31:46 uyuni sh[1171]: 1408afdf8b995f277b1ab90321b8cba07be2cdfabac1f98d86527bdf151d0174 Oct 24 03:31:46 uyuni systemd[1]: Started Uyuni server image container service. uyuni:~ #

[digdilem-work]

Comment to own, may have resolved this - requires verification before closing ticket, and ideally a fix to mgradm

This is likely caused by the file /etc/rhn/rhn.conf containing two lines with "java.hostname" in them.

I fixed this issue by:

1. Make backup of the vm
2. mgrctl term
3. vi /etc/rhn/rhn.conf
4. Verify there are two lines the same containing java.hostname = fqdn
5. DELETE one of them (commenting will not work) and save.
6. exit back to the host OS
7. Retry mgradm upgrade podman

Test and fix:
As a test of an affected system, I’ve run those commands you mention in the container and this one

cat /etc/rhn/rhn.conf 2>/dev/null | grep 'java.hostname' | cut -d' ' -f3 || true
Produces a double output
uyuni-server:/ # cat /etc/rhn/rhn.conf 2>/dev/null | grep 'java.hostname' | cut -d' ' -f3 || tru
ata-oxy-uyuni01.domain.com
ata-oxy-uyuni01.domain.com

On inspection, this file contains TWO java.hostname lines

uyuni-server:/ # grep java.hostname /etc/rhn/rhn.conf
java.hostname = ata-oxy-uyuni01.domain.com
java.hostname = ata-oxy-uyuni01.domain.com

Removing one of these lines (Commenting out doesn’t fix it, obviously since it’s grepping) allowed me to run the update “mgradm update podman”

So – it looks like the migration adds a valid java.hostname value to this file without checking whether one exists already.

I suspect this could be fixed within mgradm by restricting the output of the value by one, ie, changing that command to:

cat /etc/rhn/rhn.conf 2>/dev/null | grep 'java.hostname' | cut -d' ' -f3 | head -n 1 || true

[aric89]

That seemed to do the trick! I didn't even notice the double java.hostname in rhn.

2. is supposed to be "mgrctl term" instead.

[digdilem-work]

That seemed to do the trick!

Great stuff, glad it helped.

I didn't even notice the double java.hostname in rhn.

Not me. I suspect it existed in the original rpm install, and the migration tool added a second one, but can't verify that.

2. is supposed to be "mgrctl term" instead.

Ah yes, fixed, thanks.