Failed to start the crio service with error meaasge: " parse conmon version: No Major.Minor.Patch elements found"

[Bevisy]

What happened?

when using conmonrs as monitor, starting crio service failed.

What did you expect to happen?

start crio service successfully

How can we reproduce it (as minimally and precisely as possible)?

1. change the monitor path in crio conf

$ cat /etc/crio/
crio.conf.d/ policies/    policy.json
user@lima-crio:~/go.linux/src/github.com/user/conmon-rs$ cat /etc/crio/crio.conf.d/10-crio.conf
[crio.image]
signature_policy = "/etc/crio/policy.json"

[crio.runtime]
default_runtime = "crun"

[crio.runtime.runtimes.crun]
runtime_path = "/usr/bin/crio-crun"
runtime_root = "/run/crun"
monitor_path = "/usr/bin/conmonrs" # <-- from "/usr/bin/crio-conmon" to "/usr/bin/conmonrs"
allowed_annotations = [
    "io.containers.trace-syscall",
]

Anything else we need to know?

the failed log:

May 24 16:17:03 lima-crio crio[35209]: time="2024-05-24 16:17:03.121022574+08:00" level=info msg="Starting CRI-O, version: 1.31.0, git: a51dfb336a1d3847415dfa871e81d003e4ef79ae(clean)"
May 24 16:17:03 lima-crio crio[35209]: time="2024-05-24 16:17:03.129716397+08:00" level=info msg="Node configuration value for hugetlb cgroup is true"
May 24 16:17:03 lima-crio crio[35209]: time="2024-05-24 16:17:03.129826395+08:00" level=info msg="Node configuration value for pid cgroup is true"
May 24 16:17:03 lima-crio crio[35209]: time="2024-05-24 16:17:03.129930393+08:00" level=info msg="Node configuration value for memoryswap cgroup is true"
May 24 16:17:03 lima-crio crio[35209]: time="2024-05-24 16:17:03.129959392+08:00" level=info msg="Node configuration value for cgroup v2 is true"
May 24 16:17:03 lima-crio crio[35209]: time="2024-05-24 16:17:03.144989086+08:00" level=info msg="Node configuration value for systemd AllowedCPUs is true"
May 24 16:17:03 lima-crio crio[35209]: time="2024-05-24 16:17:03.145928067+08:00" level=info msg="[graphdriver] using prior storage driver: overlay"
May 24 16:17:03 lima-crio crio[35209]: time="2024-05-24 16:17:03.147207041+08:00" level=info msg="Using default capabilities: CAP_CHOWN, CAP_DAC_OVERRIDE, CAP_FSETID, CAP_FOWNER, CAP_SETGID, CAP_SETUID, CAP_SETPCAP, CAP_NET_BIND_SERVICE, CAP_KILL"
May 24 16:17:03 lima-crio crio[35209]: time="2024-05-24 16:17:03.156747847+08:00" level=info msg="Using runtime handler runc version 1.1.12, commit: v1.1.12-0-g51d5e946, spec: 1.0.2-dev, go: go1.20.13, libseccomp: 2.5.4"
May 24 16:17:03 lima-crio crio[35209]: time="2024-05-24 16:17:03.172501526+08:00" level=info msg="Using runtime handler crun version 1.15, commit: e6eacaf4034e84185fd8780ac9262bbf57082278, rundir: /run/crun, spec: 1.0.0, +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL"
May 24 16:17:03 lima-crio crio[35209]: time="2024-05-24 16:17:03.178804398+08:00" level=info msg="Checkpoint/restore support disabled: CRIU binary not found int $PATH"
May 24 16:17:03 lima-crio crio[35209]: time="2024-05-24 16:17:03.178864397+08:00" level=info msg="Using the internal default seccomp profile"
May 24 16:17:03 lima-crio crio[35209]: time="2024-05-24 16:17:03.178891396+08:00" level=info msg="AppArmor is disabled by the system or at CRI-O build-time"
May 24 16:17:03 lima-crio crio[35209]: time="2024-05-24 16:17:03.178924395+08:00" level=info msg="No blockio config file specified, blockio not configured"
May 24 16:17:03 lima-crio crio[35209]: time="2024-05-24 16:17:03.178956395+08:00" level=info msg="RDT not available in the host system"
May 24 16:17:03 lima-crio crio[35209]: time="2024-05-24 16:17:03.178998394+08:00" level=info msg="Using conmon executable: /usr/bin/conmonrs"
May 24 16:17:03 lima-crio crio[35209]: time="2024-05-24 16:17:03.195129065+08:00" level=fatal msg="validating runtime config: monitor fields translation: failed to translate monitor fields for runtime crun: parse conmon version: No Major.Minor.Patch elements found"
May 24 16:17:03 lima-crio systemd[1]: crio.service: Main process exited, code=exited, status=1/FAILURE
May 24 16:17:03 lima-crio systemd[1]: crio.service: Failed with result 'exit-code'.
May 24 16:17:03 lima-crio systemd[1]: Failed to start crio.service - Container Runtime Interface for OCI (CRI-O).

CRI-O and Kubernetes version

$ crio --version
crio version 1.31.0
Version:        1.31.0
GitCommit:      a51dfb336a1d3847415dfa871e81d003e4ef79ae
GitCommitDate:  2024-05-21T07:18:21Z
GitTreeState:   clean
BuildDate:      2024-05-22T06:55:30Z
GoVersion:      go1.22.3
Compiler:       gc
Platform:       linux/amd64
Linkmode:       dynamic
BuildTags:
  containers_image_ostree_stub
  libdm_no_deferred_remove
  seccomp
  selinux
LDFlags:          unknown
SeccompEnabled:   true
AppArmorEnabled:  false
$ conmonrs --version
version: 0.6.3
tag: v0.6.3
commit: 164af97800cc3eef9e975dd05e05f7f587f66359
build: 2024-05-24 16:07:52 +08:00
target: x86_64-unknown-linux-gnu
rustc 1.78.0 (9b00956e5 2024-04-29)
cargo 1.78.0 (54d8815d0 2024-03-26)

$ kubectl version --output=json
{
  "clientVersion": {
    "major": "1",
    "minor": "30",
    "gitVersion": "v1.30.1",
    "gitCommit": "6911225c3f747e1cd9d109c305436d08b668f086",
    "gitTreeState": "clean",
    "buildDate": "2024-05-14T10:50:53Z",
    "goVersion": "go1.22.2",
    "compiler": "gc",
    "platform": "linux/amd64"
  },
  "kustomizeVersion": "v5.0.4-0.20230601165947-6ce0bf390ce3",
  "serverVersion": {
    "major": "1",
    "minor": "30",
    "gitVersion": "v1.30.0",
    "gitCommit": "7c48c2bd72b9bf5c44d21d7338cc7bea77d0ad2a",
    "gitTreeState": "clean",
    "buildDate": "2024-04-17T17:27:03Z",
    "goVersion": "go1.22.2",
    "compiler": "gc",
    "platform": "linux/amd64"
  }
}

OS version

# On Linux:
$ cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 12 (bookworm)"
NAME="Debian GNU/Linux"
VERSION_ID="12"
VERSION="12 (bookworm)"
VERSION_CODENAME=bookworm
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
$ uname -a
Linux lima-crio 6.1.0-21-cloud-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.90-1 (2024-05-03) x86_64 GNU/Linux

Additional environment details (AWS, VirtualBox, physical, etc.)

lima vm