UPI bare metal installation not bootstrapping

[vdveldet]

I'm trying to install OKD on a UPI bare metal infrastructure.
The bootstrap node is setup on a VirtualBox using an OVA image. The Hardware is booted with a ISO and installed from the ignition files generated by the openshift-install command.

The bootstrap times out

[root@centos02 ~]# /usr/local/bin/openshift-install wait-for bootstrap-complete --log-level debug --dir work/
DEBUG OpenShift Installer 4.13.0-0.okd-2023-06-24-145750 
DEBUG Built from commit f834ef0165bb5cbf70559eb43dad7e1203cd9aff 
INFO Waiting up to 20m0s (until 11:06AM) for the Kubernetes API at https://api.ocp4.testlab.local:6443... 
DEBUG Loading Agent Config...                      
INFO API v1.25.0-2824+27e744f55d2e99-dirty up     
DEBUG Loading Install Config...                    
DEBUG   Loading SSH Key...                         
DEBUG   Loading Base Domain...                     
DEBUG     Loading Platform...                      
DEBUG   Loading Cluster Name...                    
DEBUG     Loading Base Domain...                   
DEBUG     Loading Platform...                      
DEBUG   Loading Networking...                      
DEBUG     Loading Platform...                      
DEBUG   Loading Pull Secret...                     
DEBUG   Loading Platform...                        
DEBUG Using Install Config loaded from state file  
INFO Waiting up to 30m0s (until 11:16AM) for bootstrapping to complete... 
INFO Cluster operator cloud-controller-manager CloudConfigControllerAvailable is True with AsExpected: Cloud Config Controller works as expected 
INFO Cluster operator cloud-controller-manager CloudConfigControllerDegraded is False with AsExpected: Cloud Config Controller works as expected 
INFO Cluster operator cloud-controller-manager TrustedCABundleControllerControllerAvailable is True with AsExpected: Trusted CA Bundle Controller works as expected 
INFO Cluster operator cloud-controller-manager TrustedCABundleControllerControllerDegraded is False with AsExpected: Trusted CA Bundle Controller works as expected 
ERROR Cluster operator network Degraded is True with RolloutHung: DaemonSet "/openshift-ovn-kubernetes/ovnkube-master" rollout is not making progress - pod ovnkube-master-4jkw5 is in CrashLoopBackOff State 
ERROR DaemonSet "/openshift-ovn-kubernetes/ovnkube-master" rollout is not making progress - pod ovnkube-master-8mp6f is in CrashLoopBackOff State 
ERROR DaemonSet "/openshift-ovn-kubernetes/ovnkube-master" rollout is not making progress - pod ovnkube-master-xvkjp is in CrashLoopBackOff State 
ERROR DaemonSet "/openshift-ovn-kubernetes/ovnkube-master" rollout is not making progress - last change 2023-09-18T10:17:30Z 
ERROR DaemonSet "/openshift-ovn-kubernetes/ovnkube-node" rollout is not making progress - pod ovnkube-node-6w7lf is in CrashLoopBackOff State 
ERROR DaemonSet "/openshift-ovn-kubernetes/ovnkube-node" rollout is not making progress - pod ovnkube-node-kl2mx is in CrashLoopBackOff State 
ERROR DaemonSet "/openshift-ovn-kubernetes/ovnkube-node" rollout is not making progress - pod ovnkube-node-r2jq4 is in CrashLoopBackOff State 
ERROR DaemonSet "/openshift-ovn-kubernetes/ovnkube-node" rollout is not making progress - last change 2023-09-18T10:17:01Z 
INFO Cluster operator network ManagementStateDegraded is False with :  
INFO Cluster operator network Progressing is True with Deploying: DaemonSet "/openshift-multus/network-metrics-daemon" is waiting for other operators to become ready 
INFO DaemonSet "/openshift-ovn-kubernetes/ovnkube-master" is not available (awaiting 3 nodes) 
INFO DaemonSet "/openshift-ovn-kubernetes/ovnkube-node" is not available (awaiting 3 nodes) 
INFO DaemonSet "/openshift-network-diagnostics/network-check-target" is waiting for other operators to become ready 
INFO Deployment "/openshift-multus/multus-admission-controller" is waiting for other operators to become ready 
INFO Deployment "/openshift-network-diagnostics/network-check-source" is waiting for other operators to become ready 
ERROR Cluster operator network Available is False with Startup: The network is starting up 
INFO Use the following commands to gather logs from the cluster 
INFO openshift-install gather bootstrap --help    
ERROR Bootstrap failed to complete: timed out waiting for the condition 
ERROR Failed to wait for bootstrapping to complete. This error usually happens when there is a problem with control plane hosts that prevents the control plane operators from creating the control plane. 

The install-config.yaml used.

[root@centos02 ~]# cat install-config.yaml 
apiVersion: v1
baseDomain: testlab.local
compute: 
- name: worker
  replicas: 0 
controlPlane: 
  name: master
  replicas: 3
metadata:
  name: ocp4
networking:
  clusterNetwork:
  - cidr: 192.168.1.0/24
    hostPrefix: 24
  networkType: OVNKubernetes 
  serviceNetwork: 
  - 172.30.0.0/16
platform:
  none: {} 
fips: false 
pullSecret: '<REMOVED>' 
sshKey: '<REMOVED>' 

ssh connectivity from the bastion to all the nodes is working log bundle attached
log-bundle-20230918115454.tar.gz

[vrutkovs]

OVN is not happy with cluster / host sublengths:

F0918 11:54:51.986158  132160 ovnkube.go:133] cluster subnet invalid: cannot use a host subnet length mask shorter than or equal to the cluster subnet mask. host subnet length: 24, cluster subnet length: 24

See https://docs.openshift.com/container-platform/4.13/networking/configuring-cluster-network-range.html

[vdveldet]

Correction on the mask length solved the issue, thanks