Error on starting Master node okd

[youssefbouzaine]

Hi,

I m actually on v4.2.0-alpha.0-620-gb66f2d3 version and i m trying to start my master nodes

after installing everything i m seeing issues on starting node

`crictl logs 49db03f5ebf67
Copying system trust bundle
I0705 18:40:20.512768       1 observer_polling.go:159] Starting file observer
W0705 18:40:20.513070       1 cmd.go:200] Using insecure, self-signed certificates
I0705 18:40:20.513243       1 crypto.go:588] Generating new CA for cluster-authentication-operator-signer@1625510420 cert, and key in /tmp/serving-cert-542487858/serving-signer.crt, /tmp/serving-cert-542487858/serving-signer.key
I0705 18:40:21.031121       1 observer_polling.go:159] Starting file observer
W0705 18:40:21.032808       1 builder.go:206] unable to get owner reference (falling back to namespace): Get https://172.30.0.1:443/api/v1/namespaces/openshift-authentication-operator/pods/authentication-operator-7dcb56f456-cv5qr: dial tcp 172.30.0.1:443: connect: no route to host
I0705 18:40:21.032927       1 builder.go:233] cluster-authentication-operator version v0.0.2-502-ga83cc70-
I0705 18:40:21.033620       1 dynamic_serving_content.go:111] Loaded a new cert/key pair for "serving-cert::/tmp/serving-cert-542487858/tls.crt::/tmp/serving-cert-542487858/tls.key"
[root@okdcontrolpanel ~]# crictl logs 49db03f5ebf67
Copying system trust bundle
I0705 18:40:20.512768       1 observer_polling.go:159] Starting file observer
W0705 18:40:20.513070       1 cmd.go:200] Using insecure, self-signed certificates
I0705 18:40:20.513243       1 crypto.go:588] Generating new CA for cluster-authentication-operator-signer@1625510420 cert, and key in /tmp/serving-cert-542487858/serving-signer.crt, /tmp/serving-cert-542487858/serving-signer.key
I0705 18:40:21.031121       1 observer_polling.go:159] Starting file observer
W0705 18:40:21.032808       1 builder.go:206] unable to get owner reference (falling back to namespace): Get https://172.30.0.1:443/api/v1/namespaces/openshift-authentication-operator/pods/authentication-operator-7dcb56f456-cv5qr: dial tcp 172.30.0.1:443: connect: no route to host
I0705 18:40:21.032927       1 builder.go:233] cluster-authentication-operator version v0.0.2-502-ga83cc70-
I0705 18:40:21.033620       1 dynamic_serving_content.go:111] Loaded a new cert/key pair for "serving-cert::/tmp/serving-cert-542487858/tls.crt::/tmp/serving-cert-542487858/tls.key"`

Logs of openshift-apiserver on master1:

https://172.30.0.1:443/api/v1/namespaces/openshift-authentication-operator/pods/authentication-operator-7dcb56f456-cv5qr: dial tcp 172.30.0.1:443: connect: no route to host

any help will be appreciated and thanks

[vrutkovs]

Please attach log bundle

[youssefbouzaine]

2021-07-05T20:41:38.334728038+00:00 stderr F I0705 20:41:38.334681 1 request.go:557] Throttling request took 1.043256933s, request: GET:https://localhost:6443/apis/operators.coreos.com/v1alpha1?timeout=32s
2021-07-05T20:41:38.359695155+00:00 stderr F I0705 20:41:38.359652 1 request.go:557] Throttling request took 1.068219442s, request: GET:https://localhost:6443/apis/samples.operator.openshift.io/v1?timeout=32s
2021-07-05T20:41:38.384780915+00:00 stderr F I0705 20:41:38.384734 1 request.go:557] Throttling request took 1.093286119s, request: GET:https://localhost:6443/apis/tuned.openshift.io/v1?timeout=32s
2021-07-05T20:41:38.409719360+00:00 stderr F I0705 20:41:38.409672 1 request.go:557] Throttling request took 1.118224407s, request: GET:https://localhost:6443/apis/metal3.io/v1alpha1?timeout=32s
2021-07-05T20:41:38.435970147+00:00 stderr F I0705 20:41:38.434660 1 request.go:557] Throttling request took 1.143200568s, request: GET:https://localhost:6443/apis/migration.k8s.io/v1alpha1?timeout=32s
2021-07-05T20:41:38.460236473+00:00 stderr F I0705 20:41:38.459738 1 request.go:557] Throttling request took 1.16824216s, request: GET:https://localhost:6443/apis/whereabouts.cni.cncf.io/v1alpha1?timeout=32s
2021-07-05T20:41:38.463073698+00:00 stderr F E0705 20:41:38.463018 1 task.go:81] error running apply for servicemonitor "openshift-cluster-machine-approver/cluster-machine-approver" (520 of 596): failed to get resource type: no matches for kind "ServiceMonitor" in version "monitoring.coreos.com/v1"
2021-07-05T20:41:38.463119050+00:00 stderr F E0705 20:41:38.463094 1 task.go:81] error running apply for servicemonitor "openshift-cluster-version/cluster-version-operator" (8 of 596): failed to get resource type: no matches for kind "ServiceMonitor" in version "monitoring.coreos.com/v1"
2021-07-05T20:41:38.463179848+00:00 stderr F E0705 20:41:38.463158 1 task.go:81] error running apply for oauthclient "console" (363 of 596): failed to get resource type: no matches for kind "OAuthClient" in version "oauth.openshift.io/v1"
2021-07-05T20:41:38.487693489+00:00 stderr F I0705 20:41:38.487634 1 sync_worker.go:666] Done syncing for project "cluster" (28 of 596)
2021-07-05T20:41:38.487693489+00:00 stderr F I0705 20:41:38.487673 1 sync_worker.go:653] Running sync for proxy "cluster" (29 of 596)

[youssefbouzaine]

i don't know if that is enough

[youssefbouzaine]

i m seeing this issues also

cat cluster-version-operator-2c9da45b15e2e1d36b896b21fa9a17b4ff0fa6120bcbdf1ffcf03f9b80c57577.log
> `E0705 20:45:28.035773       1 leaderelection.go:320] error retrieving resource lock openshift-cluster-version/version: Get https://127.0.0.1:6443/api/v1/namespaces/openshift-cluster-version/configmaps/version: dial tcp 127.0.0.1:6443: connect: connection refused
I0705 20:45:28.035813       1 leaderelection.go:247] failed to acquire lease openshift-cluster-version/version
I0705 20:45:31.856857       1 reflector.go:211] Listing and watching *v1.ClusterOperator from github.com/openshift/client-go/config/informers/externalversions/factory.go:101
E0705 20:45:31.857328       1 reflector.go:178] github.com/openshift/client-go/config/informers/externalversions/factory.go:101: Failed to list *v1.ClusterOperator: Get https://127.0.0.1:6443/apis/config.openshift.io/v1/clusteroperators?limit=500&resourceVersion=0: dial tcp 127.0.0.1:6443: connect: connection refused
I0705 20:45:39.776517       1 reflector.go:211] Listing and watching *v1.Proxy from github.com/openshift/client-go/config/informers/externalversions/factory.go:101
E0705 20:45:39.777061       1 reflector.go:178] github.com/openshift/client-go/config/informers/externalversions/factory.go:101: Failed to list *v1.Proxy: Get https://127.0.0.1:6443/apis/config.openshift.io/v1/proxies?limit=500&resourceVersion=0: dial tcp 127.0.0.1:6443: connect: connection refused
I0705 20:45:44.810924       1 reflector.go:211] Listing and watching *v1.ConfigMap from k8s.io/client-go/informers/factory.go:135
E0705 20:45:44.811878       1 reflector.go:178] k8s.io/client-go/informers/factory.go:135: Failed to list *v1.ConfigMap: Get https://127.0.0.1:6443/api/v1/namespaces/openshift-config/configmaps?limit=500&resourceVersion=0: dial tcp 127.0.0.1:6443: connect: connection refused
I0705 20:46:04.607628       1 reflector.go:211] Listing and watching *v1.ClusterVersion from github.com/openshift/client-go/config/informers/externalversions/factory.go:101
E0705 20:46:04.608472       1 reflector.go:178] github.com/openshift/client-go/config/informers/externalversions/factory.go:101: Failed to list *v1.ClusterVersion: Get https://127.0.0.1:6443/apis/config.openshift.io/v1/clusterversions?fieldSelector=metadata.name%3Dversion&limit=500&resourceVersion=0: dial tcp 127.0.0.1:6443: connect: connection refused`

[youssefbouzaine]

`cat cluster-node-tuning-operator-4f0ec590247b62581203f7d7cfeb9fb64907131781cec618edee9229a98637f0.log
I0705 20:45:05.505469       1 main.go:24] Go Version: go1.13.8
I0705 20:45:05.505636       1 main.go:25] Go OS/Arch: linux/amd64
I0705 20:45:05.505646       1 main.go:26] node-tuning Version: 4f2c0be-dirty
I0705 20:45:05.507410       1 controller.go:780] trying to become a leader
E0705 20:45:05.508329       1 leader.go:132] Failed to get Pod openshift-cluster-node-tuning-operator/cluster-node-tuning-operator-5f6978996c-znrqw
F0705 20:45:05.508364       1 controller.go:783] Get https://172.30.0.1:443/api/v1/namespaces/openshift-cluster-node-tuning-operator/pods/cluster-node-tuning-operator-5f6978996c-znrqw: dial tcp 172.30.0.1:443: connect: no route to host`
````[root@okdservices1 containers]# cat cluster-node-tuning-operator-4f0ec590247b62581203f7d7cfeb9fb64907131781cec618edee9229a98637f0.log
I0705 20:45:05.505469       1 main.go:24] Go Version: go1.13.8
I0705 20:45:05.505636       1 main.go:25] Go OS/Arch: linux/amd64
I0705 20:45:05.505646       1 main.go:26] node-tuning Version: 4f2c0be-dirty
I0705 20:45:05.507410       1 controller.go:780] trying to become a leader
E0705 20:45:05.508329       1 leader.go:132] Failed to get Pod openshift-cluster-node-tuning-operator/cluster-node-tuning-operator-5f6978996c-znrqw
F0705 20:45:05.508364       1 controller.go:783] Get https://172.30.0.1:443/api/v1/namespaces/openshift-cluster-node-tuning-operator/pods/cluster-node-tuning-operator-5f6978996c-znrqw: dial tcp 172.30.0.1:443: connect: no route to host`

[youssefbouzaine]

`cat openshift-apiserver-operator-e405c786d45344ac171f30584077536d766f1eaa93dadf0ef7973f9cb8034f31.log
W0705 20:45:44.449268       1 cmd.go:200] Using insecure, self-signed certificates
I0705 20:45:44.663573       1 observer_polling.go:159] Starting file observer
W0705 20:45:44.691507       1 builder.go:206] unable to get owner reference (falling back to namespace): Get https://172.30.0.1:443/api/v1/namespaces/openshift-apiserver-operator/pods: dial tcp 172.30.0.1:443: connect: no route to host
I0705 20:45:44.703330       1 builder.go:233] openshift-apiserver-operator version -`

[youssefbouzaine]

Any help will be appreciated thanks in advance