infrastructure kubevirt version is [0.59.0], hypershift kubevirt platform requires kubevirt version [1.0.0] or greater, infrastructure Kubernetes version is [1.26.4+52589e6ce268bd-dirty], hypershift kubevirt platform requires Kubernetes version [1.27.0] or greater #1769

joaoacpjr · 2023-10-27T12:11:52Z

joaoacpjr
Oct 27, 2023

Command used to create kubevirt cluster:

hypershift create cluster kubevirt \
--name example \
--memory 6Gi \
--cores 2 \
--containerdisk=quay.io/fedora/fedora-coreos-kubevirt:stable \
--release-image=quay.io/openshift/okd:4.13.0-0.okd-2023-06-24-145750 \
--root-volume-size 32 \
--node-pool-replicas=3 \
--root-volume-storage-class ssd \
--base-domain apps.okd-dev.mycompany.services \
--pull-secret Downloads/pull_secret.json

Output in UI:

Controle plane status:

...
ValidConfiguration  [infrastructure kubevirt version is [0.59.0], hypershift kubevirt platform requires kubevirt version [1.0.0] or greater, infrastructure Kubernetes version is [1.26.4+52589e6ce268bd-dirty], hypershift kubevirt platform requires Kubernetes version [1.27.0] or greater]

...
ValidReleaseImage
the minimum version supported for platform KubeVirt is: "4.14.0". Attempting to use: "4.13.0-0.okd-2023-06-24-145750"

Note: Even if I use another image in hypershift command, like openshift "quay.io/openshift-release-dev/ocp-release:4.14.0-x86_64" the ValidConfiguration get the same error, despit the ValidReleaseImage does not more appears.

hypershift cli version
openshift/hypershift: 2c52769df12895ea389c8e87c6175dc59aedd4d6. Latest supported OCP: 4.13.0

Operators

kubevirt-hyperconverged-operator v1.9.0
stolostron-engine v0.2.2

The OKD cluster was installed using the following cli version:

/openshift-install version
./openshift-install 4.13.0-0.okd-2023-09-30-084937
built from commit 7cbe79cbc9a8704552b95c96721b7c73e0d5e552
release image quay.io/openshift/okd@sha256:fc0dd43b87942f40d7eec16782ca2965199ced131906405127837884b5342b92
release architecture amd64

oc get nodes -o wide
NAME                           STATUS   ROLES                  AGE   VERSION           INTERNAL-IP   EXTERNAL-IP   OS-IMAGE                        KERNEL-VERSION           CONTAINER-RUNTIME
okd-dev-g94gd-master-0         Ready    control-plane,master   20h   v1.26.9+52589e6   10.0.3.6      <none>        Fedora CoreOS 38.20230902.3.0   6.4.15-200.fc38.x86_64   cri-o://1.26.4
okd-dev-g94gd-master-1         Ready    control-plane,master   20h   v1.26.9+52589e6   10.0.1.202    <none>        Fedora CoreOS 38.20230902.3.0   6.4.15-200.fc38.x86_64   cri-o://1.26.4
okd-dev-g94gd-master-2         Ready    control-plane,master   20h   v1.26.9+52589e6   10.0.3.35     <none>        Fedora CoreOS 38.20230902.3.0   6.4.15-200.fc38.x86_64   cri-o://1.26.4
okd-dev-g94gd-worker-0-fqjm6   Ready    worker                 19h   v1.26.9+52589e6   10.0.1.90     <none>        Fedora CoreOS 38.20230902.3.0   6.4.15-200.fc38.x86_64   cri-o://1.26.4
okd-dev-g94gd-worker-0-j574c   Ready    worker                 19h   v1.26.9+52589e6   10.0.2.35     <none>        Fedora CoreOS 38.20230902.3.0   6.4.15-200.fc38.x86_64   cri-o://1.26.4
okd-dev-g94gd-worker-0-t5swp   Ready    worker                 19h   v1.26.9+52589e6   10.0.1.3      <none>        Fedora CoreOS 38.20230902.3.0   6.4.15-200.fc38.x86_64   cri-o://1.26.4

hostedcluster status:

oc get hostedcluster  -A
NAMESPACE   NAME      VERSION   KUBECONFIG   PROGRESS   AVAILABLE   PROGRESSING   MESSAGE
clusters    example                          Partial    False       False         Waiting for hosted control plane to be healthy

oc describe hostedcluster example -n clusters
Name:         example
Namespace:    clusters
Labels:       <none>
Annotations:  <none>
API Version:  hypershift.openshift.io/v1beta1
Kind:         HostedCluster
Metadata:
  Creation Timestamp:  2023-10-27T11:38:38Z
  Finalizers:
    hypershift.openshift.io/finalizer
  Generation:  3
  Managed Fields:
    API Version:  hypershift.openshift.io/v1beta1
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .:
          f:kubectl.kubernetes.io/last-applied-configuration:
      f:spec:
        .:
        f:autoscaling:
        f:controllerAvailabilityPolicy:
        f:dns:
          .:
          f:baseDomain:
        f:etcd:
          .:
          f:managed:
            .:
            f:storage:
              .:
              f:persistentVolume:
                .:
                f:size:
              f:type:
          f:managementType:
        f:fips:
        f:infrastructureAvailabilityPolicy:
        f:issuerURL:
        f:networking:
          .:
          f:clusterNetwork:
          f:networkType:
          f:serviceNetwork:
        f:olmCatalogPlacement:
        f:platform:
          .:
          f:kubevirt:
          f:type:
        f:pullSecret:
        f:release:
          .:
          f:image:
        f:secretEncryption:
          .:
          f:aescbc:
            .:
            f:activeKey:
          f:type:
        f:services:
        f:sshKey:
    Manager:      kubectl-client-side-apply
    Operation:    Update
    Time:         2023-10-27T11:38:38Z
    API Version:  hypershift.openshift.io/v1beta1
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:finalizers:
          .:
          v:"hypershift.openshift.io/finalizer":
      f:spec:
        f:clusterID:
        f:infraID:
        f:platform:
          f:kubevirt:
            f:generateID:
    Manager:      hypershift-operator-manager
    Operation:    Update
    Time:         2023-10-27T11:38:39Z
    API Version:  hypershift.openshift.io/v1beta1
    Fields Type:  FieldsV1
    fieldsV1:
      f:status:
        .:
        f:conditions:
        f:controlPlaneEndpoint:
          .:
          f:host:
          f:port:
        f:version:
          .:
          f:availableUpdates:
          f:desired:
            .:
            f:image:
            f:version:
          f:history:
          f:observedGeneration:
    Manager:         hypershift-operator-manager
    Operation:       Update
    Subresource:     status
    Time:            2023-10-27T11:38:39Z
  Resource Version:  569341
  UID:               c2ec9dfd-ad31-45fd-a973-c08c836e4e45
Spec:
  Autoscaling:
  Cluster ID:                      3a448749-b0a3-4de5-9924-2cf5e68796b2
  Controller Availability Policy:  SingleReplica
  Dns:
    Base Domain:  apps.okd-dev.mycompnay.services
  Etcd:
    Managed:
      Storage:
        Persistent Volume:
          Size:                        8Gi
        Type:                          PersistentVolume
    Management Type:                   Managed
  Fips:                                false
  Infra ID:                            example-qfvd2
  Infrastructure Availability Policy:  SingleReplica
  Issuer URL:                          https://kubernetes.default.svc
  Networking:
    Cluster Network:
      Cidr:        10.132.0.0/14
    Network Type:  OVNKubernetes
    Service Network:
      Cidr:               172.31.0.0/16
  Olm Catalog Placement:  management
  Platform:
    Kubevirt:
      Generate ID:  pcvnvx4ks2
    Type:           KubeVirt
  Pull Secret:
    Name:  example-pull-secret
  Release:
    Image:  quay.io/openshift/okd:4.13.0-0.okd-2023-06-24-145750
  Secret Encryption:
    Aescbc:
      Active Key:
        Name:  example-etcd-encryption-key
    Type:      aescbc
  Services:
    Service:  APIServer
    Service Publishing Strategy:
      Type:   LoadBalancer
    Service:  OAuthServer
    Service Publishing Strategy:
      Type:   Route
    Service:  Konnectivity
    Service Publishing Strategy:
      Type:   Route
    Service:  Ignition
    Service Publishing Strategy:
      Type:   Route
    Service:  OVNSbDb
    Service Publishing Strategy:
      Type:  Route
  Ssh Key:
Status:
  Conditions:
    Last Transition Time:  2023-10-27T11:38:39Z
    Message:               Reconciliation completed succesfully
    Observed Generation:   3
    Reason:                ReconciliatonSucceeded
    Status:                True
    Type:                  ReconciliationSucceeded
    Last Transition Time:  2023-10-27T11:38:39Z
    Message:               failed to get clusterVersion: HostedControlPlane.hypershift.openshift.io "example" not found
    Observed Generation:   3
    Reason:                StatusUnknown
    Status:                Unknown
    Type:                  ClusterVersionSucceeding
    Last Transition Time:  2023-10-27T11:38:39Z
    Message:               failed to get clusterVersion: HostedControlPlane.hypershift.openshift.io "example" not found
    Observed Generation:   3
    Reason:                StatusUnknown
    Status:                Unknown
    Type:                  ClusterVersionProgressing
    Last Transition Time:  2023-10-27T11:38:39Z
    Message:               failed to get clusterVersion: HostedControlPlane.hypershift.openshift.io "example" not found
    Observed Generation:   3
    Reason:                StatusUnknown
    Status:                Unknown
    Type:                  ClusterVersionReleaseAccepted
    Last Transition Time:  2023-10-27T11:38:39Z
    Message:               failed to get clusterVersion: HostedControlPlane.hypershift.openshift.io "example" not found
    Observed Generation:   3
    Reason:                StatusUnknown
    Status:                Unknown
    Type:                  ClusterVersionUpgradeable
    Last Transition Time:  2023-10-27T11:38:39Z
    Message:               failed to get clusterVersion: HostedControlPlane.hypershift.openshift.io "example" not found
    Observed Generation:   3
    Reason:                StatusUnknown
    Status:                Unknown
    Type:                  ClusterVersionAvailable
    Last Transition Time:  2023-10-27T11:38:39Z
    Message:               The hosted control plane is not found
    Observed Generation:   3
    Reason:                StatusUnknown
    Status:                Unknown
    Type:                  Degraded
    Last Transition Time:  2023-10-27T11:38:39Z
    Message:               The hosted control plane is not found
    Observed Generation:   3
    Reason:                StatusUnknown
    Status:                Unknown
    Type:                  EtcdAvailable
    Last Transition Time:  2023-10-27T11:38:39Z
    Message:               The hosted control plane is not found
    Observed Generation:   3
    Reason:                StatusUnknown
    Status:                Unknown
    Type:                  KubeAPIServerAvailable
    Last Transition Time:  2023-10-27T11:38:39Z
    Message:               The hosted control plane is not found
    Observed Generation:   3
    Reason:                StatusUnknown
    Status:                Unknown
    Type:                  InfrastructureReady
    Last Transition Time:  2023-10-27T11:38:39Z
    Message:               The hosted control plane is not found
    Observed Generation:   3
    Reason:                StatusUnknown
    Status:                Unknown
    Type:                  ExternalDNSReachable
    Last Transition Time:  2023-10-27T11:38:39Z
    Message:               The hosted control plane is not found
    Observed Generation:   3
    Reason:                StatusUnknown
    Status:                Unknown
    Type:                  ValidHostedControlPlaneConfiguration
    Last Transition Time:  2023-10-27T11:38:39Z
    Message:               The hosted control plane is not found
    Observed Generation:   3
    Reason:                StatusUnknown
    Status:                Unknown
    Type:                  ValidReleaseInfo
    Last Transition Time:  2023-10-27T11:38:39Z
    Message:               Waiting for hosted control plane to be healthy
    Observed Generation:   3
    Reason:                WaitingForAvailable
    Status:                False
    Type:                  Available
    Last Transition Time:  2023-10-27T11:38:39Z
    Message:               [infrastructure kubevirt version is [0.59.0], hypershift kubevirt platform requires kubevirt version [1.0.0] or greater, infrastructure Kubernetes version is [1.26.4+52589e6ce268bd-dirty], hypershift kubevirt platform requires Kubernetes version [1.27.0] or greater]
    Observed Generation:   3
    Reason:                InvalidConfiguration
    Status:                False
    Type:                  ValidConfiguration
    Last Transition Time:  2023-10-27T11:38:39Z
    Message:               HostedCluster is supported by operator configuration
    Observed Generation:   3
    Reason:                AsExpected
    Status:                True
    Type:                  SupportedHostedCluster
    Last Transition Time:  2023-10-27T11:38:39Z
    Message:               Ignition server deployment not found
    Observed Generation:   3
    Reason:                NotFound
    Status:                False
    Type:                  IgnitionEndpointAvailable
    Last Transition Time:  2023-10-27T11:38:39Z
    Message:               Reconciliation active on resource
    Observed Generation:   3
    Reason:                AsExpected
    Status:                True
    Type:                  ReconciliationActive
    Last Transition Time:  2023-10-27T11:38:39Z
    Message:               the minimum version supported for platform KubeVirt is: "4.14.0". Attempting to use: "4.13.0-0.okd-2023-06-24-145750"
    Observed Generation:   3
    Reason:                InvalidImage
    Status:                False
    Type:                  ValidReleaseImage
    Last Transition Time:  2023-10-27T11:38:39Z
    Message:               ValidConfiguration condition is false
    Observed Generation:   3
    Reason:                Blocked
    Status:                False
    Type:                  Progressing
  Control Plane Endpoint:
    Host:
    Port:  0
  Version:
    Available Updates:  <nil>
    Desired:
      Image:    quay.io/openshift/okd:4.13.0-0.okd-2023-06-24-145750
      Version:
    History:
      Completion Time:    <nil>
      Image:              quay.io/openshift/okd:4.13.0-0.okd-2023-06-24-145750
      Started Time:       2023-10-27T11:38:39Z
      State:              Partial
      Verified:           false
      Version:
    Observed Generation:  2
Events:                   <none>

MCE status

oc get mce -o wide
NAME     STATUS      AGE
engine   Available   4h46m

oc describe mce engine
Name:         engine
Namespace:
Labels:       <none>
Annotations:  <none>
API Version:  multicluster.openshift.io/v1
Kind:         MultiClusterEngine
Metadata:
  Creation Timestamp:  2023-10-27T06:57:23Z
  Finalizers:
    finalizer.multicluster.openshift.io
  Generation:  6
  Managed Fields:
    API Version:  multicluster.openshift.io/v1
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:finalizers:
          .:
          v:"finalizer.multicluster.openshift.io":
      f:spec:
        f:availabilityConfig:
        f:overrides:
        f:targetNamespace:
    Manager:      Go-http-client
    Operation:    Update
    Time:         2023-10-27T06:57:24Z
    API Version:  multicluster.openshift.io/v1
    Fields Type:  FieldsV1
    fieldsV1:
      f:spec:
        .:
        f:overrides:
          f:components:
    Manager:      Mozilla
    Operation:    Update
    Time:         2023-10-27T07:16:32Z
    API Version:  multicluster.openshift.io/v1
    Fields Type:  FieldsV1
    fieldsV1:
      f:status:
        .:
        f:components:
        f:conditions:
        f:currentVersion:
        f:desiredVersion:
        f:phase:
    Manager:         Go-http-client
    Operation:       Update
    Subresource:     status
    Time:            2023-10-27T11:30:15Z
  Resource Version:  561432
  UID:               0ce785c3-f060-4853-835f-418114ad1492
Spec:
  Availability Config:  High
  Overrides:
    Components:
      Enabled:       true
      Name:          assisted-service
      Enabled:       true
      Name:          cluster-lifecycle
      Enabled:       true
      Name:          cluster-manager
      Enabled:       true
      Name:          discovery
      Enabled:       true
      Name:          hive
      Enabled:       true
      Name:          server-foundation
      Enabled:       true
      Name:          cluster-proxy-addon
      Enabled:       true
      Name:          local-cluster
      Enabled:       true
      Name:          hypershift-local-hosting
      Enabled:       true
      Name:          hypershift
      Enabled:       false
      Name:          managedserviceaccount
      Enabled:       true
      Name:          console-mce
  Target Namespace:  multicluster-engine
Status:
  Components:
    Kind:                  Component
    Last Transition Time:  2023-10-27T11:30:15Z
    Message:               No resources present
    Name:                  managedservice
    Reason:                ComponentDisabled
    Status:                True
    Type:                  NotPresent
    Kind:                  Deployment
    Last Transition Time:  2023-10-27T07:11:22Z
    Name:                  hypershift-addon-manager
    Reason:                MinimumReplicasAvailable
    Status:                True
    Type:                  Available
    Kind:                  ClusterManagementAddOn
    Name:                  hypershift-addon
    Reason:                ComponentsDeployed
    Status:                True
    Type:                  Present
    Kind:                  ManagedClusterAddOn
    Last Transition Time:  2023-10-27T11:30:15Z
    Message:               hypershift-addon add-on is available.
    Name:                  hypershift-addon
    Reason:                ManagedClusterAddOnLeaseUpdated
    Status:                True
    Type:                  Available
    Kind:                  Deployment
    Last Transition Time:  2023-10-27T07:11:43Z
    Name:                  console-mce-console
    Reason:                MinimumReplicasAvailable
    Status:                True
    Type:                  Available
    Kind:                  Deployment
    Last Transition Time:  2023-10-27T07:11:36Z
    Name:                  discovery-operator
    Reason:                MinimumReplicasAvailable
    Status:                True
    Type:                  Available
    Kind:                  Deployment
    Last Transition Time:  2023-10-27T07:11:43Z
    Name:                  hive-operator
    Reason:                MinimumReplicasAvailable
    Status:                True
    Type:                  Available
    Kind:                  Deployment
    Last Transition Time:  2023-10-27T07:12:06Z
    Name:                  infrastructure-operator
    Reason:                MinimumReplicasAvailable
    Status:                True
    Type:                  Available
    Kind:                  Deployment
    Last Transition Time:  2023-10-27T07:11:50Z
    Name:                  cluster-curator-controller
    Reason:                MinimumReplicasAvailable
    Status:                True
    Type:                  Available
    Kind:                  Deployment
    Last Transition Time:  2023-10-27T07:11:51Z
    Name:                  clusterclaims-controller
    Reason:                MinimumReplicasAvailable
    Status:                True
    Type:                  Available
    Kind:                  Deployment
    Last Transition Time:  2023-10-27T07:11:43Z
    Name:                  provider-credential-controller
    Reason:                MinimumReplicasAvailable
    Status:                True
    Type:                  Available
    Kind:                  Deployment
    Last Transition Time:  2023-10-27T07:12:01Z
    Name:                  clusterlifecycle-state-metrics-v2
    Reason:                MinimumReplicasAvailable
    Status:                True
    Type:                  Available
    Kind:                  Deployment
    Last Transition Time:  2023-10-27T07:11:45Z
    Name:                  cluster-image-set-controller
    Reason:                MinimumReplicasAvailable
    Status:                True
    Type:                  Available
    Kind:                  Deployment
    Last Transition Time:  2023-10-27T07:11:44Z
    Name:                  cluster-manager
    Reason:                MinimumReplicasAvailable
    Status:                True
    Type:                  Available
    Kind:                  ClusterManager
    Last Transition Time:  2023-10-27T11:30:15Z
    Message:               Components of cluster manager are applied
    Name:                  cluster-manager
    Reason:                ClusterManagerApplied
    Status:                True
    Type:                  Applied
    Kind:                  Deployment
    Last Transition Time:  2023-10-27T07:11:42Z
    Name:                  ocm-controller
    Reason:                MinimumReplicasAvailable
    Status:                True
    Type:                  Available
    Kind:                  Deployment
    Last Transition Time:  2023-10-27T07:12:02Z
    Name:                  ocm-proxyserver
    Reason:                MinimumReplicasAvailable
    Status:                True
    Type:                  Available
    Kind:                  Deployment
    Last Transition Time:  2023-10-27T07:12:02Z
    Name:                  ocm-webhook
    Reason:                MinimumReplicasAvailable
    Status:                True
    Type:                  Available
    Kind:                  Deployment
    Last Transition Time:  2023-10-27T07:17:55Z
    Name:                  cluster-proxy-addon-manager
    Reason:                MinimumReplicasAvailable
    Status:                True
    Type:                  Available
    Kind:                  Deployment
    Last Transition Time:  2023-10-27T07:19:34Z
    Name:                  cluster-proxy-addon-user
    Reason:                MinimumReplicasAvailable
    Status:                True
    Type:                  Available
    Kind:                  ClusterManagementAddOn
    Name:                  cluster-proxy
    Reason:                ComponentsDeployed
    Status:                True
    Type:                  Present
    Kind:                  local-cluster
    Last Transition Time:  2023-10-27T11:30:15Z
    Message:               ManagedCluster is accepted, joined, and available
    Name:                  local-cluster
    Reason:                ManagedClusterImported
    Status:                True
    Type:                  ManagedClusterImportSuccess
    Kind:                  Component
    Last Transition Time:  2023-10-27T11:30:15Z
    Message:               No resources present
    Name:                  hypershift-removals
    Reason:                ComponentDisabled
    Status:                True
    Type:                  NotPresent
  Conditions:
    Last Transition Time:  2023-10-27T07:19:49Z
    Last Update Time:      2023-10-27T07:19:49Z
    Reason:                ComponentsAvailable
    Status:                True
    Type:                  Available
    Last Transition Time:  2023-10-27T06:57:24Z
    Last Update Time:      2023-10-27T11:30:15Z
    Message:               All components deployed
    Reason:                ComponentsDeployed
    Status:                True
    Type:                  Progressing
  Current Version:         0.2.2
  Desired Version:         0.2.2
  Phase:                   Available
Events:                    <none>

vrutkovs · 2023-10-27T12:28:16Z

vrutkovs
Oct 27, 2023
Maintainer

That looks like a bug for hypershift, I can't think of what needs to be fixed in OKD to resolve that

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

infrastructure kubevirt version is [0.59.0], hypershift kubevirt platform requires kubevirt version [1.0.0] or greater, infrastructure Kubernetes version is [1.26.4+52589e6ce268bd-dirty], hypershift kubevirt platform requires Kubernetes version [1.27.0] or greater #1769

{{title}}

Replies: 1 comment

{{title}}

Select a reply

infrastructure kubevirt version is [0.59.0], hypershift kubevirt platform requires kubevirt version [1.0.0] or greater, infrastructure Kubernetes version is [1.26.4+52589e6ce268bd-dirty], hypershift kubevirt platform requires Kubernetes version [1.27.0] or greater #1769

joaoacpjr Oct 27, 2023

Replies: 1 comment

vrutkovs Oct 27, 2023 Maintainer

joaoacpjr
Oct 27, 2023

vrutkovs
Oct 27, 2023
Maintainer