Clarify responsibility for sequential Kubernetes version upgrades in control plane providers

[kvaps]

Hi team,

We are using the Kamaji control plane provider and attempted to upgrade a cluster from Kubernetes version v1.30 directly to v1.32. The operation failed with the following error:

cannot create or update TenantControlPlane: admission webhook "vtenantcontrolplane.kb.io" denied the request: unable to upgrade to a minor version in a non-sequential mode

I’ve reported this to the Kamaji team here: clastix/cluster-api-control-plane-provider-kamaji#225

This brings up a broader question: who is responsible for ensuring that control plane upgrades are performed sequentially (e.g., v1.30 → v1.31 → v1.32), especially when a user requests a jump over intermediate versions?

There seem to be two possible approaches:

• If Cluster API is responsible for orchestrating sequential upgrades (both control plane and workers), then perhaps this should be implemented either as a core feature or at least as validation logic that prevents skipping versions.
• If each control plane provider is responsible for enforcing this behavior, then the issue should remain on the Kamaji side to handle sequential enforcement internally.

As far as I can tell, Cluster API already handles worker node rollouts in a version-sequential manner and coordinates upgrades carefully between control plane and workers. Based on that, I would assume that Cluster API should also be responsible for breaking down a requested upgrade path (e.g., v1.30 → v1.33) into sequential steps and apply them one-by-one.

Am I reasoning correctly here?

Thanks!

[k8s-ci-robot]

This issue is currently awaiting triage.

If CAPI contributors determine this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[sbueringer]

If you use ClusterClass then the responsiblity is on ClusterClass and Cluster API. Given that the ClusterClass implementation today does not allow n+2 upgrades (it will in the future: #12199), I assume you are not using it, so the responsibilty to ensure that control plane upgrades are performed sequentially is on you

In general I would expect that a control plane provider blocks upgrades that it cannot support.

[fabriziopandini]

Agree with @sbueringer
Responsibility for the orchestration of the cluster upgrade is on CAPI topology controller if using CC, otherwise orchestrating the cluster upgrade is a responsibility of who / of what combines the different CAPI primitives into a Cluster.

No matter of who/what orchestrates the upgrade, each control plane provider should prevent operations it does not support.

Let us know if more clarifications are required + feel free to comment on #12199 for the discussion about supporting chained and efficient upgrades
/close

[k8s-ci-robot]

@fabriziopandini: Closing this issue.

In response to this:

Agree with @sbueringer
Responsibility for the orchestration of the cluster upgrade is on CAPI topology controller if using CC, otherwise orchestrating the cluster upgrade is a responsibility of who / of what combines the different CAPI primitives into a Cluster.

No matter of who/what orchestrates the upgrade, each control plane provider should prevent operations it does not support.

Let us know if more clarifications are required + feel free to comment on #12199 for the discussion about supporting chained and efficient upgrades
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.