Certificates rotation doesn't work on worker node when certificates are expired

[njoannic]

Hi all,

According to https://docs.k3s.io/cli/certificate#rotating-client-and-server-certificates,

"Any certificates that are expired or within 120 days to expire are automatically renewed every time K3s starts."

, certificates can be rotate even after expiration.

On my side, when I rotate certificates before expiration it works well. But when they are expired, after rotating successfully the server certificates on master node, worker node can't get new ones at restart (systemctl restart k3s-agent)!

Since the expiration of its certificates, the agent node is in "NotReady" state and can't join the cluster: Waiting to retrieve agent configuration; server is not ready: failed to retrieve configuration from server: not authorized
So it can't get new certificates provided by master node.

Am I the only one having this problem? Did I missed something?
Thanks

Notes:

• cluster: CoreOS 41.20250331.3.0, kube v1.31.1+k3s1 with one master node and one worker node
• I'm working with self-signed (by kube) certificates.
• I have not modified Certificate Authority (CA) certificates and they are valid.

[brandond]

Duplicate of

• Dynamiclistener cert cannot be renewed until after the apiserver is up #12472.

Please upgrade to a more recent release.

[njoannic]

Hi,
Thanks, I'm not sure that it is the same issue, but I will test with v1.33.1 version soon.
Perhaps it is due to our admission tokens which expire after one day.

[njoannic]

After holidays and tests, I can confirm now that it is really due to our temporary admission token.
When it never expires (default configuration of k3s), the issue doesn't happen.
The workaround is to get manually (out of the cluster, ssh) the token from /var/lib/rancher/k3s/server/node-token on the master and put it in the configuration of the agent.
Thanks

[brandond]

Hmm, that is interesting. The bootstrap tokens are just stored as Secrets, and don't contain anything that would be affected by certificate rotation.

Can you confirm that the token you were using on the agent still shows up in k3s token list output?