bug: [2.0.0] Ingress controller not syncing upstream IPs on pod restart - reports success but upstreams have stale IPs

[sopacifics]

Hi team, is anyone familiar with this bug for apisix-ingress-controller 2.0.0?

• All the details here: bug: [2.0.0] Ingress controller not syncing upstream IPs on pod restart - reports success but upstreams have stale IPs apisix-ingress-controller#2699

[sopacifics]

We are having a mayor issue now, apisix-ingress-controller 1.8.x is working find with Kubernetes 1.32 but it is not working with Kubernetes 1.33

After updating Kubernetes to 1.33 apisix-ingress-controller is having issues sync endpoint when a new IP is assigned to a POD.

APISIX is still routing to the old 10.13.211.238. The controller is not syncing.

1/9/26
10:56:34.742 PM    
10.13.162.45 - - [09/Jan/2026:22:56:31 +0000] dev.cxp.cisco.com "GET /asset-tagging/api/v1/tags HTTP/1.1" 502 631 135.134 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/143.0.0.0 Safari/537.36 Edg/143.0.0.0" 10.13.211.238:8080 502 135.105 "http://dev.cxp.cisco.com/tags"
host = ip-10-13-182-96.us-west-2.compute.internal source = /var/log/pods/apisix_apisix-596f955c4d-xxrv8_cb701631-3f1e-4b3c-a8b3-af31201d8f54/apisix/0.log sourcetype = kube:container:apisix

1/9/26
10:56:31.741 PM    
2026/01/09 22:56:31 [error] 52#52: *274336 connect() failed (110: Connection timed out) while connecting to upstream, client: 10.13.162.45, server: _, request: "GET /asset-tagging/api/v1/tags HTTP/1.1", upstream: "http://10.13.211.238:8080/tags", host: "dev.cxp.cisco.com"
host = ip-10-13-182-96.us-west-2.compute.internal source = /var/log/pods/apisix_apisix-596f955c4d-xxrv8_cb701631-3f1e-4b3c-a8b3-af31201d

kubectl get pods -n cvi-app-multiservice -o wide
NAME                                 READY   STATUS    RESTARTS   AGE     IP              NODE                                          NOMINATED NODE   READINESS GATES
asset-tagging-545f459799-d8kz2       1/1     Running   0          5m17s   10.13.179.53    ip-10-13-162-45.us-west-2.compute.internal    <none>           <none>
pas-data-processor-998c84bd8-mznzg   1/1     Running   0          172m    10.13.202.183   ip-10-13-219-245.us-west-2.compute.internal   <none>           <none>

 ~ kubectl get configmap apisix-configmap -n apisix -o yaml | grep -A 2 watch_endpointslices 

      watch_endpointslices: true
      apisix_route_version: "apisix.apache.org/v2"
      enable_gateway_api: true


 ~ % kubectl logs -n apisix -l app.kubernetes.io/name=ingress-controller --tail=100 | grep -i "endpoint"

Defaulted container "ingress-controller" out of: ingress-controller, wait-apisix-admin (init)
Defaulted container "ingress-controller" out of: ingress-controller, wait-apisix-admin (init)
W0110 06:57:48.238696       1 warnings.go:70] v1 Endpoints is deprecated in v1.33+; use discovery.k8s.io/v1 EndpointSlice 

• The config says watch_endpointslices: true but the controller is still using v1 Endpoints (deprecated API).

This is a bug in ingress-controller 1.8.x - the watch_endpointslices setting doesn't actually work properly with Kubernetes 1.33.

Apparently, the real Fix: it is too Upgrade to 2.x (which we tried but is causing similar sync error)

Looks like that version 1.8.x is fundamentally incompatible with Kubernetes 1.33. The watch_endpointslices flag exists but doesn't properly switch to the EndpointSlice API.

Any recommendation?