Add jwks-with-sni example

javorszky · javorszky · commit 8e60efabfff3 · 2025-07-07T16:29:07.000+01:00
diff --git a/examples/custom-resources/jwks-with-sni/README.md b/examples/custom-resources/jwks-with-sni/README.md
@@ -0,0 +1,225 @@
+# JWKS
+
+In this example we deploy a web application, configure load balancing with a VirtualServer, and apply a JWT policy.
+Instead of using a local secret to verify the client request such as in the
+[jwt](https://github.com/nginx/kubernetes-ingress/tree/main/examples/custom-resources/jwt) example, we will define an
+external Identity Provider (IdP) using the `JwksURI` field.
+
+We will be using a deployment of [KeyCloak](https://www.keycloak.org/) to work as our IdP in this example. In this
+example, KeyCloak is deployed as a single container for the purpose of exposing it with an Ingress Controller.
+
+## Prerequisites
+
+1. Follow the [installation](https://docs.nginx.com/nginx-ingress-controller/installation/installation-with-manifests/)
+   instructions to deploy the Ingress Controller.
+
+2. Save the public IP address of the Ingress Controller into `/etc/hosts` of your machine:
+
+    ```text
+    ...
+
+    XXX.YYY.ZZZ.III webapp.example.com
+    XXX.YYY.ZZZ.III keycloak.example.com
+    ```
+
+   Here `webapp.example.com` is the domain for the web application and `keycloak.example.com` is the domain for
+   Keycloak.
+
+## Step 1 - Deploy a TLS Secret
+
+Create a secret with the TLS certificate and key that will be used for TLS termination of the Keycloak application:
+
+```console
+kubectl apply -f tls-secret.yaml
+```
+
+## Step 2 - Deploy a Web Application
+
+Create the application deployment and service:
+
+```console
+kubectl apply -f webapp.yaml
+```
+
+## Step 3 - Deploy Keycloak
+
+1. Create the Keycloak deployment and service:
+
+    ```console
+    kubectl apply -f keycloak.yaml
+    ```
+
+1. Create a VirtualServer resource for Keycloak:
+
+    ```console
+    kubectl apply -f virtual-server-idp.yaml
+    ```
+
+## Step 4 - Configure Keycloak
+
+To set up Keycloak:
+
+1. To connect to Keycloak, use `https://keycloak.example.com`.
+
+2. Create a new Realm. We will use `jwks-example` for this example. This can be done by selecting the dropdown menu on
+   the left and selecting `Create Realm`
+
+3. Create a new Client called `jwks-client`. This can be done by selecting the `Client`s tab on the left and then
+   selecting `Create client`.
+   - When creating the Client, ensure both `Client authentication` and `Authorization` are enabled.
+
+4. Once the client is created, navigate to the `Credentials` tab for that client and copy the client secret.
+   - This can be saved in the `SECRET` shell variable for later:
+
+      ```console
+      export SECRET=<client secret>
+      ```
+
+5. Create a new User called `jwks-user` by selecting the Users tab on the left and then selecting Create client.
+
+6. Once the user is created, navigate to the `Credentials` tab for that user and select `Set password`. For this example
+   the password can be whatever you want.
+   - This can be saved in the `PASSWORD` shell variable for later:
+
+     ```console
+     export PASSWORD=<user password>
+     ```
+
+## Step 5 - Deploy the JWT Policy
+
+Create a policy with the name `jwt-policy` and configure the `JwksURI` field so that it only permits requests to our
+web application that contain a valid JWT. In the example policy below, replace `<your_realm>` with the realm created in
+Step 4. We used `jwks-example` as our realm name. The value of `spec.jwt.token` is set to `$http_token` in this example
+as we are sending the client token in an HTTP header.
+
+  ```yaml
+  apiVersion: k8s.nginx.org/v1
+  kind: Policy
+  metadata:
+    name: jwt-policy
+  spec:
+    jwt:
+      realm: MyProductAPI
+      token: $http_token
+      jwksURI: http://keycloak.default.svc.cluster.local:8080/realms/<your_realm>/protocol/openid-connect/certs
+      keyCache: 1h
+  ```
+
+Deploy the policy:
+
+  ```console
+  kubectl apply -f jwks.yaml
+  ```
+
+## Step 6 - Deploy a config map with a resolver
+
+If the value of `jwksURI` uses a hostname, the Ingress Controller will need to reference a resolver. This can be done by
+deploying a ConfigMap with the `resolver-addresses` data field
+
+```yaml
+kind: ConfigMap
+apiVersion: v1
+metadata:
+  name: nginx-config
+  namespace: nginx-ingress
+data:
+  resolver-addresses: <resolver-address>
+```
+
+In this example, we create a ConfigMap using Kubernetes' default DNS `kube-dns.kube-system.svc.cluster.local` for the
+resolver address. For more information on `resolver-addresses` and other related ConfigMap keys, please refer to our
+documentation [ConfigMap
+Resource](https://docs.nginx.com/nginx-ingress-controller/configuration/global-configuration/configmap-resource/#summary-of-configmap-keys)
+and our blog post [Using DNS for Service Discovery with NGINX and NGINX
+Plus](https://www.nginx.com/blog/dns-service-discovery-nginx-plus)
+
+NOTE: When setting the value of `jwksURI` in Step 5, the response will differ depending on the IDP used. In some cases
+the response will be too large for NGINX to properly handle. If this occurs you will need to configure the
+[subrequest_output_buffer_size](https://nginx.org/en/docs/http/ngx_http_core_module.html#subrequest_output_buffer_size)
+directive in the http context. This can currently be done using `http-snippets`. Please refer to our document on
+[snippets and custom
+templates](https://docs.nginx.com/nginx-ingress-controller/configuration/global-configuration/configmap-resource/#snippets-and-custom-templates)
+for details on how to configure this directive.
+
+The code block below is an example of the updated configmap which adds `subrequest_output_buffer_size` under the http
+context in the nginx.conf.
+
+NOTE: The value of `subrequest_output_buffer_size` is only an example value and should be changed to suit your
+environment.
+
+```yaml
+kind: ConfigMap
+apiVersion: v1
+metadata:
+  name: nginx-config
+  namespace: nginx-ingress
+data:
+  resolver-addresses: <resolver-address>
+  http-snippets: |
+    subrequest_output_buffer_size 64k;
+```
+
+```console
+kubectl apply -f nginx-config.yaml
+```
+
+## Step 7 - Configure Load Balancing
+
+Create a VirtualServer resource for the web application:
+
+```console
+kubectl apply -f virtual-server.yaml
+```
+
+Note that the VirtualServer references the policy `jwt-policy` created in Step 5.
+
+## Step 8 - Get the client token
+
+For the client to have permission to send requests to the web application they must send a Bearer token to the
+application. To get this token, run the following `curl` command:
+
+```console
+export TOKEN=$(curl -k -L -X POST 'https://keycloak.example.com/realms/jwks-example/protocol/openid-connect/token' \
+-H 'Content-Type: application/x-www-form-urlencoded' \
+--data-urlencode grant_type=password \
+--data-urlencode scope=openid \
+--data-urlencode client_id=jwks-client \
+--data-urlencode client_secret=$SECRET \
+--data-urlencode username=jwks-user \
+--data-urlencode password=$PASSWORD \
+| jq -r .access_token)
+```
+
+This command will save the token in the `TOKEN` shell variable.
+
+## Step 9 - Test the Configuration
+
+If you attempt to access the application without providing the bearer token, NGINX will reject your requests for that
+VirtualServer:
+
+```console
+curl -H 'Accept: application/json' webapp.example.com
+```
+
+```text
+<html>
+<head><title>401 Authorization Required</title></head>
+<body>
+<center><h1>401 Authorization Required</h1></center>
+</body>
+</html>
+```
+
+If a valid bearer token is provided, the request will succeed:
+
+```console
+curl -H 'Accept: application/json' -H "token: ${TOKEN}" webapp.example.com
+```
+
+```text
+Server address: 10.42.0.7:8080
+Server name: webapp-5c6fdbcbf9-pt9tp
+Date: 13/Dec/2022:14:50:33 +0000
+URI: /
+Request ID: f1241390ac51318afa4fcc39d2341359
+```
diff --git a/examples/custom-resources/jwks-with-sni/jwks.yaml b/examples/custom-resources/jwks-with-sni/jwks.yaml
@@ -0,0 +1,25 @@
+apiVersion: k8s.nginx.org/v1
+kind: Policy
+metadata:
+  name: jwt-policy
+spec:
+  jwt:
+    realm: jwks-example
+    token: $http_token
+    jwksURI: http://keycloak.nginx-ingress.svc.cluster.local:8080/realms/jwks-example/protocol/openid-connect/certs
+    keyCache: 1h
+    sniName: keycloak.example.com
+    sniEnabled: true
+---
+apiVersion: k8s.nginx.org/v1
+kind: Policy
+metadata:
+  name: jwt-other-policy
+spec:
+  jwt:
+    realm: jwks-other-example
+    token: $http_token
+    jwksURI: http://otherkeycloak.nginx-ingress.svc.cluster.local:8080/realms/jwks-example/protocol/openid-connect/certs
+    keyCache: 1h
+    sniName: otherkeycloak.example.com
+    sniEnabled: true
diff --git a/examples/custom-resources/jwks-with-sni/keycloak.yaml b/examples/custom-resources/jwks-with-sni/keycloak.yaml
@@ -0,0 +1,103 @@
+apiVersion: v1
+kind: Service
+metadata:
+  name: keycloak
+  labels:
+    app: keycloak
+spec:
+  ports:
+    - name: http
+      port: 8080
+      targetPort: 8080
+  selector:
+    app: keycloak
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: keycloak
+  namespace: nginx-ingress
+  labels:
+    app: keycloak
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: keycloak
+  template:
+    metadata:
+      labels:
+        app: keycloak
+    spec:
+      containers:
+        - name: keycloak
+          image: quay.io/keycloak/keycloak:20.0.1
+          args: ["start-dev"]
+          env:
+            - name: KEYCLOAK_ADMIN
+              value: "admin"
+            - name: KEYCLOAK_ADMIN_PASSWORD
+              value: "admin"
+            - name: KC_PROXY
+              value: "edge"
+          ports:
+            - name: http
+              containerPort: 8080
+            - name: https
+              containerPort: 8443
+          readinessProbe:
+            httpGet:
+              path: /realms/master
+              port: 8080
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: otherkeycloak
+  labels:
+    app: otherkeycloak
+spec:
+  ports:
+    - name: http
+      port: 8080
+      targetPort: 8080
+  selector:
+    app: otherkeycloak
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: otherkeycloak
+  namespace: nginx-ingress
+  labels:
+    app: otherkeycloak
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: otherkeycloak
+  template:
+    metadata:
+      labels:
+        app: otherkeycloak
+    spec:
+      containers:
+        - name: otherkeycloak
+          image: quay.io/keycloak/keycloak:20.0.1
+          args: ["start-dev"]
+          env:
+            - name: KEYCLOAK_ADMIN
+              value: "admin"
+            - name: KEYCLOAK_ADMIN_PASSWORD
+              value: "admin"
+            - name: KC_PROXY
+              value: "edge"
+          ports:
+            - name: http
+              containerPort: 8080
+            - name: https
+              containerPort: 8443
+          readinessProbe:
+            httpGet:
+              path: /realms/master
+              port: 8080
diff --git a/examples/custom-resources/jwks-with-sni/nginx-config.yaml b/examples/custom-resources/jwks-with-sni/nginx-config.yaml
@@ -0,0 +1,7 @@
+kind: ConfigMap
+apiVersion: v1
+metadata:
+  name: nginx-config
+  namespace: nginx-ingress
+data:
+  resolver-addresses: "kube-dns.kube-system.svc.cluster.local"
diff --git a/examples/custom-resources/jwks-with-sni/tls-secret.yaml b/examples/custom-resources/jwks-with-sni/tls-secret.yaml
@@ -0,0 +1,8 @@
+apiVersion: v1
+kind: Secret
+metadata:
+  name: tls-secret
+type: kubernetes.io/tls
+data:
+  tls.crt: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURFVENDQWZtZ0F3SUJBZ0lVS2hTQzBBcnhUblYrbjBhVnNENkFVTE5VQWhZd0RRWUpLb1pJaHZjTkFRRUwKQlFBd0dERVdNQlFHQTFVRUF3d05LaTVsZUdGdGNHeGxMbU52YlRBZUZ3MHlNVEF4TVRZd01qSXpNekZhRncwegpNVEF4TVRRd01qSXpNekZhTUJneEZqQVVCZ05WQkFNTURTb3VaWGhoYlhCc1pTNWpiMjB3Z2dFaU1BMEdDU3FHClNJYjNEUUVCQVFVQUE0SUJEd0F3Z2dFS0FvSUJBUURGeU1DSlhlSm9tMTdhcUVQc01NbTNlVzlpQzFHdlI4YW8KaDJhNmgvZWRXTUFndEtWSERmR2tPQ2V5NDBEdGtXTDN3U0NvZE1McnhPcnN2Lzhuc1VablFwQmNBekxBbzBJVgptYnhoS21WaS9EMkJpb2pBcDlqVXlsMjNma2RWMFdYM3NYV0JQekhSa3RyK0ozaW83YVcvNUl0WVBNWWFYM3dmCkZYRWFXVmQ4QmJDQ0hyVlZ3ckMvem9aTEF3dFE0d1I5NUI2NHdtd2d4TEhNZDlWZDRSZ1l2U0ppc1QzWi9IRkkKTGpaTGdMa0FlMGlDci9xdmFsdnVhU3BNVmJUd1lQZ2l6YWhXSVFTYjVyd29JeUhnYXFBWnRYSEhjNSsydDVoZQpMMDc2RjgrOE84b0hpdDR6WGpsR1V4TFNjTWFPTnI2ZHI0Q256NmlXZzJNTGlJcno0VnR4QWdNQkFBR2pVekJSCk1CMEdBMVVkRGdRV0JCUTdCSGpyZHlicnpWNHIwVkRrc2k3TXFPNWRKREFmQmdOVkhTTUVHREFXZ0JRN0JIanIKZHlicnpWNHIwVkRrc2k3TXFPNWRKREFQQmdOVkhSTUJBZjhFQlRBREFRSC9NQTBHQ1NxR1NJYjNEUUVCQ3dVQQpBNElCQVFDdm5TdUY4dUFUWFl2VHVjVGhEcG9jKzI5RU1LVFp2VDBmSmJrNWZMaWQzYjhFTDQxdk5tTjRwUTUrCmJtSFh1bkhLL29aSm43bWVNTngwc0ZQMW1Pa1U5MXBqZVJLWmoxOXVNQjlvTVBreXdXRENuQ1BHYWtFUHpxOS8KWjFwcERKQ0FJc2cvME8wZ1BCMDdFSm9RcU0wdDlZc3BuMlJ4djMwUGdBZ3ZuSXduUlNzUWpvOEpxQ1VuemZJLwpPdXovNVl1UkhJRHQzY0RpdTdzWG1DTW01cFJ5eUd2WGZiWEsrSVFWOHZDRTZlZS9FTlNFcnB0NUdzeVNURjZKCk5LdDhXM1VwNkUvL2dwMkRvTXBxS0tGQkE0aG5OQXVzQVphTkNQdi9EY0xueG9xQUp4S0V5cmpxelJBeTlCRXkKRzBhSTJ5bitKWW5yVW8wMmc1OWFXalZMTzg4RwotLS0tLUVORCBDRVJUSUZJQ0FURS0tLS0tCg==
+  tls.key: LS0tLS1CRUdJTiBQUklWQVRFIEtFWS0tLS0tCk1JSUV2Z0lCQURBTkJna3Foa2lHOXcwQkFRRUZBQVNDQktnd2dnU2tBZ0VBQW9JQkFRREZ5TUNKWGVKb20xN2EKcUVQc01NbTNlVzlpQzFHdlI4YW9oMmE2aC9lZFdNQWd0S1ZIRGZHa09DZXk0MER0a1dMM3dTQ29kTUxyeE9ycwp2Lzhuc1VablFwQmNBekxBbzBJVm1ieGhLbVZpL0QyQmlvakFwOWpVeWwyM2ZrZFYwV1gzc1hXQlB6SFJrdHIrCkozaW83YVcvNUl0WVBNWWFYM3dmRlhFYVdWZDhCYkNDSHJWVndyQy96b1pMQXd0UTR3Ujk1QjY0d213Z3hMSE0KZDlWZDRSZ1l2U0ppc1QzWi9IRklMalpMZ0xrQWUwaUNyL3F2YWx2dWFTcE1WYlR3WVBnaXphaFdJUVNiNXJ3bwpJeUhnYXFBWnRYSEhjNSsydDVoZUwwNzZGOCs4TzhvSGl0NHpYamxHVXhMU2NNYU9OcjZkcjRDbno2aVdnMk1MCmlJcno0VnR4QWdNQkFBRUNnZ0VBQXhBcjR6VEFCK3k0R0Z6WXlIU3MreGwzWHlaYnVvSTdFbXNlYlM4ajU1enoKUk01bmJPVkxZOGEyM3E5a1Z3bVVaYy9vNkpMK1hkWnI2UVRFTitJbisvdHM3dS9odmxnSTh2cXhqek92NUV1Ugp6RXJQK1dQZ0dOT1ZoZnovcjlXUlpiZXE0VGlRVmZXWFRLNWgwUVAxT0RhYTdkL3JGWWQ3RGFRd1h6OFkrc080CnhqV0dNNFprOW1oWm1PbG9nZjNtYyszUFhYTWV6RFRMY2kzRWNpZVlaTkhTeXIzWkg2NU8rSkdsOFZ2bkZUWS8KQytQZi9tYmJKL282dlNWWDNWQUVIM29BY05qd1dqMkdBNUhiRk5RTnV0ckhRcnNkR0ZqUVB5aHNBYjNOV1h2bwo2M3hoS1NNbHpxSWd2WXZMbENOS0VjZmJsVjRuelJ4NVhhM0dzZjJkUFFLQmdRRDlYeEs4ekhpN2g4WjlQV2sxCktDZFlvZDFVa2ViWktYUVQvOUtNcmhrOE9abG1oV2hFK1lBY3lJRElVeFZuZ2xkR0d3RVViTFcyWEVnVStQVmEKM1ZlaUNCTlRWM3FwV3lYWXdIdG9yYm5WbGtlMGh4eE9WakhvSmpZWitmV0h6MDU0algvYkdsdWp5bVJGMWpoWApuMnhNUW5RUkV0S2FGN0R2d2FGK083dGExd0tCZ1FESDFndWRlVCsvQ3M1R3g3eEkwUnhwRUt4c0FtcUV3blBECklsaHoxZHJqbGZFaTRPZ25wK0ZOK05acGJiMHRaWmUyTTM2QXpMVENIUURmQVNJTlBDMkxzOHEvTjAyR2xzcG8KalVTd3M4cWc2N2ZjcG1UN1FVVTVMZmZuaDE3S1A5ZEdCdlRuK3Vza1MwVjRFZ2M0Ti9lS2pUQi9xcjYzYWRHUwp4dmRaYzdnNjl3S0JnRE9CQWdRUzVHL3FkN1M1cVFzL01GQmFCdTNNQXNzZUhCUjhxa1lpbGNxaVFzYU9VOVhCCmlnTlAxcTNpQmJYV3p2clhQbTd5Y2pXeHFJMXExaVUwWFQzNHVrVDB3V0J2d00vQXdOVlVpelFacWxYT0tUamIKV0tYQ0xyazFFRzRjKyt5Umh1MzQrNnZkMW1oRDFZd3FRZzkyYXJXVngrMis1eDYxazZoZmFBUmRBb0dCQU1Kcgp0QmM4VE5IQVlKb3FYenYwL3BBVm9icmZ5dVJwRHhsdFErTkd6OVFXSUduUHFPNVQvZmJQUDBPSmVjRStFeEU0CkhqNlBhdGxrUUdHMmgzdWE3YkQ2ZGluOVV4YTdoQ2VlTVpNOUNNbnhLNHVuODUwampvYW4rNFd0aFlKK0JDSmsKU0VlZUxzRzczZFdJcks5OGZBQzNodFRldVBoWElvZUx2a0N3UGpCWEFvR0JBUFBteVJJRGs5bUF5M2ZINnBtVwplRWlqYlBWbFdDd3FjalI5ZjQ0L3duVEpha0h4cVVxRk04cTVLNnJJejdPMmMvcDdmTm83andrVHc0R0hIVWcrCjQyVkpGOXRrdnRDbEhOZ3l6cXNjT3FjN0p2ZDNyYnBFbGVpNGgyTHo4Z0RDNFo4WldqWDBBKzVTaTlQd3RMaFEKN3pBZEJUMHk5WjZuNGYxMVg0UWhKSkR1Ci0tLS0tRU5EIFBSSVZBVEUgS0VZLS0tLS0K
diff --git a/examples/custom-resources/jwks-with-sni/virtual-server-idp.yaml b/examples/custom-resources/jwks-with-sni/virtual-server-idp.yaml
@@ -0,0 +1,18 @@
+apiVersion: k8s.nginx.org/v1
+kind: VirtualServer
+metadata:
+  name: keycloak
+spec:
+  host: keycloak.example.com
+  tls:
+    secret: tls-secret
+    redirect:
+      enable: true
+  upstreams:
+    - name: keycloak
+      service: keycloak
+      port: 8080
+  routes:
+    - path: /
+      action:
+        pass: keycloak
diff --git a/examples/custom-resources/jwks-with-sni/virtual-server.yaml b/examples/custom-resources/jwks-with-sni/virtual-server.yaml
diff --git a/examples/custom-resources/jwks-with-sni/webapp.yaml b/examples/custom-resources/jwks-with-sni/webapp.yaml
