Skip to content

Conversation

@lexfrei
Copy link
Contributor

@lexfrei lexfrei commented Oct 23, 2025

Summary

Addresses #5901
Related: #4056

Proposes comprehensive solutions for Gateway API annotation placement confusion, including both short-term documentation improvements and long-term annotation inheritance strategy.

Background

Users frequently misconfigure annotations when using Gateway API sources:

  1. Issue Documentation gap: Provider-specific annotations placement for Gateway API sources #5901: Platform engineer placed cloudflare-proxied annotation on Gateway, expecting it to apply to all Routes, but it was silently ignored (annotations must be on Route resources)
  2. Issue Question: Gatway-api target on HTTPRoute #4056: User tried to override target annotation on HTTPRoute to specify different targets per Route, but target only works on Gateway

This proposal addresses both use cases.

Proposed Solutions

Solution 1: Documentation Improvements (Quick Win)

Status: Already proposed in PR #5918 for quick merge.

Adds clear documentation to annotations.md and gateway-api.md explaining:

  • Which annotations go on Gateway (only target)
  • Which annotations go on Routes (everything else: hostname, ttl, provider-specific)
  • Examples for Cloudflare and AWS providers
  • Common mistakes to avoid

Note: If Solution 2 is implemented, the documentation from PR #5918 will need updates to reflect the new inheritance behavior.

Solution 2: Annotation Inheritance and Merging (Long-term)

Implement annotation merging where:

  • Gateway annotations serve as defaults for all Routes
  • Route annotations override Gateway defaults
  • Enables centralized configuration without losing flexibility

Example:

kind: Gateway
metadata:
  annotations:
    # Default for all Routes using this Gateway
    external-dns.alpha.kubernetes.io/cloudflare-proxied: "true"
    external-dns.alpha.kubernetes.io/ttl: "300"
---
kind: HTTPRoute
metadata:
  annotations:
    # Override: disable proxying for this specific Route
    external-dns.alpha.kubernetes.io/cloudflare-proxied: "false"
    # Inherits: ttl=300 from Gateway

Alternatives Considered

  1. Do nothing - Users continue experiencing confusion ❌
  2. Move all annotations to Gateway only - Breaks Gateway API architecture ❌
  3. Support both with strict validation - Complex without solving core UX issue ⚠️
  4. Create dedicated GatewayDNSConfig CRD - Significant effort, scope too large ⚠️
  5. Wait for PR docs(proposal): annotations support graduation to beta #5080 - Timeline uncertain, doc improvements valuable now ⚠️

Recommendation

Phased approach:

  1. Immediate: Merge PR docs(gateway-api): clarify annotation placement for sources #5918 (documentation improvements)
  2. Future: Re-evaluate annotation merging after gathering user feedback and assessing compatibility with annotation standardization efforts

Request for Feedback

This proposal seeks maintainer input on:

  1. Is the current behavior (annotations split between Gateway/Route) intentional design?
  2. Would annotation inheritance/merging be acceptable, or does it conflict with future plans?
  3. Should we prioritize documentation alone, or is feature enhancement warranted?

Related PRs:

Addresses kubernetes-sigs#5901
Related: kubernetes-sigs#4056

Propose comprehensive solutions for Gateway API annotation placement confusion,
including both short-term documentation improvements and long-term annotation
inheritance strategy.

Documentation improvements are proposed separately in PR kubernetes-sigs#5918 for quick merge.

Co-Authored-By: Claude <[email protected]>
@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Oct 23, 2025
@k8s-ci-robot
Copy link
Contributor

Hi @lexfrei. Thanks for your PR.

I'm waiting for a github.com member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign raffo for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added docs size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Oct 23, 2025
@szuecs
Copy link
Contributor

szuecs commented Oct 24, 2025

@lexfrei do we need this proposal if we have #5918 ?

@lexfrei
Copy link
Contributor Author

lexfrei commented Oct 24, 2025

@szuecs Good question!

Short answer: #5918 (documentation) is a quick fix for the immediate problem. This proposal discusses whether we want to go further.

The difference:

PR #5918 (documentation hotfix):

  • Documents current behavior: target on Gateway, everything else on Routes
  • Helps users avoid confusion right now
  • Doesn't change any code behavior

This proposal (long-term discussion):

  • Solution 1 is just docs(gateway-api): clarify annotation placement for sources #5918 (documentation) - already covered ✅
  • Solution 2 (annotation merging/inheritance) is the interesting part:
    • Allows setting defaults on Gateway that apply to all Routes
    • Routes can override Gateway defaults when needed
    • Makes external-dns more flexible for users managing many Routes

Example use case for Solution 2:

# Today: must repeat annotations on EVERY Route
kind: HTTPRoute
metadata:
  annotations:
    external-dns.alpha.kubernetes.io/cloudflare-proxied: "true"
    external-dns.alpha.kubernetes.io/ttl: "300"
---
kind: HTTPRoute  
metadata:
  annotations:
    external-dns.alpha.kubernetes.io/cloudflare-proxied: "true"  # repeated
    external-dns.alpha.kubernetes.io/ttl: "300"  # repeated

# With Solution 2: set once on Gateway, apply to all Routes
kind: Gateway
metadata:
  annotations:
    external-dns.alpha.kubernetes.io/cloudflare-proxied: "true"  # default for all
    external-dns.alpha.kubernetes.io/ttl: "300"  # default for all
---
kind: HTTPRoute
metadata:
  annotations:
    external-dns.alpha.kubernetes.io/cloudflare-proxied: "false"  # override for this one

What I'm asking:

  1. Is docs(gateway-api): clarify annotation placement for sources #5918 (documentation) enough? → If yes, close this proposal
  2. Would annotation inheritance (Solution 2) be useful? → If yes, I can implement it
  3. Are there concerns with Solution 2? → Let's discuss alternatives

I'm fine with just merging #5918 and closing this proposal if you think documentation alone solves the problem. This proposal is mainly to gauge interest in the more flexible approach.

@mloiseleur
Copy link
Collaborator

Is the current behavior (annotations split between Gateway/Route) intentional design?

Yes, we try to stick and be aligned as much as possible with Gateway API design.
See #5319 (comment)
One can check PRs history; some PRs have been refused because of this.

Would annotation inheritance/merging be acceptable, or does it conflict with future plans?

At first glance, it looks good and makes sense to me. You'll need to include ListenerSet in your design.
But maybe I missed something, anything to add, @abursavich ?

Should we prioritize documentation alone, or is feature enhancement warranted?

  • We should always prioritize clear and concise documentation
  • It's Open Source, it comes with no warranty

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. docs needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants