Websockets Do Not Work If Any Buffering Is Configured

[Evesy]

kgateway version

v2.2.0

Kubernetes Version

v1.34.5

Describe the bug

When using WebSockets, if any sort of buffering is configured (e.g. a maxRequestSize on a TrafficPolicy, or inadvertently such as in #13641), websocket connections from clients hang indefinitely.

From the Envoy docs it suggests you should ensure upgrades are excluded from any buffer filters https://www.envoyproxy.io/docs/envoy/v1.34.1/intro/arch_overview/http/upgrades.html#http-upgrades

Expected Behavior

Websocket requests should successfully work

Steps to reproduce the bug

1. Create/deploy WebSocket application
2. Ensure ListenerPolicy with spec.default.httpSettings.upgradeConfig.enabledUpgrades: ['websocket']
3. Verify websockets work
4. Add TrafficPolicy with spec.buffer.maxRequestSize targetting the same Gateway/HTTPRoute
5. Websocket connections will no longer work

Additional Environment Detail

No response

Additional Context

No response

[andy-fong]

I am familiar with this issue. For transformation, I can probably detect the websocket upgrade request and automatically skip the body buffering.

For maxRequestSize, it might make sense for the control plane to exclude request with the Connection: upgrade header. However, I am not sure if we want that by default or only when websocket upgrade is enabled or just leave that to the user to configure the matcher.

I can definitely work on the transformation part but not as familiar with the control plane making translation decision base on setting in different policies. So, might need more input from someone more familiar that area.

[andy-fong]

The transformation in traffic policy now auto-detect websocket or CONNECT upgrade and skip buffering but setting maxRequestSize would still hang websocket. Not sure what's the best solution for that scenario because maxRequestSize uses envoy Buffer filter which is not compatible with websocket. Trying to fix that in control plane for data plane issue is usually not ideal.

[andy-fong]

The transformation fix (auto-detect websocket) is released in https://github.com/kgateway-dev/kgateway/releases/tag/v2.3.0-beta.5