Need assistance with WebSocket connection for long-running game sessions in Envoy Gateway

[SergeiCherevko]

I have a Kubernetes service pointing to 5 pods that need a WebSocket connection. This connection is expected to have high timeouts as it's for a game session that can last an hour or more.

I'm trying to set this up using Envoy Gateway. Here's the configuration I'm using:

Gateway Configuration:

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: eg
  namespace: xxx
spec:
  gatewayClassName: eg
  infrastructure:
    parametersRef:
      group: gateway.envoyproxy.io
      kind: EnvoyProxy
      name: envoy-custom-config
  listeners:
    - name: https
      protocol: HTTPS
      hostname: xxx.com
      port: 443
      tls:
        mode: Terminate
        certificateRefs:
          - kind: Secret
            name: xxx-com
            namespace: xxx
    - name: http
      protocol: HTTP
      port: 80

HTTPRoute Configuration:

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: xxx-route
  namespace: xxx
spec:
  parentRefs:
    - name: eg
  hostnames:
    - xxx.com
  rules:
    - backendRefs:
        - group: ""
          kind: Service
          name: xxx-envoy
          namespace: xxx
          port: 80
          weight: 1
      matches:
        - path:
            type: PathPrefix
            value: /

Custom Envoy Proxy Config:

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
  name: envoy-custom-config
  namespace: journal
spec:
  provider:
    type: Kubernetes
    kubernetes:
      envoyDeployment:
        replicas: 10
        container:
          resources:
            requests:
              cpu: 1
              memory: 1024Mi
            limits:
              cpu: 2
              memory: 2048Mi
  bootstrap:
    type: Merge
    value: |
      static_resources:
        listeners:
            filter_chains:
              - filters:
                  - name: envoy.filters.network.http_connection_manager
                    typed_config:
                      "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
                      upgrade_configs:
                        - upgrade_type: websocket

Questions:

Do I need to use the Merge bootstrap to upgrade the connection?
How can I configure it to support very long connections (an hour or more)?

[arkodg]

@SergeiCherevko websockets are enabled by default for HTTPRoute

[SergeiCherevko]

Im trying to increase connection limit websockets because default is 1024 and im overriding envoyproxy config

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
  name: envoy-custom-config
  namespace: xxx
spec:
  provider:
    type: Kubernetes
    kubernetes:
      envoyDeployment:
        replicas: 15
        container:
          resources:
            requests:
              cpu: 1
              memory: 1024Mi
            limits:
              cpu: 2
              memory: 2048Mi
  bootstrap:
    type: Merge
    value: |
      static_resources:
        clusters:
          circuit_breakers:
            thresholds:
              - priority: HIGH
                max_connections: 100000
                max_pending_requests: 1000
                max_requests: 100000
                max_retries: 100000

But i see only 1 replica and it looks like config is not applied, may be you see any mistakes? How to override only circuit breakers and don't touch other settings because its too much

[arkodg]

you can configure the BackendTrafficPolicy to configure circuitbreakers https://gateway.envoyproxy.io/docs/tasks/traffic/circuit-breaker/

[SergeiCherevko]

Thank you very much!

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: BackendTrafficPolicy
metadata:
  name: circuit-breaker
  namespace: journal
spec:
  targetRefs:
    - group: gateway.networking.k8s.io
      kind: HTTPRoute
      name: journal-feed-route
  circuitBreaker:
    maxConnections: 1000000000
    maxPendingRequests: 1000000000
    maxParallelRequests: 1000000000
    maxParallelRetries: 1000000000