Replace Ingress and Envoy with Envoy-Gateway

[aicam]

Proposal: Unify Proxy Architecture with Envoy Gateway

Summary

We are proposing a migration from our current dual-proxy setup (Ingress Nginx + Envoy) to a unified Envoy Gateway architecture. This change aims to simplify our infrastructure, adopt the modern Kubernetes Gateway API, and natively support the dynamic routing requirements of our ephemeral Computing Units.

Motivation & Current Limitations

Our system currently relies on Ingress Nginx for static routing and a separate Envoy instance for dynamic routing. While Ingress Nginx is a standard solution, it presents significant architectural limitations for our specific workload, particularly regarding the Computing Units in the texera-workflow-computing-unit-pool namespace.

The Problems with the Current Stack

1. Lack of Dynamic Routing: Ingress Nginx operates on a configuration reload model. Every time an upstream service changes, Nginx must reload its configuration. This is inefficient for our Computing Units, which are highly transient (dynamically created and terminated).
2. Dual-Proxy Complexity: To bypass Nginx's limitations, we introduced a secondary Envoy proxy to handle WebSocket and HTTP connections to these dynamic units. This resulted in a "double-hop" architecture (User -> Ingress -> Envoy -> Compute), increasing latency and operational maintenance.
3. Legacy Architecture: Nginx's process-based architecture is less suited for high-churn service discovery compared to Envoy's modern, threaded architecture which utilizes the xDS protocol for seamless, hot-restart-free configuration updates.

Proposed Solution: Envoy Gateway

We propose replacing both the Ingress Controller and the standalone Envoy proxy with Envoy Gateway.

Envoy Gateway is a Kubernetes-native implementation of the Gateway API. It manages Envoy proxies as the data plane, allowing us to handle both static system routes (Web App, Config Service) and dynamic ephemeral routes (Computing Units) in a single, unified layer.

Key Benefits

• Unified Architecture: Eliminates the maintenance overhead of managing two different proxy technologies.
• Native Dynamic Routing: Envoy natively supports service discovery via xDS, allowing it to route to ephemeral pods in the texera-workflow-computing-unit-pool without the need for constant configuration reloads.
• Modern Standard: Adopting the Kubernetes Gateway API future-proofs our networking stack with standard resources (Gateway, HTTPRoute) rather than vendor-specific annotations.
• Protocol Support: Seamless support for both HTTP/2 and WebSockets, which are critical for the interactive nature of our workflow system.

Architecture Comparison

Current Architecture

Traffic flows through Ingress for static routes, but requires a secondary Envoy hop for dynamic Compute Units.
Current architecture:

Target Architecture

A single Envoy Gateway layer handles all ingress traffic, routing directly to services and dynamically discovering Compute Units.

[aicam]

We realized that one of our limits is how we use Lua script, in our script we rewrite the host to the computing unit but then it makes it non-standard and we are forced to use PatchPolicy which causes many issues.