Mastering Advanced Traffic Management in Multi-Cloud Kubernetes: Scaling With Multiple Istio Ingress Gateways

In my experience managing large-scale Kubernetes deployments across multi-cloud platforms, traffic control often becomes a critical bottleneck, especially when dealing with mixed workloads like APIs, UIs, and transactional systems. While Istio’s default ingress gateway does a decent job, I found that relying on a single gateway can introduce scaling and isolation challenges.

That’s where configuring multiple Istio Ingress Gateways can make a real difference. In this article, I’ll walk you through how I approached this setup, what benefits it unlocked for our team, and the hands-on steps we used, along with best practices and YAML configurations that you can adapt in your own clusters.

Why Do We Use an Additional Ingress Gateway?

Using an additional Istio Ingress Gateway provides several advantages:

1. Traffic isolation: Route traffic based on workload-specific needs (e.g., API traffic vs. UI traffic or transactional vs. non-transactional applications).
2. Multi-tenancy: Different teams can have their gateway while still using a shared service mesh.
3. Scalability: Distribute traffic across multiple gateways to handle higher loads efficiently.
4. Security and compliance: Apply different security policies to specific gateway instances.
5. Flexibility: You can create any number of additional ingress gateways based on project or application needs.
6. Best practices: Kubernetes teams often use Horizontal Pod Autoscaler (HPA), Pod Disruption Budget (PDB), Services, Gateways, and Region-Based Filtering (via Envoy Filters) to enhance reliability and performance.

Understanding Istio Architecture

Istio IngressGateway and Sidecar Proxy: Ensuring Secure Traffic Flow

When I first began working with Istio, one of the key concepts that stood out was the use of sidecar proxies. Every pod in the mesh requires an Envoy sidecar to manage traffic securely. This ensures that no pod can bypass security or observability policies. 

• Without a sidecar proxy, applications cannot communicate internally or with external sources.
• The Istio Ingress Gateway manages external traffic entry but relies on sidecar proxies to enforce security and routing policies.
• This enables zero-trust networking, observability, and resilience across microservices.

How Traffic Flows in Istio With Single and Multiple Ingress Gateways

In an Istio service mesh, all external traffic follows a structured flow before reaching backend services. The Cloud Load Balancer acts as the entry point, forwarding requests to the Istio Gateway Resource, which determines traffic routing based on predefined policies. 

Here's how we structured the traffic flow in our setup:

1. Cloud Load Balancer receives external requests and forwards them to Istio's Gateway Resource.
2. The Gateway Resource evaluates routing rules and directs traffic to the appropriate ingress gateway:
   • Primary ingress gateway: Handles UI requests.
   • Additional ingress gateways: Route API, transactional, and non-transactional traffic separately.
3. Envoy Sidecar Proxies enforce security policies, manage traffic routing, and monitor observability metrics.
4. Requests are forwarded to the respective Virtual Services, which process and direct them to the final backend service.

This structure ensures better traffic segmentation, security, and performance scalability, especially in multi-cloud Kubernetes deployments.

Key Components of Istio Architecture

• Ingress gateway: Handles external traffic and routes requests based on policies.
• Sidecar proxy: Ensures all service-to-service communication follows Istio-managed rules.
• Control plane: Manages traffic control, security policies, and service discovery.

Organizations can configure multiple Istio Ingress Gateways by leveraging these components to enhance traffic segmentation, security, and performance across multi-cloud environments.

Comparison: Single vs. Multiple Ingress Gateways

We started with a single ingress gateway and quickly realized that as traffic grew, it became a bottleneck. Splitting traffic using multiple ingress gateways was a simple but powerful change that drastically improved routing efficiency and fault isolation. 

On the other hand, multiple ingress gateways allowed better traffic segmentation for APIs, UI, and transaction-based workloads, improved security enforcement by isolating sensitive traffic, and scalability and high availability, ensuring each type of request is handled optimally. 

The following diagram compares a single Istio Ingress Gateway with multiple ingress gateways for handling API and web traffic.

Key takeaways from the comparison:

• A single Istio Ingress Gateway routes all traffic through a single entry point, which may become a bottleneck.
• Multiple ingress gateways allow better traffic segmentation, handling API traffic and UI traffic separately.
• Security policies and scaling strategies can be defined per gateway, making it ideal for multi-cloud or multi-region deployments.

Setting Up an Additional Ingress Gateway

How Additional Ingress Gateways Improve Traffic Routing

We tested routing different workloads (UI, API, transactional) through separate gateways. This gave each gateway its own scaling behavior and security profile. It also helped isolate production incidents — for example, UI errors no longer impacted transactional requests.

The diagram below illustrates how multiple Istio Ingress Gateways efficiently manage API, UI, and transactional traffic.

Figure 3: Multi-Gateway Traffic Flow – External traffic segmentation across API, UI, and transactional ingress gateways.

How it works:

• Cloud Load Balancer forwards traffic to the Istio Gateway Resource, which determines routing rules.
• Traffic is directed to different ingress gateways:
  • The Primary ingress gateway handles UI traffic.
  • The API Ingress Gateway handles API requests.
  • The Transactional Ingress Gateway ensures financial transactions and payments are processed securely.
• The Service Mesh enforces security, traffic policies, and observability.

Step 1: Install Istio and Configure Operator

For our setup, we used Istio’s Operator pattern to manage lifecycle operations. It’s flexible and integrates well with GitOps workflows.

Prerequisites

• Kubernetes cluster with Istio installed
• Helm installed for deploying Istio components

Ensure you have Istio installed. If not, install it using the following commands:

curl -L https://istio.io/downloadIstio | ISTIO_VERSION=$(istio_version) TARGET_ARCH=x86_64 sh - export PATH="$HOME/istio-$ISTIO_VERSION/bin:$PATH"

Initialize the Istio Operator

istioctl operator init

Verify the Installation

kubectl get crd | grep istio

Alternative Installation Using Helm

Istio Ingress Gateway configurations can be managed using Helm charts for better flexibility and reusability. This allows teams to define customizable values.yaml files and deploy gateways dynamically.

Helm upgrade command:

helm upgrade --install istio-ingress istio/gateway -f values.yaml

This allows dynamic configuration management, making it easier to manage multiple ingress gateways.

Step 2: Configure Additional Ingress Gateways With IstioOperator

We defined separate gateways in the IstioOperator config (additional-ingress-gateway.yaml) — one for UI and one for API — and kept them logically grouped using Helm values files. This made our Helm pipelines cleaner and easier to scale or modify. 

Below is an example configuration to create multiple additional ingress gateways for different traffic types:

apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
metadata:
  name: additional-ingressgateways
  namespace: istio-system
spec:
  components:
    ingressGateways:
    - name: istio-ingressgateway-ui
      enabled: true
      k8s:
        service:
          type: LoadBalancer
    - name: istio-ingressgateway-api
      enabled: true
      k8s:
        service:
          type: LoadBalancer

Step 3: Additional Configuration Examples for Helm

We found that adding HPA and PDB configs early helped ensure we didn’t hit availability issues during upgrades. This saved us during one incident where the default config couldn’t handle a traffic spike in the API gateway. 

Below are sample configurations for key Kubernetes objects that enhance the ingress gateway setup:

Horizontal Pod Autoscaler (HPA)

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: ingressgateway-hpa
  namespace: istio-system
spec:
  minReplicas: 2
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 80
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: istio-ingressgateway

Pod Disruption Budget (PDB)

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: ingressgateway-pdb
  namespace: istio-system
spec:
  minAvailable: 1
  selector:
    matchLabels:
      app: istio-ingressgateway

Region-Based Envoy Filter

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: region-header-filter
  namespace: istio-system
spec:
  configPatches:
  - applyTo: HTTP_FILTER
    match:
      context: GATEWAY
      listener:
        filterChain:
          filter:
            name: envoy.filters.network.http_connection_manager
            subFilter:
              name: envoy.filters.http.router
      proxy:
        proxyVersion: ^1\.18.*
    patch:
      operation: INSERT_BEFORE
      value:
        name: envoy.filters.http.lua
        typed_config:
          '@type': type.googleapis.com/envoy.extensions.filters.http.lua.v3.Lua
          inlineCode: |
            function envoy_on_response(response_handle)
              response_handle:headers():add("X-Region", "us-eus");
            end

Step 4: Deploy Additional Ingress Gateways

Apply the configuration using istioctl:

istioctl install -f additional-ingress-gateway.yaml 

Verify that the new ingress gateways are running:

kubectl get pods -n istio-system | grep ingressgateway

After applying the configuration, we monitored the rollout using kubectl get pods and validated each gateway's service endpoint. Naming conventions like istio-ingressgateway-ui really helped keep things organized.

Step 5: Define Gateway Resources for Each Ingress

Each ingress gateway should have a corresponding gateway resource. Below is an example of defining separate gateways for UI, API, transactional, and non-transactional traffic:

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: my-ui-gateway
  namespace: default
spec:
  selector:
    istio: istio-ingressgateway-ui
  servers:
  - port:
      number: 443
      name: https
      protocol: HTTPS
    hosts:
    - "ui.example.com"

Repeat similar configurations for API, transactional, and non-transactional ingress gateways. 

Make sure your gateway resources use the correct selector. We missed this during our first attempt, and traffic didn’t route properly — a simple detail, big impact.

Step 6: Route Traffic Using Virtual Services

Once the gateways are configured, create Virtual Services to control traffic flow to respective services.

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: my-api-service
  namespace: default
spec:
  hosts:
  - "api.example.com"
  gateways:
  - my-api-gateway
  http:
  - route:
    - destination:
        host: my-api
        port:
          number: 80

Repeat similar configurations for UI, transactional, and non-transactional services.

Just a note that VirtualServices gives you fine-grained control over traffic. We even used them to test traffic mirroring and canary rollouts between the gateways.

Resilience and High Availability With Additional Ingress Gateways

One of the biggest benefits we noticed: zero downtime during regional failovers. Having dedicated gateways meant we could perform rolling updates with zero user impact.

This model also helped us comply with region-specific policies by isolating sensitive data flows per gateway — a crucial point when dealing with financial workloads.

• If the primary ingress gateway fails, additional ingress gateways can take over traffic seamlessly.
• When performing rolling upgrades or Kubernetes version upgrades, separating ingress traffic reduces downtime risk.
• In multi-region or multi-cloud Kubernetes clusters, additional ingress gateways allow better control of regional traffic and compliance with local regulations.

Deploying additional IngressGateways enhances resilience and fault tolerance in a Kubernetes environment.

Best Practices and Lessons Learned

Many teams forget that Istio sidecars must be injected into every application pod to ensure service-to-service communication. Below are some lessons we learned the hard way

When deploying additional ingress gateways, consider implementing:

• Horizontal Pod Autoscaler (HPA): Automatically scale ingress gateways based on CPU and memory usage.
• Pod Disruption Budgets (PDB): Ensure high availability during node upgrades or failures.
• Region-Based Filtering (EnvoyFilter): Optimize traffic routing by dynamically setting request headers with the appropriate region.
• Dedicated services and gateways: Separate logical entities for better security and traffic isolation.
• Ensure automatic sidecar injection is enabled in your namespace:
  Plain Text

   

  kubectl label namespace <your-namespace> istio-injection=enabled 

• Validate that all pods have sidecars using:
  Plain Text

   

  kubectl get pods -n <your-namespace> -o wide 
  
  kubectl get pods -n <your-namespace> -o                                             
  jsonpath='{.items[*].spec.containers[*].name}' | grep istio-proxy

• Without sidecars, services will not be able to communicate, leading to failed requests and broken traffic flow.

When upgrading additional ingress gateways, consider the following:

• Delete old Istio configurations (if needed): If you are upgrading or modifying Istio, delete outdated configurations:
  Plain Text

   

  kubectl delete mutatingwebhookconfigurations.admissionregistration.k8s.io istio-sidecar-injector
  
  kubectl get crd --all-namespaces | grep istio | awk '{print $1}' | xargs kubectl delete crd

• Ensure updates to proxy version, deployment image, and service labels during upgrades to avoid compatibility issues.
  YAML

   

  proxyVersion: ^1.18.*
  image: docker.io/istio/proxyv2:1.18.6

• Scaling down Istio Operator: Before upgrading, scale down the Istio Operator to avoid disruptions.
  Plain Text

   

  kubectl scale deployment -n istio-operator istio-operator --replicas=0

• Backup before upgrade:
  Plain Text

   

  kubectl get deploy,svc,cm,secret -n istio-system -o yaml > istio-backup.yaml

Monitoring and Observability With Grafana

With Istio's built-in monitoring, Grafana dashboards provide a way to segregate traffic flow by ingress type:

• Monitor API, UI, transactional, and non-transactional traffic separately.
• Quickly identify which traffic type is affected when an issue occurs in Production using Prometheus-based metrics
• Istio Gateway metrics can be monitored in Grafana & Prometheus to track traffic patterns, latency, and errors.
• It provides real-time metrics for troubleshooting and performance optimization.
• Using PrometheusAlertmanager, configure alerts for high error rates, latency spikes, and failed request patterns to improve reliability.

FYI, we extended our dashboards in Grafana to visualize traffic per gateway. This was a game-changer — we could instantly see which gateway was spiking and correlate it to service metrics. Prometheus alerting was configured to trigger based on error rates per ingress type. This helped us catch and resolve issues before they impacted end users.

Conclusion

Implementing multiple Istio Ingress Gateways significantly transformed the architecture of our Kubernetes environments. This approach enabled us to independently scale different types of traffic, enforce custom security policies per gateway, and gain enhanced control over traffic management, scalability, security, and observability. 

By segmenting traffic into dedicated ingress gateways — for UI, API, transactional, and non-transactional services — we achieved stronger isolation, improved load balancing, and more granular policy enforcement across teams.

This approach is particularly critical in multi-cloud Kubernetes environments, such as Azure AKS, Google GKE, Amazon EKS, Red Hat OpenShift, VMware Tanzu Kubernetes Grid, IBM Cloud Kubernetes Service, Oracle OKE, and self-managed Kubernetes clusters, where regional traffic routing, failover handling, and security compliance must be carefully managed.

By leveraging best practices, including:

• Sidecar proxies for service-to-service security
• HPA (HorizontalPodAutoscaler) for autoscaling
• PDB (PodDisruptionBudget) for availability
• Envoy filters for intelligent traffic routing
• Helm-based deployments for dynamic configuration

Organizations can build a highly resilient and efficient Kubernetes networking stack.

Additionally, monitoring dashboards like Grafana and Prometheus provide deep observability into ingress traffic patterns, latency trends, and failure points, allowing real-time tracking of traffic flow, quick root-cause analysis, and proactive issue resolution.

By following these principles, organizations can optimize their Istio-based service mesh architecture, ensuring high availability, enhanced security posture, and seamless performance across distributed cloud environments.

References

• Istio Architecture Overview
• Istio Ingress Gateway vs. Kubernetes Ingress
• Istio Install Guide (Using Helm or Istioctl)
• Istio Operator & Profiles for Custom Deployments 
• Best Practices for Istio Sidecar Injection
• Istio Traffic Management: VirtualServices, Gateways & DestinationRules