Migration from Ingress to Gateway API

caution

Zero downtime migration is not possible due to the nature of this change. To minimize observed downtime, the recommended strategy is to deploy Ingress and Gateway resources in parallel first, then switch traffic to the Gateway, and clean up afterwards.

This guide assumes that you have a running nevisAdmin 4 Helm installation that currently uses an NGINX Ingress controller and that you want to migrate to Envoy Gateway using the Kubernetes Gateway API.

Prerequisites

• A running nevisAdmin 4 Helm installation using Ingress.
• Helm CLI installed and configured.
• kubectl access to the target Kubernetes cluster.
• cert-manager 1.15 or later installed, unless you manage frontend TLS certificates yourself.

Migration Steps

Step 1: Reconfigure cert-manager for Gateway API Support

note

Skip this step only if you manage frontend TLS certificates yourself and do not rely on cert-manager to issue them. In all other cases, this step is required.

note

Gateway API support requires cert-manager 1.15 or later.

By default, cert-manager integrates with the Kubernetes Ingress API. To use cert-manager with Gateway API, you must enable Gateway API support in the cert-manager controller.

Create a file named cert-manager-gateway-values.yaml with the following content:

config:
  apiVersion: controller.config.cert-manager.io/v1alpha1
  kind: ControllerConfiguration
  enableGatewayAPI: true

Apply the change with Helm:

helm upgrade <cert-manager-release-name> cert-manager \
  --namespace <cert-manager-namespace> \
  --repo https://charts.jetstack.io \
  --reuse-values \
  --version <version> \
  --values cert-manager-gateway-values.yaml

Wait for the cert-manager pods to restart and become ready before proceeding.

Step 2: Deploy Envoy Gateway and Create a GatewayClass

note

Envoy Gateway 1.6.0 or later is required.

Install Envoy Gateway into your cluster using Helm:

helm install <envoy-gateway-release-name> oci://docker.io/envoyproxy/gateway-helm \
  --version <version> \
  --namespace <envoy-gateway-namespace> \
  --create-namespace

Replace <version> with the Envoy Gateway version compatible with your environment (1.6.0 or later).

After the installation, verify that the Envoy Gateway controller is running:

kubectl get pods -n <envoy-gateway-namespace>

Configure Merged Gateways

By default, the Envoy controller creates a separate proxy instance and load balancer service for every Gateway resource. This is usually not the desired behaviour. The recommended approach is to merge all gateways of the same class into a single proxy using an EnvoyProxy configuration resource.

Create a file (for example, envoy-gateway-class.yaml) with the following content:

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
  name: gateway-config
  namespace: <envoy-gateway-namespace>
spec:
  logging:
    level:
      default: warn
  mergeGateways: true
---
apiVersion: gateway.networking.k8s.io/v1
kind: GatewayClass
metadata:
  name: envoy-gateway
  namespace: <envoy-gateway-namespace>
spec:
  controllerName: gateway.envoyproxy.io/gatewayclass-controller
  parametersRef:
    group: gateway.envoyproxy.io
    kind: EnvoyProxy
    name: gateway-config
    namespace: <envoy-gateway-namespace>

Apply it:

kubectl apply -f envoy-gateway-class.yaml

Verify that the GatewayClass is accepted:

kubectl get gatewayclass envoy-gateway

The ACCEPTED field should show True.

Step 3: Upgrade the nevisadmin4-crd Helm Chart

note

This step is only required if you are upgrading nevisAdmin 4 and migrating to Gateway API at the same time. If your installation is already running a version that includes Gateway API support, skip this step.

Upgrade the CRD chart to install the Gateway API-related custom resource definitions required by nevisAdmin 4:

# For the temporary credentials, click the download button for one of the Docker images at https://portal.nevis.net/portal/secure/releases/rolling
CLOUDSMITH_PASSWORD=<cloudsmith-password>

helm upgrade nevisadmin4-crd nevisadmin4-crd \
  --repo https://dl.cloudsmith.io/$CLOUDSMITH_PASSWORD/nevissecurity/rolling/helm/charts/ \
  --version <version> \
  --reuse-values

Wait for the CRD upgrade to complete before proceeding to the next step.

Step 4: Upgrade the nevisadmin4 Helm Chart with Both Ingress and Gateway Enabled

Upgrade the nevisadmin4 Helm chart, ensuring that both Ingress and Gateway API generation are enabled. This allows the parallel deployment of both resource types, which is the basis for the cutover strategy.

It is recommended to use the --reset-then-reuse-values option to cleanly apply the new configuration on top of existing values:

# For the temporary credentials, click the download button for one of the Docker images at https://portal.nevis.net/portal/secure/releases/rolling
CLOUDSMITH_PASSWORD=<cloudsmith-password>

helm upgrade <release-name> nevisadmin4 \
  --namespace <release-namespace> \
  --repo https://dl.cloudsmith.io/$CLOUDSMITH_PASSWORD/nevissecurity/rolling/helm/charts/ \
  --version <version> \
  --reset-then-reuse-values \
  --set ingress.enabled=true \
  --set gateway.enabled=true

Verify that the nevisAdmin 4 pod is running after the upgrade:

kubectl rollout status statefulset/nevisadmin4 -n <release-namespace>

Step 5: Update Plugins and Enable Both Ingress and Gateway in the Inventory

1. Log in to the nevisAdmin 4 web interface.

2. Update plugins for each project to the latest version to ensure Gateway API support is available. For instructions, see Editing Project Pattern Libraries.

3. Open your inventory in the Infrastructure tab.

4. In the inventory's vars section, set the variables to enable both Ingress and Gateway generation:

   vars:
     __nevisadmin_ingress_enabled: true
     __nevisadmin_gateway_enabled: true

5. Save the inventory.

Step 6: Enable Gateway API Generation in the Project and Redeploy

1. Open your configuration project in the Configuration tab.
2. Locate all Virtual Host patterns in the project.
3. For each Virtual Host pattern:
   1. Open the pattern and navigate to the Gateway API tab.
   2. Set Gateway API Generation to Enabled.
   3. Optionally add a Gateway API Settings pattern at Additional Settings of the Advanced Settings tab.
   4. Save the pattern.
4. Deploy the project. nevisAdmin 4 will now generate both NevisIngress and NevisGateway resources.

Wait for the nevisOperator to reconcile the NevisGateway resources. Verify that the Gateway and HTTPRoute resources have been created and are in a ready state:

kubectl get gateway -n <deployment-namespace>
kubectl get httproute -n <deployment-namespace>

Once the Gateway's load balancer has been provisioned by your cloud provider, note the external IP address assigned to the Envoy load balancer service:

kubectl get svc -n <envoy-gateway-namespace>

Step 7: Switch Traffic from Ingress to Gateway API

Once the NevisGateway resources are reconciled and the Envoy load balancer is running, you can perform the traffic cutover. Choose one of the following approaches:

Option A: DNS Update (Recommended)

Update your DNS records to point your domain(s) to the Envoy load balancer IP address instead of the NGINX Ingress load balancer IP address. This approach relies on DNS TTL for propagation and involves a brief DNS resolution transition period.

1. Retrieve the Envoy load balancer IP:

   kubectl get svc -n <envoy-gateway-namespace> -o jsonpath='{.items[*].status.loadBalancer.ingress[*].ip}'

2. Update your DNS A records (or CNAME entries) to point to this IP.

3. Wait for DNS propagation and verify that traffic flows through Envoy Gateway.

Option B: Reuse the NGINX Load Balancer IP (Fixed IP Required)

caution

This option requires that your cloud provider supports configuring a fixed load balancer IP. If no fixed IP is configured, the cloud provider will automatically provision a new IP when the Envoy load balancer service is created, making reuse impossible.

1. Retrieve the current load balancer IP assigned to the NGINX Ingress controller:

   kubectl get svc -n <nginx-namespace> <nginx-ingress-controller> \
     -o jsonpath='{.status.loadBalancer.ingress[0].ip}'

   Note this IP address — it is the value you will assign to the Envoy load balancer service.

2. Stop the NGINX Ingress controller and delete its LoadBalancer Service to release the IP:

   kubectl scale deployment <nginx-ingress-controller-deployment> -n <nginx-namespace> --replicas=0
   kubectl delete svc <nginx-ingress-controller-service> -n <nginx-namespace>

3. Configure the Envoy Gateway load balancer service to use the previously assigned NGINX load balancer IP. This can be done in the EnvoyProxy resource created in Step 2 by setting the loadBalancerIP field and/or the appropriate cloud-provider annotations:

   spec:
     provider:
       type: Kubernetes
       kubernetes:
         envoyService:
           loadBalancerIP: <nginx-load-balancer-ip>
           annotations:
            <provider-specific-annotation-key>: <annotation-value>

4. Verify that the Envoy load balancer acquires the fixed IP.

   kubectl get svc -n <envoy-gateway-namespace> -o jsonpath='{.items[*].status.loadBalancer.ingress[*].ip}'

Step 8: Verify Traffic Flow Through Envoy Gateway

After completing the cutover, confirm that all traffic is being served by Envoy Gateway:

• Access your application endpoints and verify they respond correctly.

• Check the Envoy Gateway logs for traffic:

  kubectl logs -n <envoy-gateway-namespace> -l app.kubernetes.io/component=proxy -f

• Monitor your observability tooling (metrics, traces, logs) for any errors or anomalies.

Step 9: Cleanup — Disable and Remove Ingress Resources

Once traffic has been confirmed to flow through Envoy Gateway, remove the Ingress resources and disable Ingress generation.

1. Disable Ingress Generation in the Inventory

Open the inventory in the Infrastructure tab and update the variable:

vars:
  __nevisadmin_ingress_enabled: false
  __nevisadmin_gateway_enabled: true

Save the inventory.

2. Disable Ingress Generation in the nevisadmin4 Helm Chart

Upgrade the Helm chart to disable Ingress:

helm upgrade <release-name> nevisadmin4 \
  --namespace <release-namespace> \
  --repo https://dl.cloudsmith.io/$CLOUDSMITH_PASSWORD/nevissecurity/rolling/helm/charts/ \
  --version <version> \
  --reset-then-reuse-values \
  --set ingress.enabled=false \
  --set gateway.enabled=true

3. Delete the NevisIngress Resources Manually

List and delete the remaining NevisIngress custom resources:

kubectl get nevisingress -n <deployment-namespace>
kubectl delete nevisingress --all -n <deployment-namespace>

4. Stop the NGINX Ingress Controller (If Not Already Stopped)

If you used Option A for the traffic cutover and the NGINX Ingress controller is still running, remove it using one of the following options depending on how NGINX was installed.

Standalone NGINX Helm release

helm uninstall <nginx-ingress-release> -n <nginx-namespace>

NGINX subchart bundled with the nevisadmin4 Helm chart

Disable the NGINX subchart by upgrading the nevisadmin4 Helm chart with the subchart disabled:

helm upgrade <release-name> nevisadmin4 \
  --namespace <release-namespace> \
  --repo https://dl.cloudsmith.io/$CLOUDSMITH_PASSWORD/nevissecurity/rolling/helm/charts/ \
  --version <version> \
  --reset-then-reuse-values \
  --set nginx.enabled=false

Rollback

If issues arise after cutting over to Gateway API, you can revert to Ingress by:

1. Re-enabling the NGINX Ingress controller (if stopped).
2. Updating the DNS records (or load balancer IP) to point back to the NGINX load balancer.
3. Setting __nevisadmin_ingress_enabled: true in the inventory and redeploying the project.