Skip to main content
Version: 8.2411.x.x RR

Kubernetes Upgrade

To upgrade the environment running on Kubernetes to the next release, follow these steps. Read all the instructions before you begin the upgrade process.

Upgrade for helm installations​

To upgrade the installation, copy over the new images from Upload Nevis Docker Images then run:

# For the temporary credentials, click the download button for one of the Docker images at https://portal.nevis.net/portal/secure/releases/rolling
CLOUDSMITH_PASSWORD=<cloudsmith-pass>
RELEASE_NAME=<release-name>
RELEASE_NAMESPACE=<release-namespace>

helm upgrade nevisadmin4-crd nevisadmin4-crd --repo https://dl.cloudsmith.io/$CLOUDSMITH_PASSWORD/nevissecurity/rolling/helm/charts/
helm get values $RELEASE_NAME -n $RELEASE_NAMESPACE > old-values.yaml
helm upgrade $RELEASE_NAME nevisadmin4 -n $RELEASE_NAMESPACE -f old-values.yaml \
    --repo https://dl.cloudsmith.io/$CLOUDSMITH_PASSWORD/nevissecurity/rolling/helm/charts/ \
    --version <version-to-upgrade>

If the version is not provided, it will upgrade to the latest one.

Helm changelog​

  • v8.2411.0
    • Updated the used Nevis component versions to the 8.2411.0 release.
  • v8.2405.0
    • Updated the used Nevis component versions to the 8.2405.0 release.
    • Added configuration values for OpenTelemetry and Product Analytics
    • Gitea repositories will now be initialized when using the bootstrap option
  • v7.2402.1
    • Fixed the usage of the Git credentials when using the git.username and git.password values.
  • v7.2402.0
    • Updated the used Nevis component versions to the 7.2402.0 release.
    • Added values nevisAdmin4.livenessProbe and nevisAdmin4.readinessProbe to be able to define custom liveness and readiness probes.
  • v7.2311.1
    • Added value database.type to support PostgreSQL database.
  • v7.2311.0
    • Switched to new version schema.
    • Updated the used Nevis component versions to the 7.2311.0 release.
    • Added value nevisOperator.image.tag to make it possible to specify a separate version for nevisOperator.
    • Added value nevisAdmin4.extraEnvs to configure additional environment values for nevisAdmin 4.
    • Added value nevisAdmin4.enabled to make it possible to only deploy the nevisOperator component.
    • Added value nevisAdmin4.ingress.annotations to configure additional annotations for the ingress of nevisAdmin 4.
  • v1.4.3
    • Updated the used Nevis component versions to the 4.20.2 release.
  • v1.4.2
    • Fixed an issue that could cause the installation to fail if https authentication was used for git.
  • v1.4.1
    • Fixed an issue that could cause the installation to fail if tls was enabled for nevisAdmin 4.
  • v1.4.0
    • Updated the used Nevis component versions to the 4.20.0 release.
    • Added value nevisOperator.defaultImagePullPolicy to configure the imagePullPolicy of the deployed Nevis components.
    • Secret handling was refactored to use the specified secrets directly, this improves compatibility with tools that depends on helm template.
      • The following values are now deprecated: git.httpCredentialSecret, git.sshCredentialSecret, database.root.credentialSecret. They are still honored but will be removed in the future.
      • caution

        When using the new database.root.preparedCredentialSecret property, the root-creds secret will no longer be created. Adjust the Root Credential and Root Credential Namespace in the database patterns of nevisAdmin 4 to use the prepared secret before the migration to this value.

  • v1.3.2
    • Use standard duration string for the CA certificate
  • v1.3.1
    • Fixed an issue that could cause the upgrade to fail if nginx.controller.ingressClassResource was defined in the provided values.yaml.
    • Fixed an issue that caused the wrong ingress class to be used in nevisOperator if nginx.controller.ingressClassResource.enabled was set to false.
  • v1.3.0
    • Updated the used Nevis component versions to the 4.19.1 release.
  • v1.2.0
    • Updated the used Nevis component versions to the 4.19 release.
    • Added optional mariadb and gitea dependency.
    • Added value repositoryUrlMap to allow configuring separate git repository url for different component namespaces.
    • Added values for cors and ldap.
    • Added values for overriding nevisAdmin 4 configuration files.
    • Fixed an issue that caused an incorrect passphrase generated for the tls keystore.
    • Added support for Kubernetes v1.26, and removed support for v1.21.
  • v1.1.1
    • Fixed an issue that the initial random password of nevisAdmin 4 was generated with each install. This caused the password in the Kubernetes secret and the actual password to be different.
  • v1.1.0
    • Updated the used Nevis component versions to the 4.18 release.
    • Added separate service account for the nevisAdmin 4 instance.
    • Added new label and annotation values.
    • Adjusted used securityContext for restricted Pod Security Standard.
    • Added multiple new values for credentials that can be used in place of prepared secrets.
    • Updated ingress-nginx chart to 4.4.0.
    • Added support for Kubernetes v1.25, and removed support for v1.20.
  • v1.0.1
    • Added cert-manager related values.
    • Fixed null error when using the --dry-run option.

Performing the upgrade using the wizard script​

EXPERIMENTAL

The Kubernetes upgrade wizard script walks you through the Kubernetes upgrade process with minimal user input required. It can also roll back changes, to some extent.

Download the script kubernetes-upgrade.zip.

This script is in an experimental state. For production environments, it is therefore better to perform the Kubernetes upgrade manually. For instructions, see further below.

Arguments & flags​

ArgumentsDescription
-h, --helpPrints usage information.
-r, --run-optionSets what you want to upgrade. Options: all, nevisadmin, operator, images, crd, permissions, cert-manager, nginx; Default: all; For more details on each option, see the table below.
-n, --nevisadmin4-namespaceDefines the namespace of nevisAdmin4. Must be set if --run-options is set to nevisadmin or all.
-o, --operator-namespaceDefines the namespace of nevisOperator. Must be set if --run-options is set to operator or all.
-c, --container-registrySets the container registry to which the new images are uploaded. This is also the registry from where to pull the images for nevisAdmin4 and nevisOperator. Has to be set.
-i, --internal-registrySets the container registry URL from where the images are pulled by the pods.Default: the value of --container-registry.
--rollbackUse this flag to roll back the operation(s) set in --run-options to their previous state, instead of having them upgraded. Rollback limitations: Note that you cannot roll back the following changes once they are executed by the script: Images being copied to the container registry specified in the --container-registry option; Database schema upgrades of nevisAdmin4; Custom resource definition updates; Permission updates; nginx update; cert-manager, in case it was upgraded from 1.4 to 1.7 (upgrades from 0.10 to 1.4 can be rolled back)

Run options​

--run-optionsDescription
imagesCopies the images to the container registry specified in the --container-registry parameter, from the source registry specified in the config.yaml file.
nevisadminUpdates the database schema of nevisAdmin 4, then nevisAdmin 4 itself. Finally, it restarts nevisAdmin 4.
operatorUpgrades nevisOperator.
crdUpdates the custom resource definitions.
permissionsUpdates the permissions.
cert-managerUse this run-option if cert-manager was installed using the manifest provided in this installation guide. It installs the new version of cert-manager. It also adds an additional CA ClusterIssuer called ca-issuer, which can be used for automatic key management. Do not use this run-option If cert-manager was installed separately.
nginxUse this run-option if nginx was installed with the manifest provided in this installation guide. It installs the new version of nginx. Do not use this run-option If nginx was installed separately.
allRuns all of the above, except for cert-manager and nginx.

The following code snippet shows a sample upgrade script, with the default values used by the installation guide:

./upgrade.sh -r all -c <your-registry>.azurecr.io -o nevisoperator-system -n nevisadmin4

Additional configurations​

The script includes the file config.yaml, which contains some additional configuration options.

The following configuration options are noteworthy:

  • source-registry and source-registry-image-prefix: The script will copy images from this repository to the registry specified by the --container-registry argument. The image prefix is used to determine the full path of the images. For example, in case of docker.cloudsmith.io/nevissecurity/rolling/nevisproxy:4.5 the value of source-registry-image-prefix would be nevissecurity/rolling.
  • db-schema-job: Specifies the name of the DB schema migration job. In case you renamed the job, replace the default value with the new name.

Manual Upgrade Instructions​

The following changes were made to the default configuration found in "Installation on supported platforms". We recommend that you apply them to your own setup as well.

Make sure you are using nevisAdmin4 4.14+, cert-manager 1.6+ and ingress-nginx 1.0+ before upgrading to Kubernetes 1.22

Upgrade from nevisAdmin 7.2402.0 to 8.2405.0+​

kubectl apply -f operator.nevis-security.ch_neviscomponents-8.2405.0.yaml

Instructions for previous versions​

Upgrade from nevisAdmin 4.6 to 4.7+​

  • You have to change the nevisAdmin4 deployment as described in Upgrade nevisAdmin4 from 4.6 to 4.7.

Upgrade from nevisAdmin 4.7 to 4.8​

  • Increase the default wait_timeout to "1800" for the Azure database. This is to avoid closing SQL connections. You can configure this setup manually in the Azure portal, by selecting the database resource, then choosing Server parameters.

  • Increase the threadSynchronizationTimeout to 720 for nevisAdmin4. This is to avoid timeout issues in larger deployments. Add the following code snippet to kubernetes-guide/src/nevis-cluster/kubernetes-nevisadmin4/configs/var/opt/nevisadmin4/conf/nevisadmin4.yml :

    ch:
     nevis:
     admin:
     deployment:
     threadSynchronizationTimeout: 720

  • Increase the default memory limit for the nevisAdmin 4 instance. This is to prevent OOM errors with larger projects. Add the following code snippet to kubernetes-guide/src/nevis-cluster/kubernetes-nevisadmin4/configs/var/opt/nevisadmin4/conf/env.conf:

    JAVA_OPTS='-Dnevisadmin.base.dir=/var/opt/nevisadmin4/data -Dlogback.includeConfigFile=/var/opt/nevisadmin4/conf/logback.xml -Dlogdir=/var/opt/nevisadmin4/logs -Dlogback.stdout.level=INFO -Xmx3072m'

    You can regenerate the configuration with the following command:

    source environment
    ./gradlew bootstrap

    Then re-create the secret containing the nevisAdmin 4 configuration as described in Deploying nevisAdmin 4 Component.

  • Add session stickiness to the proxy ingress. This is to avoid possible issues during redeployment with certain configuration changes. It is also necessary to make the nevisIDM admin UI work when nevisIDM uses multiple replicas. Add the following annotations to the ingress resource, in addition to the current ones:

     nginx.ingress.kubernetes.io/affinity: "cookie"
     nginx.ingress.kubernetes.io/session-cookie-name: "route"
     nginx.ingress.kubernetes.io/session-cookie-expires: "172800"
     nginx.ingress.kubernetes.io/session-cookie-max-age: "172800"

    Edit the currently used ingress:

    # by default component namespace is "component"
    kubectl edit ingress proxy -n <component-namespace>

    The original definition can be found in kubernetes-guide/src/nevis-cluster/kubernetes-component-namespace/ingress-proxy.yaml.

Upgrade from nevisAdmin 4.8 to 4.9+​

A new NevisIngress resource type is added to the nevisOperator, which is used to automatically create ingresses for the components, instead of doing it manually. Before using the new nevisOperator version, the CRD of this resource has to be applied to the cluster: operator.nevis-security.ch_nevisingresses.yaml. This file can also be found here: <kubernetes-guide/src/nevis-cluster/kubernetes-operator/crd/operator.nevis-security.ch_nevisingresses.yaml>

kubectl apply -f operator.nevis-security.ch_nevisingresses.yaml

  • As the nevisOperator now automatically generates the ingresses, the ones that were created manually should be cleaned up after the first successful deployment, that is, when the ingresses generated by nevisOperator are in place.

  • It is recommended to update the NGINX Ingress Controller to the latest version. This can be done by applying the following resource: [nginx-deploy.yaml], which is a part of kubernetes-guide/src/nevis-cluster/kubernetes-nevisadmin4/nginx.yaml

    kubectl apply -f nginx-deploy.yaml

Upgrade from nevisAdmin 4.9 to 4.10+​

  • As the nevisOperator now handles autoscaling, and because of a change in the certificates API in Kubernetes 1.18, you need additional permissions if RBAC is enabled. Adjust the following YAML file[operator-rbac-4-10.yaml] if the used namespace or service account differs from the default nevisoperator-system and default. Apply the resource.

    kubectl apply -f operator-rbac-4-10.yaml

Upgrade from nevisAdmin 4.10 to 4.11+​

  • Apply the additional permissions needed for pod logs: [operator-rbac-4-11.yaml]

    kubectl apply -f operator-rbac-4-11.yaml

  • If you are using the default setup, it is recommended that you increase the storage size from 10GB to 50GB (the new default) for the Azure MariaDB database, as the maximum IOPS is determined by the storage size.

Upgrade from nevisAdmin 4.11 to 4.12+​

  • Apply the additional permissions needed for nevisOperator in [operator-rbac-4-12.yaml]:

    kubectl apply -f operator-rbac-4-12.yaml

  • (Optional) As an experimental feature, cert-manager can be used to generate certificates when using automatic key management. For more information, see cert-manager. If this is used, the upgrade of the cert-manager to 1.4+ is required.

If cert-manager was installed separately, follow the official guide: http://cert-manager.io/docs/installation/upgrading/. Make sure to enable the feature gate ExperimentalCertificateSigningRequestControllers.

If cert-managerwas installed with the manifest provided in the installation guide, do the following:

  1. (Optional) Create a backup for the existing issuers:

    kubectl get --all-namespaces -o yaml issuer,clusterissuer,cert > backup.yaml

  2. Remove the current installation with the [cert-manager-remove.yaml] manifest:

    kubectl delete -f cert-manager-remove.yaml

  3. Install the new version of cert-manager with [cert-manager.yaml]:

    kubectl apply -f cert-manager.yaml

  4. Add issuers with [issuers.yaml], which also creates an additional CA ClusterIssuer called ca-issuer for automatic key management:

    kubectl apply -f issuers.yaml

Upgrade from nevisAdmin 4.12 to 4.13+​

  • Apply the additional permissions needed for nevisOperator in [operator-rbac-4-13.yaml]:

    kubectl apply -f operator-rbac-4-13.yaml

  • From 4.13 onwards cert-manager is the recommended way to generate certificates when using automatic key management. See the instructions below.

Upgrade from nevisAdmin 4.13 to 4.14+​

  • In old versions of the upgrade instructions, the root CA for the cert-manager issuer was created with the default duration, which is too short. As the automatic renewal of this certificate by cert-manager could cause a temporary downtime, it is recommended to create a new issuer for nevisOperator. If nevisOperator was not migrated to cert-manager yet, follow [Migratingtocert-manager] instead.

    kubectl get certificate -n cert-manager selfsigned-ca -o yaml
    apiVersion: cert-manager.io/v1
    kind: Certificate
    metadata:
     name: operator-ca
     namespace: cert-manager
    spec:
     duration: 175320h
     isCA: true
     commonName: operator-ca
     secretName: operator-ca
     privateKey:
     algorithm: ECDSA
     size: 256
     issuerRef:
     name: selfsigned-issuer
     kind: ClusterIssuer
     group: cert-manager.io
    ---
    apiVersion: cert-manager.io/v1
    kind: ClusterIssuer
    metadata:
     name: operator-issuer
    spec:
     ca:
     secretName: operator-ca
    kubectl edit configmap nevisoperator-operator-config-6fc9hfhdhk -n nevisoperator-system
     key-management:
     cert-manager:
     issuer:
     name: operator-issuer
     signing-ca-secret:
     name: operator-ca
     namespace: cert-manager
    kubectl rollout restart deployment nevisoperator-controller-manager -n nevisoperator-system

  • It is recommended to upgrade cert-manager to the latest version, as it is strictly required when using Kubernetes 1.22+. If you use version 1.0+, simply apply the official manifest:

    kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v1.7.0/cert-manager.yaml

    If you use an older version, first see "Migrating to cert-manager" below.

  • As the new version of cert-manager uses a different annotation for the Ingress, update the nevisadmin4 Ingress with the following annotation, if it doesn't already have it. This is only needed if cert-manager is used to generate the certificate for nevisAdmin4.

    cert-manager.io/cluster-issuer: letsencrypt-prod

  • It is recommended to upgrade ingress-nginx to the latest version, as it is strictly required when using Kubernetes 1.22+. If ingress-nginx was deployed using the manifest provided in the installation guide, simply apply the following: [ingress-nginx-update.yaml]

    kubectl apply -f ingress-nginx-update.yaml

    Otherwise refer to the official documentation at `http://kubernetes.github.io/ingress-nginx/.

  • Additionally, it is recommended to use a blocklist to prevent the misuse of the snippet annotation. Extend the nginx-configuration or the ingress-nginx-controller ConfigMap with the following property:

    annotation-value-word-blocklist: load_module,lua_package,_by_lua,location,root,proxy_pass,serviceaccount,',\

Upgrade from nevisAdmin 4.14 to 4.15+​

  • It is recommended to upgrade ingress-nginx to the latest version, as it contains multiple security improvements. For more information see: Kubernetes Blog If ingress-nginx is deployed according to the manifest provided in the installation guide, simply apply the following: [ingress-nginx-update.yaml]

    kubectl apply -f ingress-nginx-update.yaml

    Alternatively, you can use the wizard script with the nginx run option Otherwise see the official documentation.

Upgrade from nevisAdmin 4.15 to 4.16+​

Starting from version 4.15, the Kubernetes cluster can no longer be used to create certificates. Follow Migrating to cert-manager if it is not already used by the nevisOperator.

  • If SSL enforcement is enabled for the database nevisAdmin4 uses, the nevisadmin4-dbschema-wait initcontainer of the nevisAdmin4 StatefulSet has to be extended with the following variable:

    env:
      - name: NEVIS_DBSCHEMA_USE_SSL
        value: "true"

  • It is recommended to upgrade ingress-nginx to the latest version. This requires Kubernetes 1.20 If ingress-nginx is deployed according to the manifest provided in the installation guide, simply apply the following: ingress-nginx-update.yaml

    kubectl apply -f ingress-nginx-update.yaml

    Alternatively, you can use the wizard script with the nginx run option. Otherwise see the official documentation

  • Some unnecessary permissions were removed that are no longer used: operator-rbac-4-16.yaml

    kubectl apply -f operator-rbac-4-16.yaml

    Alternatively, you can use the wizard script with the permissions run option.

  • It is recommended to upgrade cert-manager to the latest version. This requires Kubernetes 1.20

    kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.9.1/cert-manager.yaml

    Alternatively, you can use the wizard script with the cert-manager run option.

  • Apply the new CRDs for the nevisOperator operator.nevis-security.ch_neviscomponents-4.16.yaml, operator.nevis-security.ch_nevisdatabases-4.16.yaml

kubectl apply -f operator.nevis-security.ch_neviscomponents-4.16.yaml
kubectl apply -f operator.nevis-security.ch_nevisdatabases-4.16.yaml

Upgrade from nevisAdmin 4.16 to 4.17+​

kubectl apply -f operator-rbac-4-17.yaml

Alternatively, you can use the wizard script with the permissions run option.

kubectl apply -f operator.nevis-security.ch_neviscomponents-4.17.yaml
kubectl apply -f operator.nevis-security.ch_gitcredentials-4.17.yaml

Alternatively, you can use the wizard script with the crd run option.

Upgrade from nevisAdmin 4.17 to 4.18+​

Upgrade from nevisAdmin 4.18 to 4.19+​

kubectl apply -f nevisadmin4-rbac-4.19.yaml

Alternatively, you can use the wizard script with the permissions run option.

kubectl apply -f operator.nevis-security.ch_neviscomponents-4.19.yaml
kubectl apply -f operator.nevis-security.ch_nevisdatabases-4.19.yaml

Upgrade from nevisAdmin 4.19 to 4.20+​

kubectl apply -f operator.nevis-security.ch_neviscomponents-4.20.yaml
kubectl apply -f operator.nevis-security.ch_nevisdatabases-4.20.yaml
kubectl apply -f operator.nevis-security.ch_nevisingresses-4.20.yaml

Upgrade from nevisAdmin 4.20 to 7.2311.0+​

Upgrade from nevisAdmin 7.2311.0 to 7.2402.0+​

kubectl apply -f operator.nevis-security.ch_neviscomponents-7.2402.0.yaml
kubectl apply -f operator.nevis-security.ch_nevisingresses-7.2402.0.yaml
kubectl apply -f operator.nevis-security.ch_nevisdatabases-7.2311.0.yaml

Uploading New Docker Images​

Upload the Docker images for the current release as described in Uploading Nevis Docker Images of the Example Installation on Azure chapter.

Upgrading nevisAdmin 4​

  1. (Optional) You can use the kubectl cp command to back up the files described in the Database and File System Backup chapter.

    # Find out which pod is used for nevisadmin
    kubectl get pods --all-namespaces | grep nevisadmin4-

    # Example for getting the configuration files.
    # Update the pod name and namespace and copy the configuration files to the local machine
    kubectl cp <namespace>/<podname>:/var/opt/nevisadmin4/conf conf

  2. Upgrade the nevisAdmin4 database: Copy the migration job called nevisadmin4-dbschema from kubernetes-guide or build/nevis-cluster/kubernetes-nevisadmin4/nevisadmin4.yamlto a separate file, then update the image version. Check the following example:

    apiVersion: batch/v1
    kind: Job
    metadata:
     name: nevisadmin4-dbschema
     namespace: nevisadmin4
    spec:
     activeDeadlineSeconds: 300
     template:
     spec:
     restartPolicy: Never
     containers:
     - name: nevisadmin4-dbschema
     image: nevisguidetest.azurecr.io/nevis/nevisadmin4-dbschema:<CURRENT-VERSION>
     imagePullPolicy: Always
     env:
     - name: NEVIS_DBSCHEMA_URL
     value: nevisguidetest.mariadb.database.azure.com:3306/nevisadmin4
     - name: NEVIS_DBSCHEMA_HOSTNAME
     value: nevisguidetest.mariadb.database.azure.com
     - name: NEVIS_DBROOT_USER
     value: nevisguidetest
     - name: NEVIS_DBROOT_PASSWORD
     value: gdfgdf434FVUHGVHGddsd---
     - name: NEVIS_DBSCHEMA_USER
     value: nevisguidetestuser
     - name: NEVIS_DBSCHEMA_PASSWORD
     value: nevisguidetestuser
     - name: NEVIS_DBUSER_USER
     value: nevisguidetestschema
     - name: NEVIS_DBUSER_PASSWORD
     value: nevisguidetestschema
     command:
     - "nevis-dbschema"
     - "migrate"
     resources:
     requests:
     cpu: 20m
     memory: 200Mi
     limits:
     cpu: 1000m
     memory: 1000Mi

    If the old files are no longer available, you can get the configuration used by looking at the old job:

    # namespace is where nevisadmin4 instance is running, by default it is nevisadmin4
    kubectl get job nevisadmin4-dbschema -n <namespace> -o yaml

    Recreate the job:

    # delete old job, by default the namespace is nevisadmin4
    kubectl delete job nevisadmin4-dbschema -n <namespace>

    # create new one, with the previously updated file
    kubectl apply -f <filename> -n <namespace>

  3. Upgrade the running nevisAdmin4 instance. Change the image versions used by the current deployment:

    # image version is given with the registry, example: nevisguidetest.azurecr.io/nevis/nevisadmin4-dbschema:4.8
    # namespace is where nevisadmin4 instance is running, by default it is nevisadmin4
    # full example: kubectl set image statefulset/nevisadmin4 nevisadmin4=nevisguidetest.azurecr.io/nevis/nevisadmin4:4.8 nevisadmin4-dbschema-wait=nevisguidetest.azurecr.io/nevis/nevisadmin4-dbschema:4.8 -n nevisadmin4

    kubectl set image statefulset/nevisadmin4 nevisadmin4=<nevisadmin4-registry-and-version> nevisadmin4-dbschema-wait=<nevisadmin4-dbschema-registry-and-version> -n <namespace>

Upgrading nevisOperator​

Change the image versions used by the current deployment:

# image version is given with the registry, example: nevisguidetest.azurecr.io/nevis/nevisoperator:4.8
# namespace is where the operator instance is running, by default it is nevisoperator-system
# full example: kubectl set image deployment/nevisoperator-controller-manager manager=nevisguidetest.azurecr.io/nevis/nevisoperator:4.8 -n nevisoperator-system

kubectl set image deployment/nevisoperator-controller-manager manager=<registry-and-version> -n <namespace>

Upgrading the operator may cause a rolling deployment to start if the new operator version causes a change in the deployment definitions. See also the Release Notes.

If your environment is not working smoothly after the upgrade, check the chapter Kubernetes Deployment Troubleshooting for debugging instructions.

Upgrading Kubernetes​

As of version 4.15, nevisAdmin 4 supports the Kubernetes versions 1.19, 1.20, 1.21, 1.22 and 1.23.

If you use Azure, you can perform the upgrade from a previous version to 1.23 either from the Azure Portal or with the az tool. For more information, visit:

Downtime during upgrade​

From 4.12 onward, PodDisruptionBudgets are automatically created for the deployments, see Kubernetes Infrastructure Inventory YAML file format among the template files. This means that if at least two replicas are used, at least one replica is constantly available during the upgrade process. It is recommended to also create a PDB for the nginx controller, as without that there may be some service downtime. See an example for this in the nginx.yaml, which is provided in Kubernetes-Based Installation among the template files.

Migrating to cert-manager​

Using the cluster as the signer is deprecated as of 4.13, and will be removed in a future release.

From now on, cert-manager is the recommended way to generate certificates when using automatic key management. For more information, see cert-manager. For this purpose, cert-manager 1.0+ is required. If you already use such a version, the upgrade process can be skipped, and only the new Issuers need to be applied and the configuration for nevisOperator modified.

Upgrading cert-manager​

If cert-manager was installed separately, follow the official guide.

If cert-manager was installed with the manifest provided in the installation guide, do the following:

(Optional) Create a backup for the existing issuers, this is only necessary if you created additional Issuers manually:

kubectl get --all-namespaces -o yaml issuer,clusterissuer,cert > backup.yaml

Remove the current installation with the cert-manager-remove.yaml manifest:

kubectl delete -f cert-manager-remove.yaml

Install the new version of cert-manager with using the official manifest

kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v1.7.0/cert-manager.yaml

Adding the Issuers​

Add issuers with issuers.yaml, which also creates an additional CA ClusterIssuer called ca-issuer for automatic key management:

kubectl apply -f issuers.yaml

Extending nevisOperator configuration​

First make sure you applied the newest RBAC file for nevisOperator, this is automatically done by the upgrade script.

To use the issuers created above, add the following to the nevisOperator config, and then restart nevisOperator. It is also possible to create your own Issuer and refer to that one. For more information about the used parameters see: cert-manager

key-management:
 cert-manager:
 issuer:
 name: ca-issuer
 signing-ca-secret:
 name: ca-root-secret
 namespace: cert-manager
 certificate-duration: 8760h

Either add it to the file in kubernetes-guide/src/nevis-cluster/kubernetes-operator/operator-config.yaml and apply it after generating the configuration with the gradle task, or edit the existing configmap:

kubectl edit configmap nevisoperator-operator-config-6fc9hfhdhk -n nevisoperator-system

You can restart nevisOperator with:

kubectl rollout restart deployment nevisoperator-controller-manager -n nevisoperator-system

Upon restart nevisOperator will replace all the key material used by the current deployments, this can cause a short downtime, while the components are restarting with the new key material.