Kubernetes Upgrade
To upgrade the environment running on Kubernetes to the next release, follow these steps. Read all the instructions before you begin the upgrade process.
Upgrade for helm installations
To upgrade the installation, copy over the new images from Upload Nevis Docker Images then run:
# For the temporary credentials, click the download button for one of the Docker images at https://portal.nevis.net/portal/secure/releases/lts-202405
CLOUDSMITH_PASSWORD=<cloudsmith-pass>
RELEASE_NAME=<release-name>
RELEASE_NAMESPACE=<release-namespace>
helm upgrade nevisadmin4-crd nevisadmin4-crd --repo https://dl.cloudsmith.io/$CLOUDSMITH_PASSWORD/nevissecurity/lts-202405/helm/charts/
helm get values $RELEASE_NAME -n $RELEASE_NAMESPACE > old-values.yaml
helm upgrade $RELEASE_NAME nevisadmin4 -n $RELEASE_NAMESPACE -f old-values.yaml \
--repo https://dl.cloudsmith.io/$CLOUDSMITH_PASSWORD/nevissecurity/lts-202405/helm/charts/ \
--version <version-to-upgrade>
If the version is not provided, it will upgrade to the latest one.
Helm changelog
- v7.2405.0
- Updated the used Nevis component versions to the
7.2405.0
release. - Added configuration values for OpenTelemetry and Product Analytics
- Gitea repositories will now be initialized when using the
bootstrap
option
- Updated the used Nevis component versions to the
- v7.2402.1
- Fixed the usage of the Git credentials when using the
git.username
andgit.password
values.
- Fixed the usage of the Git credentials when using the
- v7.2402.0
- Updated the used Nevis component versions to the
7.2402.0
release. - Added values
nevisAdmin4.livenessProbe
andnevisAdmin4.readinessProbe
to be able to define custom liveness and readiness probes.
- Updated the used Nevis component versions to the
- v7.2311.1
- Added value
database.type
to support PostgreSQL database.
- Added value
- v7.2311.0
- Switched to new version schema.
- Updated the used Nevis component versions to the
7.2311.0
release. - Added value
nevisOperator.image.tag
to make it possible to specify a separate version for nevisOperator. - Added value
nevisAdmin4.extraEnvs
to configure additional environment values for nevisAdmin 4. - Added value
nevisAdmin4.enabled
to make it possible to only deploy the nevisOperator component. - Added value
nevisAdmin4.ingress.annotations
to configure additional annotations for the ingress of nevisAdmin 4.
- v1.4.3
- Updated the used Nevis component versions to the
4.20.2
release.
- Updated the used Nevis component versions to the
- v1.4.2
- Fixed an issue that could cause the installation to fail if https authentication was used for git.
- v1.4.1
- Fixed an issue that could cause the installation to fail if tls was enabled for nevisAdmin 4.
- v1.4.0
- Updated the used Nevis component versions to the
4.20.0
release. - Added value
nevisOperator.defaultImagePullPolicy
to configure theimagePullPolicy
of the deployed Nevis components. - Secret handling was refactored to use the specified secrets directly, this improves compatibility with tools that depends on
helm template
.- The following values are now deprecated:
git.httpCredentialSecret
,git.sshCredentialSecret
,database.root.credentialSecret
. They are still honored but will be removed in the future. - caution
When using the new
database.root.preparedCredentialSecret
property, theroot-creds
secret will no longer be created. Adjust theRoot Credential
andRoot Credential Namespace
in the database patterns of nevisAdmin 4 to use the prepared secret before the migration to this value.
- The following values are now deprecated:
- Updated the used Nevis component versions to the
- v1.3.2
- Use standard duration string for the CA certificate
- v1.3.1
- Fixed an issue that could cause the upgrade to fail if
nginx.controller.ingressClassResource
was defined in the providedvalues.yaml
. - Fixed an issue that caused the wrong ingress class to be used in nevisOperator if
nginx.controller.ingressClassResource.enabled
was set tofalse
.
- Fixed an issue that could cause the upgrade to fail if
- v1.3.0
- Updated the used Nevis component versions to the
4.19.1
release.
- Updated the used Nevis component versions to the
- v1.2.0
- Updated the used Nevis component versions to the
4.19
release. - Added optional mariadb and gitea dependency.
- Added value
repositoryUrlMap
to allow configuring separate git repository url for different component namespaces. - Added values for cors and ldap.
- Added values for overriding nevisAdmin 4 configuration files.
- Fixed an issue that caused an incorrect passphrase generated for the tls keystore.
- Added support for Kubernetes
v1.26
, and removed support forv1.21
.
- Updated the used Nevis component versions to the
- v1.1.1
- Fixed an issue that the initial random password of nevisAdmin 4 was generated with each install. This caused the password in the Kubernetes secret and the actual password to be different.
- v1.1.0
- Updated the used Nevis component versions to the
4.18
release. - Added separate service account for the nevisAdmin 4 instance.
- Added new label and annotation values.
- Adjusted used
securityContext
forrestricted
Pod Security Standard. - Added multiple new values for credentials that can be used in place of prepared secrets.
- Updated ingress-nginx chart to
4.4.0
. - Added support for Kubernetes
v1.25
, and removed support forv1.20
.
- Updated the used Nevis component versions to the
- v1.0.1
- Added cert-manager related values.
- Fixed null error when using the
--dry-run
option.
Performing the upgrade using the wizard script
The Kubernetes upgrade wizard script walks you through the Kubernetes upgrade process with minimal user input required. It can also roll back changes, to some extent.
Download the script kubernetes-upgrade.zip.
This script is in an experimental state. For production environments, it is therefore better to perform the Kubernetes upgrade manually. For instructions, see further below.
Arguments & flags
Arguments | Description |
---|---|
-h, --help | Prints usage information. |
-r, --run-option | Sets what you want to upgrade. Options: all , nevisadmin , operator , images , crd , permissions , cert-manager , nginx ; Default: all ; For more details on each option, see the table below. |
-n, --nevisadmin4-namespace | Defines the namespace of nevisAdmin4. Must be set if --run-options is set to nevisadmin or all . |
-o, --operator-namespace | Defines the namespace of nevisOperator. Must be set if --run-options is set to operator or all . |
-c, --container-registry | Sets the container registry to which the new images are uploaded. This is also the registry from where to pull the images for nevisAdmin4 and nevisOperator. Has to be set. |
-i, --internal-registry | Sets the container registry URL from where the images are pulled by the pods.Default: the value of --container-registry . |
--rollback | Use this flag to roll back the operation(s) set in --run-options to their previous state, instead of having them upgraded. Rollback limitations: Note that you cannot roll back the following changes once they are executed by the script: Images being copied to the container registry specified in the --container-registry option; Database schema upgrades of nevisAdmin4; Custom resource definition updates; Permission updates; nginx update; cert-manager, in case it was upgraded from 1.4 to 1.7 (upgrades from 0.10 to 1.4 can be rolled back) |
Run options
--run-options | Description |
---|---|
images | Copies the images to the container registry specified in the --container-registry parameter, from the source registry specified in the config.yaml file. |
nevisadmin | Updates the database schema of nevisAdmin 4, then nevisAdmin 4 itself. Finally, it restarts nevisAdmin 4. |
operator | Upgrades nevisOperator. |
crd | Updates the custom resource definitions. |
permissions | Updates the permissions. |
cert-manager | Use this run-option if cert-manager was installed using the manifest provided in this installation guide. It installs the new version of cert-manager. It also adds an additional CA ClusterIssuer called ca-issuer , which can be used for automatic key management. Do not use this run-option If cert-manager was installed separately. |
nginx | Use this run-option if nginx was installed with the manifest provided in this installation guide. It installs the new version of nginx. Do not use this run-option If nginx was installed separately. |
all | Runs all of the above, except for cert-manager and nginx . |
The following code snippet shows a sample upgrade script, with the default values used by the installation guide:
./upgrade.sh -r all -c <your-registry>.azurecr.io -o nevisoperator-system -n nevisadmin4
Additional configurations
The script includes the file config.yaml, which contains some additional configuration options.
The following configuration options are noteworthy:
source-registry
andsource-registry-image-prefix
: The script will copy images from this repository to the registry specified by the--container-registry
argument. The image prefix is used to determine the full path of the images. For example, in case ofdocker.cloudsmith.io/nevissecurity/lts-202405/nevisproxy:4.5
the value ofsource-registry-image-prefix
would benevissecurity/lts-202405
.db-schema-job
: Specifies the name of the DB schema migration job. In case you renamed the job, replace the default value with the new name.
Manual Upgrade Instructions
The following changes were made to the default configuration found in "Installation on supported platforms". We recommend that you apply them to your own setup as well.
Make sure you are using nevisAdmin4 4.14+, cert-manager 1.6+ and ingress-nginx 1.0+ before upgrading to Kubernetes 1.22
Upgrade from nevisAdmin 7.2402.0 to 7.2405.0+
- Apply the new CRD for the nevisOperator operator.nevis-security.ch_neviscomponents.yaml.
kubectl apply -f operator.nevis-security.ch_neviscomponents-7.2405.0.yaml
Instructions for previous versions
Upgrade from nevisAdmin 4.6 to 4.7+
- You have to change the nevisAdmin4 deployment as described in Upgrade nevisAdmin4 from 4.6 to 4.7.
Upgrade from nevisAdmin 4.7 to 4.8
Increase the default
wait_timeout
to "1800" for the Azure database. This is to avoid closing SQL connections. You can configure this setup manually in the Azure portal, by selecting the database resource, then choosing Server parameters.Increase the
threadSynchronizationTimeout
to720
for nevisAdmin4. This is to avoid timeout issues in larger deployments. Add the following code snippet tokubernetes-guide/src/nevis-cluster/kubernetes-nevisadmin4/configs/var/opt/nevisadmin4/conf/nevisadmin4.yml
:ch:
nevis:
admin:
deployment:
threadSynchronizationTimeout: 720Increase the default memory limit for the nevisAdmin 4 instance. This is to prevent OOM errors with larger projects. Add the following code snippet to
kubernetes-guide/src/nevis-cluster/kubernetes-nevisadmin4/configs/var/opt/nevisadmin4/conf/env.conf
:JAVA_OPTS='-Dnevisadmin.base.dir=/var/opt/nevisadmin4/data -Dlogback.includeConfigFile=/var/opt/nevisadmin4/conf/logback.xml -Dlogdir=/var/opt/nevisadmin4/logs -Dlogback.stdout.level=INFO -Xmx3072m'
You can regenerate the configuration with the following command:
source environment
./gradlew bootstrapThen re-create the secret containing the nevisAdmin 4 configuration as described in Deploying nevisAdmin 4 Component.
Add session stickiness to the proxy ingress. This is to avoid possible issues during redeployment with certain configuration changes. It is also necessary to make the nevisIDM admin UI work when nevisIDM uses multiple replicas. Add the following annotations to the ingress resource, in addition to the current ones:
nginx.ingress.kubernetes.io/affinity: "cookie"
nginx.ingress.kubernetes.io/session-cookie-name: "route"
nginx.ingress.kubernetes.io/session-cookie-expires: "172800"
nginx.ingress.kubernetes.io/session-cookie-max-age: "172800"Edit the currently used ingress:
# by default component namespace is "component"
kubectl edit ingress proxy -n <component-namespace>The original definition can be found in
kubernetes-guide/src/nevis-cluster/kubernetes-component-namespace/ingress-proxy.yaml
.
Upgrade from nevisAdmin 4.8 to 4.9+
A new NevisIngress
resource type is added to the nevisOperator, which is used to automatically create ingresses for the components, instead of doing it manually. Before using the new nevisOperator version, the CRD of this resource has to be applied to the cluster: operator.nevis-security.ch_nevisingresses.yaml
. This file can also be found here: <kubernetes-guide/src/nevis-cluster/kubernetes-operator/crd/operator.nevis-security.ch_nevisingresses.yaml>
kubectl apply -f operator.nevis-security.ch_nevisingresses.yaml
As the nevisOperator now automatically generates the ingresses, the ones that were created manually should be cleaned up after the first successful deployment, that is, when the ingresses generated by nevisOperator are in place.
It is recommended to update the NGINX Ingress Controller to the latest version. This can be done by applying the following resource: [nginx-deploy.yaml], which is a part of
kubernetes-guide/src/nevis-cluster/kubernetes-nevisadmin4/nginx.yaml
kubectl apply -f nginx-deploy.yaml
Upgrade from nevisAdmin 4.9 to 4.10+
As the nevisOperator now handles autoscaling, and because of a change in the certificates API in Kubernetes 1.18, you need additional permissions if RBAC is enabled. Adjust the following YAML file[operator-rbac-4-10.yaml] if the used namespace or service account differs from the default
nevisoperator-system
anddefault
. Apply the resource.kubectl apply -f operator-rbac-4-10.yaml
Upgrade from nevisAdmin 4.10 to 4.11+
Apply the additional permissions needed for pod logs: [operator-rbac-4-11.yaml]
kubectl apply -f operator-rbac-4-11.yaml
If you are using the default setup, it is recommended that you increase the storage size from 10GB to 50GB (the new default) for the Azure MariaDB database, as the maximum IOPS is determined by the storage size.
Upgrade from nevisAdmin 4.11 to 4.12+
Apply the additional permissions needed for nevisOperator in [operator-rbac-4-12.yaml]:
kubectl apply -f operator-rbac-4-12.yaml
(Optional) As an experimental feature, cert-manager can be used to generate certificates when using automatic key management. For more information, see cert-manager. If this is used, the upgrade of the cert-manager to 1.4+ is required.
If cert-manager was installed separately, follow the official guide: http://cert-manager.io/docs/installation/upgrading/. Make sure to enable the feature gate ExperimentalCertificateSigningRequestControllers
.
If cert-managerwas installed with the manifest provided in the installation guide, do the following:
(Optional) Create a backup for the existing issuers:
kubectl get --all-namespaces -o yaml issuer,clusterissuer,cert > backup.yaml
Remove the current installation with the [cert-manager-remove.yaml] manifest:
kubectl delete -f cert-manager-remove.yaml
Install the new version of cert-manager with [cert-manager.yaml]:
kubectl apply -f cert-manager.yaml
Add issuers with [issuers.yaml], which also creates an additional CA ClusterIssuer called ca-issuer for automatic key management:
kubectl apply -f issuers.yaml
Upgrade from nevisAdmin 4.12 to 4.13+
Apply the additional permissions needed for nevisOperator in [operator-rbac-4-13.yaml]:
kubectl apply -f operator-rbac-4-13.yaml
From 4.13 onwards
cert-manager
is the recommended way to generate certificates when using automatic key management. See the instructions below.
Upgrade from nevisAdmin 4.13 to 4.14+
In old versions of the upgrade instructions, the root CA for the cert-manager issuer was created with the default duration, which is too short. As the automatic renewal of this certificate by cert-manager could cause a temporary downtime, it is recommended to create a new issuer for nevisOperator. If nevisOperator was not migrated to cert-manager yet, follow [Migratingtocert-manager] instead.
kubectl get certificate -n cert-manager selfsigned-ca -o yaml
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: operator-ca
namespace: cert-manager
spec:
duration: 175320h
isCA: true
commonName: operator-ca
secretName: operator-ca
privateKey:
algorithm: ECDSA
size: 256
issuerRef:
name: selfsigned-issuer
kind: ClusterIssuer
group: cert-manager.io
---
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: operator-issuer
spec:
ca:
secretName: operator-cakubectl edit configmap nevisoperator-operator-config-6fc9hfhdhk -n nevisoperator-system
key-management:
cert-manager:
issuer:
name: operator-issuer
signing-ca-secret:
name: operator-ca
namespace: cert-managerkubectl rollout restart deployment nevisoperator-controller-manager -n nevisoperator-system
It is recommended to upgrade cert-manager to the latest version, as it is strictly required when using Kubernetes 1.22+. If you use version 1.0+, simply apply the official manifest:
kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v1.7.0/cert-manager.yaml
If you use an older version, first see "Migrating to cert-manager" below.
As the new version of cert-manager uses a different annotation for the Ingress, update the nevisadmin4 Ingress with the following annotation, if it doesn't already have it. This is only needed if cert-manager is used to generate the certificate for nevisAdmin4.
cert-manager.io/cluster-issuer: letsencrypt-prod
It is recommended to upgrade ingress-nginx to the latest version, as it is strictly required when using Kubernetes 1.22+. If ingress-nginx was deployed using the manifest provided in the installation guide, simply apply the following: [ingress-nginx-update.yaml]
kubectl apply -f ingress-nginx-update.yaml
Otherwise refer to the official documentation at `http://kubernetes.github.io/ingress-nginx/.
Additionally, it is recommended to use a blocklist to prevent the misuse of the snippet annotation. Extend the
nginx-configuration
or theingress-nginx-controller
ConfigMap with the following property:annotation-value-word-blocklist: load_module,lua_package,_by_lua,location,root,proxy_pass,serviceaccount,',\
Upgrade from nevisAdmin 4.14 to 4.15+
It is recommended to upgrade ingress-nginx to the latest version, as it contains multiple security improvements. For more information see: Kubernetes Blog If ingress-nginx is deployed according to the manifest provided in the installation guide, simply apply the following: [ingress-nginx-update.yaml]
kubectl apply -f ingress-nginx-update.yaml
Alternatively, you can use the wizard script with the nginx run option Otherwise see the official documentation.
Upgrade from nevisAdmin 4.15 to 4.16+
Starting from version 4.15, the Kubernetes cluster can no longer be used to create certificates. Follow Migrating to cert-manager if it is not already used by the nevisOperator.
If SSL enforcement is enabled for the database nevisAdmin4 uses, the nevisadmin4-dbschema-wait initcontainer of the nevisAdmin4 StatefulSet has to be extended with the following variable:
env:
- name: NEVIS_DBSCHEMA_USE_SSL
value: "true"It is recommended to upgrade ingress-nginx to the latest version. This requires Kubernetes 1.20 If ingress-nginx is deployed according to the manifest provided in the installation guide, simply apply the following: ingress-nginx-update.yaml
kubectl apply -f ingress-nginx-update.yaml
Alternatively, you can use the wizard script with the nginx run option. Otherwise see the official documentation
Some unnecessary permissions were removed that are no longer used: operator-rbac-4-16.yaml
kubectl apply -f operator-rbac-4-16.yaml
Alternatively, you can use the wizard script with the permissions run option.
It is recommended to upgrade cert-manager to the latest version. This requires Kubernetes 1.20
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.9.1/cert-manager.yaml
Alternatively, you can use the wizard script with the cert-manager run option.
Apply the new CRDs for the nevisOperator operator.nevis-security.ch_neviscomponents-4.16.yaml, operator.nevis-security.ch_nevisdatabases-4.16.yaml
kubectl apply -f operator.nevis-security.ch_neviscomponents-4.16.yaml
kubectl apply -f operator.nevis-security.ch_nevisdatabases-4.16.yaml
Upgrade from nevisAdmin 4.16 to 4.17+
- Apply the new permissions for nevisoperator: operator-rbac-4-17.yaml
kubectl apply -f operator-rbac-4-17.yaml
Alternatively, you can use the wizard script with the permissions run option.
- Apply the new CRDs for the nevisOperator operator.nevis-security.ch_neviscomponents.yaml, operator.nevis-security.ch_gitcredentials.yaml
kubectl apply -f operator.nevis-security.ch_neviscomponents-4.17.yaml
kubectl apply -f operator.nevis-security.ch_gitcredentials-4.17.yaml
Alternatively, you can use the wizard script with the crd
run option.
Upgrade from nevisAdmin 4.17 to 4.18+
Apply the new CRDs for the nevisOperator operator.nevis-security.ch_neviscomponents.yaml, operator.nevis-security.ch_nevisdatabases.yaml
kubectl apply -f operator.nevis-security.ch_neviscomponents-4.18.yaml
kubectl apply -f operator.nevis-security.ch_nevisdatabases-4.18.yamlAlternatively, you can use the wizard script with the
crd
run option.It's recommended to upgrade cert-manager, follow: https://cert-manager.io/docs/installation/upgrading/#upgrading-using-static-manifests
Alternatively, you can use the wizard script with the
cert-manager
run option.It is recommended to upgrade ingress-nginx. If ingress-nginx was deployed using the manifest provided in the installation guide, simply apply the following: ingress-nginx-update.yaml
kubectl apply -f ingress-nginx-update.yaml
Alternatively, you can use the wizard script with the
nginx
run option.If it wasn't installed with the manifest, refer to the official documentation
Upgrade from nevisAdmin 4.18 to 4.19+
- Apply the new permissions for nevisAdmin 4: nevisadmin4-rbac-4.19.yaml
kubectl apply -f nevisadmin4-rbac-4.19.yaml
Alternatively, you can use the wizard script with the permissions run option.
- Apply the new CRDs for the nevisOperator operator.nevis-security.ch_neviscomponents.yaml, operator.nevis-security.ch_nevisdatabases.yaml
kubectl apply -f operator.nevis-security.ch_neviscomponents-4.19.yaml
kubectl apply -f operator.nevis-security.ch_nevisdatabases-4.19.yaml
Upgrade from nevisAdmin 4.19 to 4.20+
- Apply the new CRDs for the nevisOperator operator.nevis-security.ch_neviscomponents.yaml, operator.nevis-security.ch_nevisdatabases.yaml, operator.nevis-security.ch_nevisingresses.yaml
kubectl apply -f operator.nevis-security.ch_neviscomponents-4.20.yaml
kubectl apply -f operator.nevis-security.ch_nevisdatabases-4.20.yaml
kubectl apply -f operator.nevis-security.ch_nevisingresses-4.20.yaml
Upgrade from nevisAdmin 4.20 to 7.2311.0+
- Apply the new CRD for the nevisOperator operator.nevis-security.ch_nevisdatabases.yaml.
kubectl apply -f operator.nevis-security.ch_nevisdatabases-7.2311.0.yaml
Upgrade from nevisAdmin 7.2311.0 to 7.2402.0+
- Apply the new CRD for the nevisOperator operator.nevis-security.ch_neviscomponents.yaml, operator.nevis-security.ch_nevisingresses.yaml.
kubectl apply -f operator.nevis-security.ch_neviscomponents-7.2402.0.yaml
kubectl apply -f operator.nevis-security.ch_nevisingresses-7.2402.0.yaml
Uploading New Docker Images
Upload the Docker images for the current release as described in Uploading Nevis Docker Images of the Example Installation on Azure chapter.
Upgrading nevisAdmin 4
(Optional) You can use the
kubectl cp
command to back up the files described in the Database and File System Backup chapter.# Find out which pod is used for nevisadmin
kubectl get pods --all-namespaces | grep nevisadmin4-
# Example for getting the configuration files.
# Update the pod name and namespace and copy the configuration files to the local machine
kubectl cp <namespace>/<podname>:/var/opt/nevisadmin4/conf confUpgrade the nevisAdmin4 database: Copy the migration job called
nevisadmin4-dbschema
fromkubernetes-guide
orbuild/nevis-cluster/kubernetes-nevisadmin4/nevisadmin4.yaml
to a separate file, then update the image version. Check the following example:apiVersion: batch/v1
kind: Job
metadata:
name: nevisadmin4-dbschema
namespace: nevisadmin4
spec:
activeDeadlineSeconds: 300
template:
spec:
restartPolicy: Never
containers:
- name: nevisadmin4-dbschema
image: nevisguidetest.azurecr.io/nevis/nevisadmin4-dbschema:<CURRENT-VERSION>
imagePullPolicy: Always
env:
- name: NEVIS_DBSCHEMA_URL
value: nevisguidetest.mariadb.database.azure.com:3306/nevisadmin4
- name: NEVIS_DBSCHEMA_HOSTNAME
value: nevisguidetest.mariadb.database.azure.com
- name: NEVIS_DBROOT_USER
value: nevisguidetest
- name: NEVIS_DBROOT_PASSWORD
value: gdfgdf434FVUHGVHGddsd---
- name: NEVIS_DBSCHEMA_USER
value: nevisguidetestuser
- name: NEVIS_DBSCHEMA_PASSWORD
value: nevisguidetestuser
- name: NEVIS_DBUSER_USER
value: nevisguidetestschema
- name: NEVIS_DBUSER_PASSWORD
value: nevisguidetestschema
command:
- "nevis-dbschema"
- "migrate"
resources:
requests:
cpu: 20m
memory: 200Mi
limits:
cpu: 1000m
memory: 1000MiIf the old files are no longer available, you can get the configuration used by looking at the old job:
# namespace is where nevisadmin4 instance is running, by default it is nevisadmin4
kubectl get job nevisadmin4-dbschema -n <namespace> -o yamlRecreate the job:
# delete old job, by default the namespace is nevisadmin4
kubectl delete job nevisadmin4-dbschema -n <namespace>
# create new one, with the previously updated file
kubectl apply -f <filename> -n <namespace>Upgrade the running nevisAdmin4 instance. Change the image versions used by the current deployment:
# image version is given with the registry, example: nevisguidetest.azurecr.io/nevis/nevisadmin4-dbschema:4.8
# namespace is where nevisadmin4 instance is running, by default it is nevisadmin4
# full example: kubectl set image statefulset/nevisadmin4 nevisadmin4=nevisguidetest.azurecr.io/nevis/nevisadmin4:4.8 nevisadmin4-dbschema-wait=nevisguidetest.azurecr.io/nevis/nevisadmin4-dbschema:4.8 -n nevisadmin4
kubectl set image statefulset/nevisadmin4 nevisadmin4=<nevisadmin4-registry-and-version> nevisadmin4-dbschema-wait=<nevisadmin4-dbschema-registry-and-version> -n <namespace>
Upgrading nevisOperator
Change the image versions used by the current deployment:
# image version is given with the registry, example: nevisguidetest.azurecr.io/nevis/nevisoperator:4.8
# namespace is where the operator instance is running, by default it is nevisoperator-system
# full example: kubectl set image deployment/nevisoperator-controller-manager manager=nevisguidetest.azurecr.io/nevis/nevisoperator:4.8 -n nevisoperator-system
kubectl set image deployment/nevisoperator-controller-manager manager=<registry-and-version> -n <namespace>
Upgrading the operator may cause a rolling deployment to start if the new operator version causes a change in the deployment definitions. See also the Release Notes.
If your environment is not working smoothly after the upgrade, check the chapter Kubernetes Deployment Troubleshooting for debugging instructions.
Upgrading Kubernetes
As of version 4.15, nevisAdmin 4 supports the Kubernetes versions 1.19, 1.20, 1.21, 1.22 and 1.23.
If you use Azure, you can perform the upgrade from a previous version to 1.23 either from the Azure Portal or with the az
tool.
For more information, visit:
Downtime during upgrade
From 4.12 onward, PodDisruptionBudgets
are automatically created for the deployments, see Kubernetes Infrastructure Inventory YAML file format among the template files. This means that if at least two replicas are used, at least one replica is constantly available during the upgrade process. It is recommended to also create a PDB for the nginx controller, as without that there may be some service downtime. See an example for this in the nginx.yaml, which is provided in Kubernetes-Based Installation among the template files.
Migrating to cert-manager
Using the cluster as the signer is deprecated as of 4.13, and will be removed in a future release.
From now on, cert-manager
is the recommended way to generate certificates when using automatic key management. For more information, see cert-manager. For this purpose, cert-manager
1.0+ is required. If you already use such a version, the upgrade process can be skipped, and only the new Issuers
need to be applied and the configuration for nevisOperator modified.
Upgrading cert-manager
If cert-manager was installed separately, follow the official guide.
If cert-manager was installed with the manifest provided in the installation guide, do the following:
(Optional) Create a backup for the existing issuers, this is only necessary if you created additional Issuers manually:
kubectl get --all-namespaces -o yaml issuer,clusterissuer,cert > backup.yaml
Remove the current installation with the cert-manager-remove.yaml
manifest:
kubectl delete -f cert-manager-remove.yaml
Install the new version of cert-manager with using the official manifest
kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v1.7.0/cert-manager.yaml
Adding the Issuers
Add issuers with issuers.yaml
, which also creates an additional CA ClusterIssuer called ca-issuer
for automatic key management:
kubectl apply -f issuers.yaml
Extending nevisOperator configuration
First make sure you applied the newest RBAC file for nevisOperator, this is automatically done by the upgrade script.
To use the issuers created above, add the following to the nevisOperator config, and then restart nevisOperator. It is also possible to create your own Issuer
and refer to that one. For more information about the used parameters see: cert-manager
key-management:
cert-manager:
issuer:
name: ca-issuer
signing-ca-secret:
name: ca-root-secret
namespace: cert-manager
certificate-duration: 8760h
Either add it to the file in kubernetes-guide/src/nevis-cluster/kubernetes-operator/operator-config.yaml
and apply it after generating the configuration with the gradle task, or edit the existing configmap:
kubectl edit configmap nevisoperator-operator-config-6fc9hfhdhk -n nevisoperator-system
You can restart nevisOperator with:
kubectl rollout restart deployment nevisoperator-controller-manager -n nevisoperator-system
Upon restart nevisOperator will replace all the key material used by the current deployments, this can cause a short downtime, while the components are restarting with the new key material.