Creating Kubernetes Cluster on Azure

In this tutorial, you will set up Kubernetes cluster in Azure from scratch.

We use Terraform to set up our cluster. Terraform allows managing the infrastructure as code, which allows for additional benefits such as version control.

For an even quicker installation procedure, check Azure deployment automation, where the cluster is set up automatically with nevisAdmin4 running in a few minutes.

Production Usage

This tutorial makes several assumptions. For a production grade setup, review all instructions and Terraform files carefully. There are several aspects that affect the performance, security, and costs of the created cluster.

We will create the following resources in Azure:

• A storage account, which will hold our Terraform configuration.
• A Kubernetes cluster.
• A virtual network, which the cluster will use.
• Two IP address, one to access the cluster, and one to the cluster itself.
• A container registry, which will hold our docker images.
• An Azure database service for PostgreSQL, on which we will create databases containing Nevis-related data.

Prerequisites

Prepare the following:

• Access to an Azure subscription.
• Permission to create resource groups and resources. This includes Application Administrator for creating the service principal, and Owner to assign the required Role to it.
• A Linux environment with the following tools installed:
  • terraform: Terraform command line interface.
  • az: Azure command line interface.

This guide assumes basic knowledge of Linux, Terraform and the Azure CLI. If you are new to these topics, we recommend using the Azure deployment automation instead.

Download Template

The Nevis Kubernetes Support GitHub repository contains a Terraform template for creating a new Kubernetes cluster in Azure. Check out this repository or download the repository content as a ZIP file.

The template is located in the terraform/aks-cluster-setup folder and consists of the following files:

File	Description
bootstrap/terraform-storage.tf	Defines the storage account to store the state of the Terraform managed infrastructure.
aks-cluster.tf	Defines the actual Kubernetes cluster.
container-registry.tf	Defines a Docker container registry accessible in the cluster.
azure.tf	Links the Azure cluster/registry to a storage account.
db.tf	Defines the Azure database server.
db_config.tf	Configures the Azure database server.
variables.tf	Contains the variables used in the template files
terraform.tfvars	Contains the variable values

Set Terraform Variables

Fill out the missing values in the terraform.tfvars file.

For some variables, uniqueness constraints apply and not all characters are allowed.

The supported Kubernetes versions are listed in the Kubernetes Versions Support Policy, but newer versions usually work as well.

Azure Login

Set the following environment variables, use the same values as in the terraform.tfvars:

export SUBSCRIPTION_ID=
export RESOURCE_GROUP_NAME=
export STORAGE_ACCOUNT_NAME=

Now you can log in to Azure as follows:

az login
az account set --subscription $SUBSCRIPTION_ID
az configure --defaults group=$RESOURCE_GROUP_NAME

Terraform Bootstrap

First, you have to bootstrap Terraform before you can create the cluster.

Execute the following commands:

cd bootstrap

# initialize terraform
terraform init

# plan the infrastructure change, ignore the undeclared variable errors
terraform plan -var-file=../terraform.tfvars -out plan

# apply the infrastructure change. Will create a resource group and storage account inside the resource group
terraform apply plan

Now you can get the access key for the storage account and store it in your environment as ARM_ACCESS_KEY.

export ARM_ACCESS_KEY=`az storage account keys list --resource-group $RESOURCE_GROUP_NAME --account-name $STORAGE_ACCOUNT_NAME --query [0].value | tr -d '"'`

Create Cluster

Once you have a storage account, you can create the Kubernetes cluster. This will also set up a database service, as configured in the db.tf and db_config.tf files.

Run the following commands:

cd ..

# initialize terraform
terraform init -backend-config="access_key=$ARM_ACCESS_KEY" \
 -backend-config="storage_account_name=$STORAGE_ACCOUNT_NAME" \
 -backend-config="resource_group_name=$RESOURCE_GROUP_NAME"

# inspect changes
terraform plan -out clusterplan

# apply changes, this will output the randomly generated database password
# it can happen that the created service principal is not ready yet for the cluster, which can result in an error, in this case simply run the command again
terraform apply clusterplan

Cluster creation on Azure might take up to a quarter of an hour. After this, the running costs of your subscription will increase because of the creation of various infrastructure resources.

Some considerations:

• If you are setting up an environment that you plan to keep for a longer time, consider putting the various files and settings into a version control system such as Git.
• To better record your infrastructure as code, consider setting your choices as defaults in the Terraform files, so that the state of your infrastructure is fully defined there.

Connect to Cluster

Use the az aks command to connect to the Kubernetes cluster. You can use output from Terraform instead of passing in the resource group and cluster name.

az aks get-credentials \
  --resource-group $(terraform output -raw aks_cluster_resource_group) \
  --name $(terraform output -raw aks_cluster_name) \
  --overwrite-existing