Configuration project directory structure and YAML file formats - Git

Introduction

The nevisAdmin 4 GUI stores the current application state in a database. To interface with other systems, such as Git, nevisAdmin 4 can export the project configuration data into text file formats. This page describes the formats themselves and the directory structure in which they are stored.

There are three main features relying on this:

• Version control with Git. Git is designed for working with text files.

• Exporting and importing a project (Zip).

• The Generation Engine]. This is a stand-alone tool for generating and deploying the component configuration files. It works without a running nevisAdmin 4 application and does not rely on a database. Instead, it reads the configuration data from YAML files, which may be created by hand or can be checked out from the Git repository.

Working with project files directly

When working with the GUI, you mostly do not see the files. But still there are some situations where you will encounter them:

• When using Git tools to perform merge / branch operations, the files and their contents will be shown.
• The file-level interface is useful to do bulk operations, such as:

The workflow for doing changes at the file level and then re-importing the changes into the GUI is as follows:

• Publish your project to Git
• Check out locally
• Work with the files directly. Refer to the sections below for information about the file structure to use and format conventions to adhere to.
• Push your changes to Git
• Update your project to see the changes in the GUI

At the same time changing the project in the GUI and remotely is not supported. The reason is that this could cause conflicts. The GUI is not designed to handle these.

Common conventions

nevisAdmin 4 uses the following keywords in the project files

expression	Description	Permitted characters	Example
id	Identifier that is unique to an object (collision free). It is a sequence of hex encoded random bytes. Once generated and assigned, an ID may never change.	a-z A-Z 0-9	patternId = edd4e30213107e0941af28d8
key	A key is an identifier of an object or a field of an object that has recognition value to a user and may not be randomly generated. There are technical systems that rely on the key having a specific syntax or the key is used to identify objects toward a user. Keys may not change!	A-Z _ -	tenantKey = SIVENprojectKey= SIVEN-FAB-MASTER
name	The user-given (or 'smart-generated') name of an object for display to users. This may change at any time and is not used for referencing objects. On principle all text characters are allowed, including punctuation, quotes, exotic character sets etc..	any character except single and double quotes and reverse ticks	patternName = "Mobile Banking Login"

Common expressionsFile Format Versioning

Most YAML files start with the schemaVersion property. This versions the data format, including the possible properties and their semantics. When the format is changed in a major way, the schemaVersion will be increased. Currently the version is always "1.0".

Note, some files allow the use of complex data structures not defined in the file format: an example is the properties property in pattern YAML files. The schemaVersion does not change when such embedded structures are changed.

The export.yml file contains an exportVersion property instead of schemaVersion. This property contains the version of the export used to help detect format changes.

Project Key Syntax

Note: the project key is not used within the export / file system interface directly. It is provided by the context in which a project is imported or managed, this section is only here for clarification.

• Project keys MUST start with the tenant key
• Project keys MUST only use capital letters
• Project keys should list repository and branch for clarity, i.e. <TENANT>-<REPOSITORY>-<BRANCH>. This is the default suggested to users on creating a new project.
• The allowed characters are A-Z, 0-9 and underscore "_" and dash "-"

Sanitation Rules for using object names in file names

Some names (ie: patternName) is used in the generated yaml file names. These values are provided by users with very little restrictions, and as such they are sanitized before they are used in file or directory names. To be able to do this, nevisAdmin 4

• first strips all diacritical marks
• replaces any non-alphanumeric characters (a-zA-Z0-9) except '-' and '' by underscores ''
• as a final step any sequence of multiple underscores ('__') is replaced by exactly one underscore.

This means that a generated file or directory name might be slightly different from the name entered in the GUI.

Project Directory Structure and File Formats

<project directory>/

The root directory of the project is given by the use case. This can be a sub-directory of a Git repository, a directory used by the Generation Engine, or a directory exported in a Zip package.

<project directory>/bundles.yml

The plugin bundles file lists all libraries and versions that the configuration project uses.

The project's pattern libraries are stored in the file bundles.yml.

bundles.yml

schemaVersion: "1.0"
bundles:
- "nevisadmin-plugin-nevisproxy:1.1.0-20181112T085833Z"
- "nevisadmin-plugin-monitoring:1.1.0-20181109T103230Z"
- "nevisadmin-plugin-base-generation:4.1.0-20181112T161418Z"
- "nevisadmin-plugin-nevisidm:1.1.0-20181112T095117Z"
- "nevisadmin-plugin-nevisauth:1.1.0-20181112T085934Z"

This is just a sequence of bundle keys with versions. Quotation marks are necessary otherwise YAML will parse as sequence of maps instead of as plain sequence.

<project directory>/bundles/

The plugin bundles directory contains the libraries used by the configuration project as JAR files. It is only present in Zip exports of projects.

<project directory>/bundles/<plugin-name>-<version>-<time-stamp>.jar

The file names are constructed from the name, version, and time-stamp of the library.

<project directory>/variables.yml

The variables file that defines all variables of the project and their default values.

All variables defined in a project are stored in the variables.yml file.

variables.yml

schemaVersion: "1.0"
variables:
 proxy-bind-address: # Set by the user. Variable name must be strictly alphanumeric!
 className: "ch.nevis.admin.v4.plugin.base.generation.property.URLProperty" # defines the class representing the variable type
 parameters: # configures the property type, mainly to be able to display on variables screen
 minRequired: 0
 portInputMode: "REQUIRED"
 hostNameInputMode: "REQUIRED"
 schemeInputMode: "REQUIRED"
 pathInputMode: "NONE"
 allowedSchemes: "http,https"
 value: "http://dummy-proxy-bind-address:8000" # the sample value for this variable
 requireOverloading: true # true: during deployment, the target inventory must specify a value for this variable
 proxy-host-name:
 className: "ch.nevis.admin.v4.plugin.base.generation.property.URLProperty"
 parameters:
 minRequired: 1
 maxAllowed: 1
 portInputMode: "OPTIONAL"
 hostNameInputMode: "REQUIRED"
 schemeInputMode: "OPTIONAL"
 pathInputMode: "NONE"
 allowedSchemes: "http,https"
 value:
 - "http://dummy-proxy-host-name"
 requireOverloading: true # true: during deployment, the target inventory must specify a value for this variable

<project directory>/export.yml

The export file contains meta data about the exported project. It will be only present in Zip exports of projects.

The time of the Zip export can be found in the export.yml file.

export.yml

exportVersion: 1
timestamp: 2019-02-19T10:04:16.512Z

<project directory>/patterns/

The directory containing the patterns. Each configured pattern is stored in a separate file.

<project directory>/patterns/<pattern-name>_<pattern-id>.yml

Pattern file names are constructed out of the sanitized pattern name and its ID.

<project directory>/patterns/<pattern-id>_<property-name>/

Resources are stored in directories dedicated for pattern properties.

<project directory>/patterns/<pattern-id>_<property-name>/<file-name>.<file-extension>

The files keep their original names.

Patterns file structure

schemaVersion: "1.0"
pattern:
 id: "bd73c0740ae895fc8ae722f9" # unique ID identifying the pattern, do not change after initial deployment
 className: "ch.nevis.admin.v4.plugin.nevisproxy.patterns.NevisProxyDeployable" # the type (class) of the pattern
 name: "Webshop Proxy"
 deploymentHosts: "proxy"   # deployment target of the pattern (in case of instance patterns)
 label: "webshop"     # custom group the pattern is assigned to
 link:        # If the pattern was copied from another project, the link holds a reference to the original pattern
 sourceProjectKey: "DEFAULT-WEBSHOP-MASTER"
 sourcePatternId: "bd73c0740ae895fc8ae722f9"
 author: "admin"
 lastCopied: "2020-08-13T09:32:56Z"
 properties:      # properties are defined via their keys. All further data about properties is defined by the pattern type
 host:
 - "pattern://6abafff4a15cd1c122d1e4f6"    # referencing another pattern
 path: "/webshop/"
 backends: "var://webshop-backend"     # referencing a variable

Notes:

• For a list of available patterns/properties, see the reports Pattern library.
• Variables are referenced by setting a scalar property value with the prefix "var://" followed by the variable name.
• Patterns are referenced by setting a scalar property value with the prefix "pattern://" followed by the pattern ID.
• See also the section "File Format Versioning" above.

Pattern Resources (Attached Files)

For various patterns, files can be uploaded to further customize the behavior of Nevis. For example:

• Defining custom error pages
• Uploading JARs with custom AuthStates
• Managing mod_security rules files
• Configuring login screen templates
• Adding resource files such as images and CSS definitions that are required by HTML pages or templates

Resources must belong to a pattern property. They are stored in the same folder as the patterns, in sub-folders called <pattern-id>_<property-name>. The files inside this directory keep their original names and extensions. To add a file manually it's enough to copy it to this folder.

If the folder does not exist yet, it means that the property is not yet set for this pattern. In this case first set the property on the pattern like <property-name>: "res://<pattern-id>#<property-name>", then create the resource's sub-directory and copy the file(s) in there.