Appendix H - Deprecation List

This page lists every currently deprecated functionality of nevisProxy, with also a planned removal date. Check regularly if your current configuration is concerned about these planned removals. If one of the below feature is still used in your setup, then there is also a deprecation notice in the navajo log file, either during startup or during incoming request. If you are affected, adapt your configuration by using a replacement or contact support if you depend on the deprecated functionality.

ServerAdmin attribute of the Server tag

Replacement: no replacement
Deprecated since: 5.2.0
Plan to remove: version 9.x

The SSLCertificateChainFile attribute of the SSL tag

Replacement: Use the SSLCertificateFile Apache directive instead.
Deprecated since: 5.3.0
Plan to remove: version 9.x

The SSLRequire attribute of the SSL tag

Replacement: in general replaced by the Require expression by Apache.
Deprecated since: Before LTS19
Plan to remove: version 9.x

The Protocol parameter of the HttpConnectorServlet

Replacement: no replacement
Deprecated since: 8.2405.0
Plan to remove: version 9.x

The Loglevel attribute of the Server tag

Replacement: Use LogLevel instead
Deprecated since: 8.2405.0
Plan to remove: version 9.x

The AwaitingResponse parameter of the HttpConnectorServlet

Replacement: no replacement
Deprecated since: 8.2405.0
Plan to remove: version 9.x

The LegacyRegexpMatching parameter of the HttpConnectorServlet

Replacement: no replacement
Deprecated since: 8.2405.0
Plan to remove: version 9.x

The CheckAlwaysClientCert parameter of the IdentityCreationFilter and SecurityRoleFilter

Replacement: no replacement
Deprecated since: 8.2411.0
Plan to remove: version 9.x

Description

• Type: boolean
• Usage Constraints: optional, basic feature
• Default: false
• If the client presents a certificate during the first login request, then there is a check to confirm that the same certificate is present during subsequent requests. This check can be disabled by setting the attribute to false.

The H2SerializeHeaders attribute of the H2 tag in navajo.xml

Replacement: no replacement
Deprecated since: 8.2411.0
Plan to remove: version 9.x

ch.nevis.navajo.SessionCleanupWaitTimeout

Replacement: use ch.nevis.navajo.ListenerWaitTimeout instead
Deprecated since: 8.2411.0
Plan to remove: version 9.x

Description

Type: Integer
Unit: msec
Default: 30000

The maximal time to wait for a reaper-call, before shutting down an instance.

ch.nevis.isiweb4.auth.ExternalHint

Replacement: use request:reauthenticate() instead
Deprecated since: 8.2411.0
Plan to remove: version 9.x

Description

Attribute recognized by the IdentityCreationFilter. May be used to temporarily lock an authenticated session in a Lua script. Example:

request:setAttribute("ch.nevis.isiweb4.auth.ExternalHint", "lock")

org.apache.request.ParsedUri

Replacement: The parameter AllowEncodedSlashes may be used instead
Deprecated since: 8.2411.0
Plan to remove: version 9.x

Description

Type: Boolean
Default: true

If set to true, this parameter decodes the encoded URI of an incoming request for internal nevisProxy processing. For example the parameter turns the URI /UIFont%20CMSStyle.swift into /UIFont CMSStyle.swift, as %20 stands for ' '. If set to false, the parameter does not modify anything and forwards the undecoded URI unchanged.

This parameter is directly related to the parameter URLEncoding of the servlet HttpConnectorServlet. The URLEncoding parameter encodes the outgoing URI from nevisProxy to the back-end application. We highly recommend setting both parameters to true. If your setup requires one of the parameters to be false, set the other one to false too, on all HttpConnectorServlets. The parameter URLEncoding can also be set to false centrally by adding the code line ch.nevis.isiweb4.servlet.connector.http.URLEncoding=false to the bc.properties configuration file.

The Lua request method getRequestUri

Replacement: Use the method getRequestPath instead
Deprecated since: 8.2411.0
Plan to remove: version 9.x

Description

Returns the URI of the request. The URI consists of the context path, the servlet path and the path info, if given.

The Lua request method setRequestUri

Replacement: Use the method setRequestPath combined with setQuery instead
Deprecated since: 8.2411.0
Plan to remove: version 9.x

Description

Sets the URI of the request. Only use this method for modifying the request URI of a side call request, as the modification of the original request's URI may result in undefined behavior.

request:setRequestUri("/set/side/call/uri")

The Lua request methods getContextPath and setContextPath

Replacement: none
Deprecated since: 8.2505.0
Plan to remove: version 9.x

Description

getContextPath()

Returns the context path on which the web.xml is mapped (see navajo.xml).

setContextPath(path)

Overwrites the context path; the overwritten context path is visible to all filters mapped after this LuaFilter.

The value client-side of the ProxyPolicy parameter in the HttpConnectorServlet

Replacement: none
Deprecated since: 8.2505.0
Plan to remove: version 9.x

Description

• client-side: The proxy acts as a client-side proxy for HTTP.

The commands nevisproxy genrules nevisproxy whitelist and the practices URI whitelisting and Self-learning of input validation

Replacement: Use the ModSecurityFilter instead
Deprecated since: 8.2505.0
Plan to remove: version 9.x

Description

URI whitelisting

A whitelist of all allowed URIs (path portion) may be generated based on existing access.log files. These rules are used to restrict access to known handlers of the application only, and may be used in addition to the parameter whitelisting described in the previous section, Self-learning of input validation. The following example demonstrates a shell script which generates/updates a request path whitelisting.

#!/bin/sh
PATH=/usr/bin:/bin:/opt/nevisproxy/bin
 
# the OPTIONS variable controls the rule generation, enter "nevisproxy default genrules"
 
# for a list of available options
OPTIONS="-h"
 
# PFX is the name (path portion) of your application
PFX="appl"
 
 
# this sample script uses the "default" proxy instance
 
# the log file contains all relevant URIs and is used every time you update the rules
LOG=/var/opt/nevisproxy/default/logs/$PFX.log
 
# the rule db file contains the URI pattern you may edit
DB=/var/opt/nevisproxy/default/logs/$PFX.db
TMP=$DB.tmp
 
# filters the access.log file by the defined application path /$PFX cat /var/opt/nevisproxy/default/logs/access.log | \
cut -d ' ' -f 7 | cut -d '?' -f 1 | grep "^/$PFX/" > $LOG.tmp
touch $DB
touch $LOG
cat $LOG >> $LOG.tmp
nevisproxy default genrules -i $LOG.tmp -c $DB $OPTIONS > $TMP
rm -f $LOG.tmp
grep "# ADD line " $TMP | cut d ' ' -f 5 > $LOG
grep "^QS_PermitUri +QSF" $TMP > $DB
rm -f $TMP
 
echo "ch.nevis.isiweb4.filter.validation.$PFX.Description=generated for /$PFX"
echo "ch.nevis.isiweb4.filter.validation.$PFX.BlockOnError=true"
echo "ch.nevis.isiweb4.filter.validation.$PFX.DecodingRules=URL_decode_unicode"
echo "ch.nevis.isiweb4.filter.validation.$PFX.RegexpType=PCRE"
cat $DB | awk '{print substr($0, 27, length($0)-27)}' | \
    awk -F'"' '{ print "ch.nevis.isiweb4.filter.validation..URIWhiteList." NR-1 "=" $2}'| \
    sed -e "s::$PFX:g"

Such a script may be executed multiple times until it has learned all possible URIs and the output (stdout) can be added to the customer profiles to be mapped to the application which has been defined by the PFX variable within the script profile, see the chapter "Managing custom profiles.

Self-learning of input validation

The InputValidationFilter provides a learning mechanism to generate the ParameterRules patterns, which are used to protect an application.

This learning mechanism is based on two components:

• The first component is an audit log that contains the data transferred from clients to the server. This audit log is generated by the InputValidationFilter that has been mapped to the application's URL.
• The second component is a rule generator that processes the audit log data and creates ParameterRules rules matching the captured data.

The following steps are necessary to learn a new rule set:

1. Add an InputValidationFilter and map it to the URL of the application. Configure the audit.log facility of the filter. You also need to specify the decoding you want to apply to the incoming data. It is important to use exactly the same decoding during the learning phase that you are going to use later when the rules are enforced.

Example filter configuration:

<filter>
  <filter-name>SelfLearningValidationFilter</filter-name>
  <filter-class>ch::nevis::isiweb4::filter::validation::InputValidationFilter</filter-class>
  <init-param>
    <param-name>AuditLog.FileName</param-name>
    <param-value>/var/opt/nevisproxy/<instance_name>/logs/test_input_audit.log</param-value>
  </init-param>
  <init-param>
    <param-name>AuditLog.PassPhrase</param-name>
    <param-value>${exec: cat /var/opt/neviskeybox/default/public/node_keypass}</param-value>
  </init-param>
  <init-param>
    <param-name>DecodingRules</param-name>
    <param-value>URL_decode_unicode</param-value>
  </init-param>
</filter>
 
<filter-mapping>
  <filter-name>SelfLearningValidationFilter</filter-name>
  <url-pattern>/appl/servlet/</url-pattern>
</filter-mapping>

2. Run the rule generator multiple times, for example every hour. The rule generator processes the audit log as well as the existing rules and updates them. Repeat this process multiple times (usually for several days) until the rules do not change anymore.

You can invoke the rule generator with the nevisproxy whitelist command, which has the following syntax:

Usage:
  nevisproxy <instance> whitelist <application> <passphrase> <audit log> <rule file> ['utf8'] ['varnames']
 
Options:
 application: name of the rule set to generate
 passphrase:  the passphrase to decrypt the data (or a program writing the passphrase to stdout)
 audit log:   audit log file to read the collected data from
 rule file:   rule file to read/write the generated rules
 utf8:        enables utf8 multibyte character support
 varnames:    generates ParameterRules rules with variable parameter names

The next code block gives an example of the nevisproxy whitelist command:

nevisproxy siven whitelist test-profile some_pwd /var/opt/nevisproxy/siven/logs/test_input_audit.log /var/opt/nevisproxy/siven/conf/profile.properties varnames

The command generates a file called profile.properties. This file contains the profile "test-profile" with parameter rules based on the content of the test_input_audit.log file:

# profile 'test-profile' implementing input validation
# statistics 'rule.'<id>'='<name>':'<total updates>':'<total hits>' '<updates>':'<hits>
# rule.0=test_param_1:2:18 0:6
# rule.1=test_param_2:2:18 0:6
# rule.2=test_param_3:1:4 0:2
ch.nevis.isiweb4.filter.validation.test-profile.Description=input validation, generated Mon Apr 20 13:32:08 2020, 0 updates
ch.nevis.isiweb4.filter.validation.test-profile.BlockOnError=true
ch.nevis.isiweb4.filter.validation.test-profile.RegexpType=PCRE
ch.nevis.isiweb4.filter.validation.test-profile.ParameterRules.0=test_param_1:^[a-zA-Z0-9]{6,13}$:allow:silent
ch.nevis.isiweb4.filter.validation.test-profile.ParameterRules.1=test_param_1:.*:deny
ch.nevis.isiweb4.filter.validation.test-profile.ParameterRules.2=test_param_2:^[a-zA-Z0-9]{8,31}$:allow:silent
ch.nevis.isiweb4.filter.validation.test-profile.ParameterRules.3=test_param_2:.*:deny
ch.nevis.isiweb4.filter.validation.test-profile.ParameterRules.4=test_param_3:^[\?]{0,1}$:allow:silent
ch.nevis.isiweb4.filter.validation.test-profile.ParameterRules.5=test_param_3:.*:deny
ch.nevis.isiweb4.filter.validation.test-profile.ParameterRules.6=.*:.*:allow:log

3. Once the rules no longer change after you invoked the rule generator, you can upload the profile into the InputValidationFilter (the audit log configuration is no longer necessary):

<filter>
  <filter-name>SelfLearningValidationFilter</filter-name>
  <filter-class>ch::nevis::isiweb4::filter::validation::InputValidationFilter</filter-class>
  <init-param>
    <param-name>Profile</param-name>
    <param-value>test-profile</param-value>
  </init-param>
</filter>

4. Restart the proxy instance and monitor the navajo.log file. Update the rule set if there are any false rule violations (false positives) within the navajo.log file.

Where does the InputValidationFilter learn?

The InputValidationFilter learns rules by activating the audit log either within a production or test environment:

Environment	Advantages	Disadvantages
Production	- You can invoke the rule generator until there are no further updates to the rules. - You can directly activate the new profile.- The generator can learn from real data that provides an accurate rule set.	- There is the risk of learning rules from invalid user input or even from an attempted attack.
Test	- There is no risk of learning rules from an invalid pattern or potential attack.	- You need to define extensive test patterns to learn from. - You need to test the generated profile in the production environment in permissive mode (BlockOnError=false).

The helper tool qsfilter2

Replacement: none
Deprecated since: 8.2505.0
Plan to remove: version 9.x

Description

Rule generator. Creates QS_Permit directives and rule patterns from audit log files.

The helper tool wlsg

Replacement: none
Deprecated since: 8.2505.0
Plan to remove: version 9.x

Description

Internal binary for rule generation for the self-learning feature of input validation.

The helper tool qsrotate

Replacement: none
Deprecated since: 8.2505.0
Plan to remove: version 9.x

Description

Log rotation tool similar to Apache's rotatelogs.

The helper tool qssign

Replacement: none
Deprecated since: 8.2505.0
Plan to remove: version 9.x

Description

A log data integrity check tool. It reads log data from stdin (pipe) and writes the signed data to stdout adding a sequence number and signature to every log line. qssign.rb is a Logstash filter plug-in that you can use to verify the signatures of log messages in real time.

The helper tool qstail

Replacement: none
Deprecated since: 8.2505.0
Plan to remove: version 9.x

Description

Shows the end of a log file beginning at a defined pattern.

The helper tool qsgrep

Replacement: none
Deprecated since: 8.2505.0
Plan to remove: version 9.x

Description

Searches a file for a pattern and prints the data in a new format.

The helper tool qsexec

Replacement: none
Deprecated since: 8.2505.0
Plan to remove: version 9.x

Description

Command execution triggered by patterns within log files.

The helper tool qscheck

Replacement: none
Deprecated since: 8.2505.0
Plan to remove: version 9.x

Description

Monitor tool testing the TCP connectivity to servers used by mod_proxy.

The helper tool qshead

Replacement: none
Deprecated since: 8.2505.0
Plan to remove: version 9.x

Description

A utility that reads from stdin and prints all lines to stdout until reaching the defined pattern.

The helper tool qsdt

Replacement: none
Deprecated since: 8.2505.0
Plan to remove: version 9.x

Description

Simple tool to measure the elapsed time between related log messages.

The helper tool ipfilter

Replacement: none
Deprecated since: 8.2505.0
Plan to remove: version 9.x

Description

Simple tool to replace/hide IP addresses in log files.

The MQServlet

Replacement: none
Deprecated since: 8.2505.0
Plan to remove: version 9.x

Description

A connector where any POST body will be sent as an MQ message to the defined broker. This servlet can be either used like any normal servlet (i.e., REST clients) and/or with side calls from LuaFilter scripts.

Classname

ch::nevis::nevisproxy::servlet::sink::MQServlet

Library

libSinkServlet.so.1

Configuration

BrokerURI

Type: string
Usage Constraints: required, basic

Defines the broker uri. See the official documentation of activeMQ for more information. Example: tcp://your.broker.ch:61616

TargetURI

Type: string
Usage Constraints: required, basic

Define target name of queue or topic. Example: my.example.uri

Username

Type: string
Usage Constraints: required, basic

Define the username for accessing the broker.

Password

Type: string
Usage Constraints: required, basic

Define the password for accessing the broker.

Type

Type: enum
Usage Constraints: optional, basic
Default: Topic

Define target type, possible values are "Queue" or "Topic".