Scaling

Scaling nevisProxy is needed to optimize the following characteristics:

• The maximum number of parallel requests.
• The maximum number of open connections (i.e., clients that are able to connect to nevisProxy at the same time as well as the number of connections that are opened to content providers and other services). This is requiring threads, file descriptors for sockets, pipes, files and other OS resources.
• The maximum number of initiated and authenticated sessions. This is requiring memory for session data to be stored.

The rules and configuration attributes to perform this scaling are shown in the following table:

Resource	Rules	Configuration

| sessions . Usual values for the maxEntrySize parameter of the HttpSession node within the navajo.xml file vary between 64000 and 192000 for the local session cache. The size of the cookie cache entries must not be adapted. The size of the TLS cache entries may vary depending on the requirement to store the client's certificate. Use 1536 for 1-Way and 8000 for 2-Way TLS configurations. All the session stores may be configured on a per connector or engine level. Usually, the SSLCache is configured for each connector, while the HttpSessionCache and the CookieCache are configured within the engine and thus are shared for all services. See also the chapter Remote cache size if you are using the remote session store to share sessions between nevisProxy instances. Required resources: Memory for session entries. | Mandatory: HTTP session | | parallel requests | | parallel requests |

Therefore, the resulting maximum estimate is the sum of the following parts:

• Memory per process: ~500MB
• Memory per thread: MaxClients x ~1MB
• Shared memory (memorySize within Core): services x 15MB
• HttpSessionCache: maxEntries x (maxEntrySize + 32000bytes) This value is not needed if you are using the remote session cache, see the chapter Remote cache size.
• SSLSessionCache: maxEntries x maxEntrySize
• CookieCache: maxEntries x ~1500bytes

Note that the above computation represents the approximate memory consumption at process start and does not reflect the real memory usage. Measuring the real memory consumption and file descriptor usage during a load test using realistic test data provides more accurate results. See the chapter Sample scaling further below for a measured setup.

Important notes

nevisProxy does not support the use of more than 100 connectors in the navajo.xml! If you need a large number of connectors, better use a configuration of several instances, with small setups (up to 10 connectors) per instance.* The new Apache httpd 2.4 needs twice as much resources per thread than the Apache httpd 2.2.

• We suggest using the Dynamic Session Management Engine (instead of the Container Session Handling). First, the Dynamic Session Engine is able to hold more sessions in the same amount of memory than the "old" container session handling can. Furthermore, the Dynamic Session Engine allows you to see at a glance how much memory you need (as you have to configure this).

Note that the HttpSessionCache and CookieCache are not relevant for the Dynamic Session Management Engine. If you are going to use the Dynamic Session Engine, set the attribute maxEntries for both the HttpSessionCache and the CookieCache to "0" in the calculation described in the chapter Scaling" for more information and recommendations.

Sample scaling

This section documents a sample setup for:

• 2000 parallel sessions
• 500 parallel requests

Assuming that the average request latency (the time nevisProxy requires to accept the client's connection, read the request, forward it to the content provider, wait for the request processing by the application and stream back the response to the client) is 0.5 seconds. This setup is able to serve 1000 requests per second.

Assuming furthermore that the clients use a session for about 20 minutes and do not perform a logout (using an inactivity time-out of 10 minutes, which is a common B2C pattern), a session's average lifetime is therefore 30 minutes and we are able to serve 4000 users per hour.

As an emergency time-out, sessions are limited to 2 hours (7200 seconds). Every session unconditionally time-outs after this period.

The resulting configuration fragments are shown below.

navajo.xml:

<Server ...
 Timeout="30"
 MaxClients="500"
 KeepAlive="on"
 MaxKeepAliveRequests="20"
 KeepAliveTimeout ="5"
/>
<Connector>...
 <SSLCache maxEntries="4000"
   maxInactiveInterval="120"
   maxEntrySize="1536"
   maxLifetime="7200"
/>
...
</Connector>

<Engine>
 <CookieCache maxEntries="4000"
   maxInactiveInterval="120"
   maxLifetime="7200"
/>

 <HttpSession maxEntrySize="192000"
   maxEntries="2000"
   maxAttributes="200"
   maxInactiveInterval="120"
/>
...
</Engine>

web.xml:

<filter>
 <filter-name>SSOAuthenticationFilter</filter-name>
 ...
 <init-param>
   <param-name>InactiveInterval</param-name>
   <param-value>600</param-value>
   <description>Maximum interval between two requests</description>
 </init-param>
 ...
</filter>

Memory calculation for this example: 500MB + 500 1MB + 1 15MB + 2000 (192000 + 32000) + 4000 1536 + 4000 + 1500 = *1.4GB

Further remarks about the configuration above:

• Enabling and limiting keepalive on the server allows a client to retrieve multiple resources on the same network connection without being forced to reconnect for each resource (better performance). The client, in this case, binds one nevisProxy thread, which is not available for other clients (less parallelism). It is worth knowing that most browsers are establishing several TCP connections to a HTTP server.
• The inactivity time-out of 120 seconds set on all caches . As a consequence, a user performing a multistep authentication loses the login context and needs to start over in case a single login step took more than 2 minutes.
• The number of entries in the SSLCache, CookieCache, HttpCache has no impact on the performance

Memory limitation

An instance may allocate additional memory during operation. The required memory depends on the processed traffic (how much memory each thread has to allocate at the very same time). You might want to limit the maximum memory consumption of a nevisProxy instance to protect your system resources, especially if you have multiple nevisProxy instances running on a single server. The nevisProxy command line tool nevisproxy features an option called "memlimit". This option allows you to view the current memory usage of all nevisProxy instances on a server as well as to configure a memory limitation for each instance. The memory limitation is implemented by a cron job which measures the memory consumption (virtual memory size of the process in kilobytes) of each nevisProxy instance every 10 minutes. If an instance uses more memory, a new worker thread is started by sending a USR1 signal to the main process of the instance.

Sample output of the nevisproxy memlimit command

----------------------------------------------
nevisProxy, process memory limitation settings
----------------------------------------------

instance: memory: limitation:
-------------------- -------- -----------
1) default 980484 unlimited
2) ext1 1082260 1500000
3) i68044 954624 unlimited
4) nevapl08_ext0 3203672 unlimited
5) proxy6 1856988 4000000
6) proxy7 2031732 6000000
7) remotecache 986808 4300000

h) Help
q) Quit

select instance:

If you are using a hardware security module (HSM), enable the Nevis cryptoki wrapper library (see the chapter "Security Modules" on page ) when enabling the "memlimit option.

Alternatively to this "soft" limitation, a "hard" limit may be configured using the CMD_ULIMIT parameter within the env.conf file of the instance. Setting a "hard" limit will immediately stop the process (the process will most probably crash when hitting this limit because it is no longer able to allocate memory).

Sample entry within the env.conf file

RTENV_SECURITY_CHECK=no_shell
CMD_ULIMIT="-v 4300000"

Request size limitation

Some features (e.g. input validation) within nevisProxy may require reading the whole HTTP request body from the client and store them within the process memory. Doing so for many requests in parallel may cause the process to allocate too much memory, see also the chapter Memory limitation.

The maximum allocatable size might need to be increased by the property bc.lang.system.MaxAllocSize . Use this property with great care, and be aware that it can lead to an abnormal high RAM usage if many parallel requests with a big payload arrive at the proxy. The system could even end up in a reboot because both all physical RAM and the swap are used up and the kernel panics. It is advised to set up a memory observer which gracefully restarts the proxy if too much memory is allocated or held by the process.

The property ch.nevis.navajo.request.BufferSize defines the maximum request body size which is going to be processed by the content parser. Ensure you set this parameter to the value of the largest request you ever expect needs to be parsed and buffered by nevisProxy.

It is recommended that you define limitations for the allowed size of HTTP request bodies as well as limit the number of parallel HTTP requests with large request bodies. See also Configuring HTTP protocol validation and limitation. You can also limit the request and response size with ModSecurity rules, by using the ModsecurityFilter.

Remote cache size

When using the remote cache to share user sessions between several nevisProxy instances, the number of active sessions is limited by the memory configured for the database.

The size of each session entry depends on the number of user attributes which shall be stored (specified by the nevisAuth configuration) as well as the application cookies and stickiness information. We recommend determining the memory requirement by measuring the average memory consumption within your environment.

Example command MySQL:

ndb_mgm -e "ALL REPORT MEMORY USAGE"