Scaling
As nevisProxy is the entry point for accessing applications and services, it is a critical component to limit access to these services (see also chapter Attack prevention strategies by mod_qos], but also needs to be correctly scaled to provide the required throughput. Scaling a component involves the following parameters:
- Processes and threads to process requests
- CPU and Memory for processing requests
- Average and maximum response time of backend systems (content providers, DNS, etc.)
- Expected average session lifetime
As it is not easy to compute a correct setup, load tests (simulating the intended usage and load patterns) as well as measuring load in production is necessary, because the resource requirements (especially concerning memory consumption) highly relates on your configuration settings (enabled features, number of stored attributes, etc).
Scaling considerations
Scaling nevisProxy is needed to optimize the following characteristics:
- The maximum number of parallel requests.
- The maximum number of open connections (for example, clients that are able to connect to nevisProxy at the same time as well as the number of connections that are opened to content providers and other services). This is requiring threads, file descriptors for sockets, pipes, files, and other OS resources.
- The maximum number of initiated and authenticated sessions. This is requiring memory for session data to be stored.
The rules and configuration attributes to perform this scaling are shown in the following table:
| Resource | Rules | Configuration |
|---|---|---|
| Shared memory | Used for storing the nevisProxy’s runtime structures such as filters and servlets configurations. |
Required resource is memory, for exact consumption check the log line during startup:
19:17:56.462 4.5.0.2 NavajoStar 44079.14061 5-NOTICE: Service 'nevisproxy' memory configured 10000000 used 81421 available 9984115
memorySize within the Core tag in navajo.xml
| Resource | Rules | Configuration |
|---|---|---|
| Parallel requests (frontend) | The parameter MaxClients specifies the maximum number of worker threads that will be available to serve clients. For more information, see Apache directive(https://httpd.apache.org/docs/2.4/mod/mpm_common.html#maxrequestworkers). Required resources are the memory (for threads and processes), CPU for parallel processing, file descriptors for accepting connections. | MaxClients within the Server tag in navajo.xml |
| Parallel requests (backend) | Depends on the configuration of the connector servlet (for example, the KeepAlive setting), but usually the maximum number of connections to all content providers is equal to the server's MaxClients setting. If connection pooling is enabled, this could mean at most connector servlets *MaxClientsfile descriptors. Required resources are CPU and memory for request processing, file descriptors and threads. | Http(s)ConnectorServlet and BackendConnectorServlet |
| Sessions | If the session storing is solved by aLocalSessionStoreServlet, it requires the configured amount of memory. |
By increasing the log level of the trace groupNPReaperOp to INFO, nevisProxy logs some statistics, including the used and available memory:
13:06:35.263 4.5.0.2 NPReaperOp 90606.1396 6-INFO : LocalCacheTable statistics for LocalSessionStoreServlet: used session: 1, timed out sessions: 0, Used memory: 82888 (0%), available Memory: 49910576
By default, the LocalSessionStoreServlet traces a NOTICE level message if the used memory reaches 70%. At 90%, a CRITICAL message (CACH-0067) is logged and no new sessions will be created. At 95%, an EMERGENCY message (CACH-0068) is logged and no sessions will be created or can be changed anymore.
The parameter MemorySize of the LocalSessionStoreServlet
| Resource | Rules | Configuration |
|---|---|---|
| SSL session cache | Local, in-memory storing of TLS session information for speeding up parallel request processing and enabling client identification based on TLS sessions. Recommended solution is the(https://httpd.apache.org/docs/2.4/mod/mod_socache_shmcb.html)-based shared object cache provider by Apache, while other solution is using TLS cache configuration. | SSLSessionCache or SSLCache tag in navajo.xml |
Therefore, the resulting maximum estimate is the sum of the following parts:
- Memory for the two processes: 2 * ~500MB
- Memory for threads: MaxClients * ~2MB
- Shared memory: memorySize within the Core tag in navajo.xml (10MB by default)
- SSL session cache (optional): Size of data file if shmbc is used (provided by Apache) or maxEntries *maxEntrySize (Navajo cache)
- Local session store (optional): MemorySize of the LocalSessionStoreServlet number of uniqueLocalSessionStoreServlets*
- Any optional setting, configuration: Optional apache modules, lua libraries, modSecurity integration, etc.
Note that the above computation represents the approximate minimal memory consumption at process start and does not reflect the real memory usage. Measuring the real memory consumption, CPU and file descriptor usage during a load test using realistic test data provides more accurate results.
nevisProxy does not support the use of more than 100 connectors in navajo.xml. If you need a larger number of connectors, better use a configuration of several instances, with small setups (up to 10 connectors) per instance.
Request size limitation
Some features (for example, input validation) within nevisProxy may require reading the whole HTTP request body from the client and store them within the process memory. Doing so for many requests in parallel may cause the process to allocate too much memory.
The maximum allocable size might need to be increased by the property bc.lang.system.MaxAllocSize. For more information, see the table. Use this property with great care, and be aware that it can lead to an abnormal high RAM usage if many parallel requests with a big payload arrive at the proxy. The system could even end up in a reboot because both all physical RAM and the swap are used up and the kernel panics. It is advised to set up a memory observer which gracefully restarts the proxy if too much memory is allocated or held by the process. This is a low-level technical parameter, consult support before changing it.
The property ch.nevis.navajo.request.BufferSize defined in defines the maximum request body size that is going to be processed by the content parser. Ensure you set this parameter to the value of the largest request you ever expect needs to be parsed and buffered by nevisProxy.
It is recommended that you define limitations for the allowed size of HTTP request bodies as well as limit the number of parallel HTTP requests with large request bodies. For more information, see. You can also limit the request and response size with ModSecurity rules, using the ModsecurityFilter.