Appendix C - Performance Samples
Test setup
The hosts used for testing were of the type(https://docs.microsoft.com/en-us/azure/cloud-services/cloud-services-sizes-specs#dv3-series) (Azure) with CentOS 7.5, with the following parameters:
| vCPU | Memory: GiB | Temp Storage (SSD) GiB | Max NICs / Network Bandwidth |
|---|---|---|---|
| 8 | 32 | 200 | 4 / high |
nevisProxy configuration
The purpose of the performance samples here is to show the differences of the session store solutions. That is why we used a simple nevisProxy configuration, consisting of a SessionManagementFilter, an IdentityCreationFilter, and a TestServlet as a backend. The load generating tool created a number of users under a given time, where each user had an initial request, then a login with a nevisAuth authentication, followed by some subsequent authentication requests. The same tests were run with all session store solutions.
Local session store
The first nevisProxy setup, with a local session store, uses a LocalSessionStoreServlet to save the sessions. In this configuration, a load balancer needs to make sure that session stickiness is enabled. This means that each request for the same session has to be sent to the same nevisProxy instance.
| Number of Users | Number of Proxies | Number of Instances per Host | Request/Sec | Mean Response Time |
|---|---|---|---|---|
| 64 000 | 4 | 1 | 1500 | 6 ms |
| 160 000 | 4 | 1 | 3800 | 11 ms |
Remote session store
The second nevisProxy setup, with a remote session store, includes a MySQLSessionStoreServlet, using two DB hosts with replication between them to save the sessions. This setup works without the need of session stickiness, but it is slower due to the fact that the data has to be saved into a database.
| Number of Users | Number of Proxies | Number of Instances per Host | Request/Sec | Mean Response Time |
|---|---|---|---|---|
| 10 000 | 4 | 1 | 220 | 25 ms |
| 25 000 | 4 | 1 | 550 | 36 ms |
| 50 000 | 4 | 1 | 1200 | 228 ms |
Hybrid session store
The final setup, with a hybrid session store, includes the MultiLevelSessionStoreServlet. This setup combines the two solutions above. It uses a local session store to have a fast session cache for the requests, but also stores the sessions in the configured remote session store (two DB hosts with replication) for fail-over scenarios. Because this setup contains a local session store, the use of a session-sticky load balancer is required.
| Number of Users | Number of Proxies | Number of Instances per Host | Request/Sec | Mean Response Time |
|---|---|---|---|---|
| 64 000 | 4 | 1 | 1500 | 6 ms |
| 160 000 | 4 | 1 | 3800 | 10 ms |
| 200 000 | 4 | 1 | 4700 | 17 ms |
The system configuration, the nevisProxy configuration and infrastructure can heavily affect the performance. To measure the real performance, running load tests on the target environment is therefore always recommended. If you have significantly different results on your system or if you require a higher performance, get in contact with Nevis for a consult.
SSD vs HDD
Performance tests of HDD disks against SDD-based disks gave the following results (see the next table). For all tests, we used four proxy instances, each on a different host. The proxy hosts used an SSD for each test.
| Disc type (DB hosts only) | Number of Users | Request/Sec | Mean Response Time |
|---|---|---|---|
| SSD | 24000 | 510 | 368 |
| HDD | 24000 | 508 | 249 |
| SSD | 80000 | 1659 | 197 |
| HDD | 80000 (17% failed) | 1378 | 10745 |