LoadBalancerServlet

The LoadBalancerServlet can be used to activate load balancing between two or more Servlets (usually BackendConnectorServlets).

The LoadBalancerServlet will mark a Servlet as down in the following situations:

• in case that no IsAliveURI is set, or IsAliveURI.EnablePing is false, or isAliveUri.EnablePing.OnlyInspectPingResponseCode is false:
  • the Servlet will be marked as down when an incoming request with a status code >= 500 is returned. If FailoverThresholdCount is set, then it will be marked as down if more than FailoverThresholdCount requests per FailoverThresholdCount.LifeTime returned a status code >= 500.
• in case that an IsAliveURI is set and that IsAliveURI.EnablePing is true:
  • the Servlet will be marked as down when the call to the IsAliveURI does not return one of the configured IsAliveURI.StatusCodes

The LoadBalancerServlet will mark a Servlet as up in the following situations:

• if no IsAliveURI set:
  • the Servlet will be marked as up after RetryTimeout seconds of the Servlet being marked as down.
• if a IsAliveURI is set
  • the Servlet will be marked as up if, RetryTimeout seconds after being marked as down, the call to the IsAliveURI returns one of the configured IsAliveURI.StatusCodes.

If a new request comes in, the LoadBalancerServlet look for the next Servlet which is up (depending on the Algorithm). If the Servlet responds with a status code >= 500 and no data hasn't been sent yet to the Servlet, then the same request will try the next available Servlet if no IsAliveURI is set, or IsAliveURI.EnablePing is false, or isAliveUri.EnablePing.OnlyInspectPingResponseCode is false.

Classname

ch::nevis::nevisproxy::servlet::loadbalancer::LoadBalancerServlet

Library

libLoadBalanceServlets.so.1

TraceGroup

BC.Tracer.DebugProfile.NPLoadBalance

Configuration

Servlets

Type: comma-sepratated list of servlet names
Usage Constraints: required

Define the servlets over which load balancing has to be done.

RetryTimeout

Type: integer
Unit: seconds
Usage Constraints: required

Timeout after which a connection to a servlet which was down should be retried again.

Algorithm

Type: enum
Possible values: RoundRobin, Failover
Default: RoundRobin

The load balancing algorithm to use.

RoundRobin: an incoming request will use the next servlet used of the previous request and so on. Servlets which are down will be skipped until they come available again. Failover: always the same servlet will be used for all requests, until it goes down. In that case all requests will switch to the next setvlet until this one goes down again and so on.

IsAliveURI

Type: string
Usage Constraints: required, optional

If an IsAliveUri is set, then the LoadBalancerServlet calls this URI when the backend is down. The URI is called all RetryTimeout seconds until the backend returns a status code configured in IsAliveURI.StatusCodes. Once the backend is up, the URI is not called any more, except when the backend is marked down again after a normal request.

IsAliveURI.StatusCodes

Type: comma separated list of numbers
Usage Constraints: optional
Default: 200

If the IsAliveURI is triggered, then a backend will be marked as up if one of the configured IsAliveURI.StatusCodes is returned.

IsAliveURI.EnablePing

Type: boolean
Usage Constraints: optional
Default: false

The configured IsAliveURI will be called after each RetryTimeout seconds. If none of the configured IsAliveURI.StatusCodes is returned, then the backend will be considered as down (independently of the configured FailoverThresholdCount) otherwise it will be considered as up and running.

isAliveUri.EnablePing.OnlyInspectPingResponseCode

Type: boolean
Usage Constraints: optional
Default: false

This parameter will only be evaluated if isAliveUri.EnablePing is true.
With this parameter you can configure in which cases a Servlet will be marked as down.

Parameter Value	Health Check Behavior	Impact on Live Traffic
false (Default)	Reactive: Status is updated by both scheduled pings and live traffic errors.	Immediate failover. If a live request fails, the servlet is instantly marked as "down." The request (and subsequent traffic) is immediately routed to a healthy servlet.
true	Strict Ping: Status is updated only by scheduled pings.	Errors propagate to frontend. If a live request fails, the servlet remains marked as "up." The 5xx error is returned to the user, and traffic continues hitting the broken servlet until the next scheduled ping detects the failure.

FailoverThresholdCount

Type: integer
Usage Constraints: optional, advanced

The number of requests that needs to fail before a servlet is considered defective. Increase this to prevent a failover if an otherwise healthy servlet occasionally fails to satisfy isolated requests.
The failure counters are maintained independently per servlet.
Until the failure counter reaches the configured threshold, the servlet continues to be considered available.

FailoverThresholdCount.LifeTime

Type: integer
Unit: seconds
Range: min: 1
Usage Constraints: required if a FailoverThresholdCount is set

The validity period of the failed request counter. Once the configured time has passed since the last failed request to the servlet, the counter is reset to zero.

SessionSticky

Type: enum
Possible values: Never, Always, IfSessionExists, NoSessionCookie
Default: IfSessionExists
Usage Constraints: optional

This parameter configures the session stickiness and has the following values:

• Never: no session stickiness.
• Always: the same Servlet is used for the same session defined by the SessionManagementFilter. If the Servlet goes down, the next one will be used and the session will stick on this new Servlet.
• IfSessionExists: the same Servlet is used for the same session defined by the SessionManagementFilter only if a session already exists (for example because of an IdentityCreationFilter).
• NoSessionCookie: this feature allows you to implement sticky load balancing without creating real sessions in nevisProxy. Instead of storing the information in the session, an encrypted cookie is sent to the client. This cookie contains the encrypted name of the used Servlet. When the client sends back the NoSessionCookie with the next request, the LoadBalancerServlet verifies, decrypts and reads the cookie to retrieve the Servlet name.

SessionSticky.NoSessionCookie.Name

Type: string
Usage Constraints: optional, advanced
Default: NPNoSession

The name of the cookie if the parameter SessionSticky is set to NoSessionCookie.

SessionSticky.NoSessionCookie.Domain

Type: string
Usage Constraints: optional, advanced

The domain of the cookie if the parameter SessionSticky is set to NoSessionCookie.

SessionSticky.NoSessionCookie.Path

Type: string
Usage Constraints: optional, advanced

The path of the cookie if the parameter SessionSticky is set to NoSessionCookie.

SessionSticky.NoSessionCookie.HttpOnly

Type: boolean
Usage Constraints: optional, advanced
Default: true Secure default: true

If set to true, the HttpOnly attribute is added to the cookie, if the parameter SessionSticky is set to NoSessionCookie.

SessionSticky.NoSessionCookie.Secure

Type: boolean
Usage Constraints: optional, advanced
Default: true
Secure default: true

If set to true, the SessionSticky.NoSessionCookie will have the secure attribute set.

SessionSticky.NoSessionCookie.EncryptionKey

Type: string
Usage Constraints: mandatory if SessionSticky.NoSessionCookie is true.

Symmetric encryption key used to encrypt the information within the cookie (the stickiness information).

Examples

Round robin distribution

In this example the Servlets Backend1 and Backend2 or served in a round robin way for each incoming request. Both Servlets can be either a Http[s]ConnectorServlet (with just one InetAddress configured) or a BackendConnectorServlet.

<servlet>
    <servlet-name>LoadBalancerServlet</servlet-name>
    <servlet-class>ch::nevis::nevisproxy::servlet::loadbalancer::LoadBalancerServlet</servlet-class>
    <init-param>
        <param-name>Servlets</param-name>
        <param-value>Backend1, Backend2</param-value>
    </init-param>
    <init-param>
        <param-name>RetryTimeout</param-name>
        <param-value>10</param-value>
    </init-param>
    <init-param>
        <param-name>Algorithm</param-name>
        <param-value>RoundRobin</param-value>
    </init-param>
</servlet>

Failover distribution over two lines

Assume you have two line with to backends. The first line has the Servlets called Backend1 and Backend2, the second line the Servlets called FallBack1 and Fallback2.

This four Servlets can be either a Http[s]ConnectorServlet (with just one InetAddress configured) or a BackendConnectorServlet.

As long as Backend1 and Backend2 are up, only those will be served. As soon as they go down (for example for maintenance) it switches to FallBack1 and Fallback2.

<servlet>
    <servlet-name>LoadBalancerServlet_Line1</servlet-name>
    <servlet-class>ch::nevis::nevisproxy::servlet::loadbalancer::LoadBalancerServlet</servlet-class>
    <init-param>
        <param-name>Servlets</param-name>
        <param-value>Backend1, Backend2</param-value>
    </init-param>
    <init-param>
        <param-name>RetryTimeout</param-name>
        <param-value>10</param-value>
    </init-param>
    <init-param>
        <param-name>Algorithm</param-name>
        <param-value>RoundRobin</param-value>
    </init-param>
</servlet>

<servlet>
    <servlet-name>LoadBalancerServlet_Line2</servlet-name>
    <servlet-class>ch::nevis::nevisproxy::servlet::loadbalancer::LoadBalancerServlet</servlet-class>
    <init-param>
        <param-name>Servlets</param-name>
        <param-value>FallBack1, FallBack2</param-value>
    </init-param>
    <init-param>
        <param-name>RetryTimeout</param-name>
        <param-value>10</param-value>
    </init-param>
    <init-param>
        <param-name>Algorithm</param-name>
        <param-value>RoundRobin</param-value>
    </init-param>
</servlet>

<servlet>
    <servlet-name>LoadBalancerServlet</servlet-name>
    <servlet-class>ch::nevis::nevisproxy::servlet::loadbalancer::LoadBalancerServlet</servlet-class>
    <init-param>
        <param-name>Servlets</param-name>
        <param-value>LoadBalancerServlet_Line1, LoadBalancerServlet_Line2</param-value>
    </init-param>
    <init-param>
        <param-name>RetryTimeout</param-name>
        <param-value>10</param-value>
    </init-param>
    <init-param>
        <param-name>Algorithm</param-name>
        <param-value>Failover</param-value>
    </init-param>
    <init-param>
        <param-name>IsAliveURI</param-name>
        <param-value>/alive/</param-value>
    </init-param>
</servlet>