Load Balancing Containerized Applications

The container technology is growing at a rapid pace among startups and enterprises. Using containers is relatively easy, but when creating highly scalable production grade applications using micro service architectures on multi-host environment, things are not that simple anymore. One of the most crucial component for building any such applications is to have a high quality load balancer that can be used to route traffic between different parts of the service and handle the traffic coming form clients among other things.

When choosing a load balancer, typically users select between HA proxy and Nginx. Both solutions have pros and cons. With the official Docker haproxy image, you can add your config file to the official image, then build and run it, or you can use volumes to mount your host config file to the container. Either way, you will end up updating the config file whenever your applications are changed and then restart haproxy container.

Kontena Load Balancer

To make it really easy for users Kontena Load Balancer automates all these nasty parts and all the user needs to do is add some load balancer configurations to service description. Kontena will take care of the rest.

The Kontena Load Balancer provides fully automated configuration with zero downtime when load balancer configuration changes. It supports very dynamic routing rules for virtual host and path based routing. Also both HTTP and TCP traffics are supported. Naturally SSL certificates are supported out of the box.

Behind the scenes Kontena Load Balancer is a HAproxy / confd service that is configured to watch changes in etcd. Load Balancers may be described in kontena.yml and services are connected automatically by linking services to these load balancer services. If the load balanced service is scaled/re-deployed then the load balancer will reload it's configuration on the fly without dropping connections.

Getting Started

Let's see how Kontena Load Balancer works by creating a simple web application sitting behind the load balancer. For this purpose, we can use off-the-shelf Nginx web server image. First, we need to describe our application service web in kontena.yml file and link to it a load balancer service. We can also add some load balancer options as environment variables. Then we just add our load balancer service internet_lb description. See the example below:

web:  
  image: nginx:latest
  environment:
    - KONTENA_LB_INTERNAL_PORT=80
  links:
    - internet_lb

internet_lb:  
  image: kontena/lb:latest
  ports:
    - 80:80

Now we can start our application with $ kontena app deploy command. Once the application is deployed and running, the load balancer internet_lb is listening to port 80 and forwarding traffic to our web service. Very easy!

Scaling

In the example above, we created a simple web application behind load balancer. If you want to scale the web application, you can simply use kontena app scale <SERVICE_NAME> <NUMBER> command. So, for example, to scale the web service up to 3 instances, you cloud just call:

$ kontena app scale web 3

Kontena will automatically update load balancer configuration without any downtime and route traffic to all instances of this service. Alternatively, you can define the number of instances for service in the kontena.yml file.

Routing

In our previous example, we had single web application with a load balancer distributing traffic evenly to all instances of this application. Application could be scaled up and down without any downtime or troublesome configuration file updates.

When dealing with complex micro service architectures, it is also necessary to distribute traffic and route between different services. Kontena's load balancer is fully equipped to deal with micro service architectures and you can use single load balancer to serve multiple services. It will route traffic to services based on routing rules that are defined for each service. When routing rules change, load balancer will take them into account automatically.

Let's look at the following example application separated into multiple services:

There are two front-end applications that are reachable from Internet via Internet load balancers and backend applications that are reachable only via internal load balancers.

With Kontena, this kind of application can be easily defined in single kontena.yml file.

internet-lb:  
  image: kontena/lb:latest
  instances: 2
  ports:
    - 80:80
    - 443:443
www:  
  image: example/www:latest
  instances: 3
  environment:
    - KONTENA_LB_INTERNAL_PORT=8080
    - KONTENA_LB_VIRTUAL_HOSTS=www.example.com,example.com
    - KONTENA_LB_VIRTUAL_PATH=/
  links:
    - internet-lb
dashboard:  
  image: example/dashboard:latest
  instances: 3
  environment:
    - KONTENA_LB_INTERNAL_PORT=8080
    - KONTENA_LB_VIRTUAL_HOSTS=dashboard.example.com
  links:
    - internet-lb

internal-lb:  
  image: kontena/lb:latest
  instances: 2

users:  
  image: example/users:latest
  instances: 2
  environment:
    - KONTENA_LB_INTERNAL_PORT=8080
    - KONTENA_LB_VIRTUAL_PATH=/users
  links:
    - internal-lb

mailer:  
  image: example/mailer:latest
  instances: 2
  environment:
    - KONTENA_LB_INTERNAL_PORT=5050
    - KONTENA_LB_VIRTUAL_PATH=/mailer
  links:
    - internal-lb

pusher:  
  image: example/pusher:latest
  instances: 2
  environment:
    - KONTENA_LB_INTERNAL_PORT=5050
    - KONTENA_LB_VIRTUAL_PATH=/pusher
  links:
    - internal-lb

Now all the services are described and each service links to correct load balancer and describes its own routing rules. Kontena will take care the rest. The whole system can be deployed with just single command:

$ kontena app deploy

Once deployed and running, all the parts of this micro service architecture can be scaled with kontena app scale command.

Load Balancer Configuration

Kontena has wide range of load balancing options to configure. By combining these options you can achieve very complex and flexible load balancer. Configuring settings is done by passing environment variables to balanced services. Here are listed the environment variables you can use:

  • KONTENA_LB_MODE: mode of load balancing, possible values: http (default), tcp
  • KONTENA_LB_BALANCE: load balancing algorithm to use, possible values are: roundrobin (default), source, leastcon
  • KONTENA_LB_INTERNAL_PORT: service port that is attached to load balancer
  • KONTENA_LB_EXTERNAL_PORT: service port that load balancer starts to listen (only for tcp mode)
  • KONTENA_LB_VIRTUAL_HOSTS: comma separated list of virtual hosts (only for http mode)
  • KONTENA_LB_VIRTUAL_PATH: path that is used to match request, example "/api" (only for http mode)
  • KONTENA_LB_CUSTOM_SETTINGS: extra settings, each line will be appended to either related backend section or listen session in the HAProxy configuration file

SSL Support

Adding a SSL certificate to a service is done again by setting it as environment variable to the balanced service.

  • SSL_CERTS: one or more ssl certificates that are used to terminate ssl connections, first certificate is used as default

Future

This is just beginning. We are looking to add features for weighted routing (A/B testing), ACL rules, secure secrets storage (for SSL certificates among others) and filtering in the months to come.

Let us know what you think and/or help us by contributing to Kontena open source project. Looking forward to see you again!

About Kontena

Kontena is a new open source Docker platform including orchestration, service discovery, overlay networking and all the tools required to run your containerized workloads. Kontena is built to maximize developer happiness. It works on any cloud, it's easy to setup and super simple to use. Give it a try! If you like it, please star it on Github and follow us on Twitter. We hope to see you again!