Load Balancer

Load balancers distribute client requests across multiple servers to optimize resource utilization and prevent overload.

Why Load Balancing?

Availability - If one server fails, traffic goes to others
Scalability - Handle more traffic by adding servers
Performance - Distribute load to prevent bottlenecks
Reliability - Redundancy ensures uptime
Fairness - Prevent any single server from being overwhelmed

Load Balancing Algorithms

Round-Robin

Distribute requests equally across all servers

Request 1 → Server A
Request 2 → Server B
Request 3 → Server C
Request 4 → Server A (cycle repeats)

Pros: Simple, fair Cons: Doesn't account for server capacity

Least Connections

Send request to server with fewest active connections

Server A: 2 connections
Server B: 5 connections
Server C: 1 connection

Next request → Server C

Pros: Accounts for current load Cons: More complex calculations

IP Hash

Use client IP address to determine server

Hash(client_ip) % number_of_servers = server_index

Pros: Sticky sessions (same client always goes to same server) Cons: Not truly random if hash changes

Weighted Round-Robin

Assign capacity weights to servers

Server A: weight 3
Server B: weight 1
Server C: weight 1

Total weight = 5
Server A gets 60% of requests
Servers B, C each get 20%

Pros: Accommodate different server capacities Cons: Manual configuration needed

Least Response Time

Send request to server with fastest response time

Server A: avg response 100ms
Server B: avg response 500ms
Server C: avg response 150ms

Next request → Server A

Pros: Optimizes user experience Cons: Requires response time tracking

Random

Select server randomly

Next request → random server

Pros: Simple, good for many identical servers Cons: May not be optimal

Health Checks

# Nginx upstream health check
upstream backend {
    server backend1.com max_fails=3 fail_timeout=30s;
    server backend2.com max_fails=3 fail_timeout=30s;
    server backend3.com backup; # backup only used if others fail
}

# HAProxy health check
backend app_servers
    server server1 10.0.0.1:8000 check inter 2000 rise 2 fall 5
    server server2 10.0.0.2:8000 check inter 2000 rise 2 fall 5

Health check parameters:

max_fails: Number of failed attempts before marking down
fail_timeout: How long to wait before retrying
check: Enable health checks
inter: Interval between checks (milliseconds)
rise: Successful checks before marking up
fall: Failed checks before marking down

Nginx Load Balancing

Basic Setup

upstream backend {
    server backend1.com:8000;
    server backend2.com:8000;
    server backend3.com:8000;
}

server {
    listen 80;
    server_name example.com;

    location / {
        proxy_pass http://backend;
    }
}

With Health Checks

upstream backend {
    server backend1.com:8000 max_fails=2 fail_timeout=30s;
    server backend2.com:8000 max_fails=2 fail_timeout=30s;
    server backend3.com:8000 backup;
}

server {
    listen 80;
    server_name example.com;

    location / {
        proxy_pass http://backend;
        proxy_connect_timeout 5s;
        proxy_read_timeout 10s;
    }
}

Different Algorithms

# Least connections
upstream backend {
    least_conn;
    server backend1.com:8000;
    server backend2.com:8000;
}

# IP hash
upstream backend {
    ip_hash;
    server backend1.com:8000;
    server backend2.com:8000;
}

# Weighted
upstream backend {
    server backend1.com:8000 weight=5;
    server backend2.com:8000 weight=3;
    server backend3.com:8000 weight=1;
}

# Least response time (Plus version)
upstream backend {
    least_time last_byte;
    server backend1.com:8000;
    server backend2.com:8000;
}

HAProxy

Dedicated load balancer software

Installation

sudo apt install haproxy
sudo systemctl start haproxy

Basic Configuration

# /etc/haproxy/haproxy.cfg
global
    maxconn 4000

defaults
    mode http
    timeout connect 5000
    timeout client 50000
    timeout server 50000

frontend web_in
    bind *:80
    default_backend web_servers

backend web_servers
    balance roundrobin
    server server1 10.0.0.1:8000 check
    server server2 10.0.0.2:8000 check
    server server3 10.0.0.3:8000 check

Session Persistence (Sticky Sessions)

# IP-based sticky sessions
upstream backend {
    ip_hash; # same client IP always goes to same server
    server backend1.com:8000;
    server backend2.com:8000;
}

# Cookie-based sticky sessions
upstream backend {
    server backend1.com:8000;
    server backend2.com:8000;
}

server {
    location / {
        proxy_pass http://backend;
        proxy_cookie_path / "/";
        proxy_cookie_flags ~ secure httponly samesite=lax;
    }
}

Monitoring Load Balancer

# Check upstream status
nginx -T # test config
sudo systemctl reload nginx

# Monitor requests
tail -f /var/log/nginx/access.log

# Check active connections
netstat -an | grep ESTABLISHED | wc -l

# HAProxy stats
# Access at http://localhost:8404/stats (if configured)

Failure Scenarios

# Handle backend failures
upstream backend {
    server backend1.com:8000 max_fails=3 fail_timeout=30s;
    server backend2.com:8000 max_fails=3 fail_timeout=30s;
    server backend3.com:8000 backup; # only used if others fail
}

server {
    location / {
        proxy_pass http://backend;
        proxy_next_upstream error timeout invalid_header http_500 http_502 http_503;
        proxy_next_upstream_tries 3;
    }
}

Best Practices

Use appropriate algorithm for your use case
Implement health checks
Monitor backend server performance
Configure session persistence if needed
Use backup servers for redundancy
Set appropriate timeouts
Implement logging and monitoring
Test failover scenarios
Document load balancing strategy
Consider multiple layers of load balancing

Load Balancer

On this page