Load Balancing Strategies Explained

Load balancing distributes traffic across multiple servers. Here’s how different strategies work and when to use them.

Why Load Balance?

• Scalability: Handle more traffic
• Availability: Survive server failures
• Performance: Route to fastest server

Algorithms

Round Robin

Distribute requests sequentially:

Request 1 → Server A
Request 2 → Server B
Request 3 → Server C
Request 4 → Server A (cycle repeats)

1
2
3
4
5

upstream backend {
    server 192.168.1.1;
    server 192.168.1.2;
    server 192.168.1.3;
}

Best for: Homogeneous servers, stateless applications.

Weighted Round Robin

Account for server capacity:

1
2
3
4
5

upstream backend {
    server 192.168.1.1 weight=5;  # 50% of traffic
    server 192.168.1.2 weight=3;  # 30% of traffic
    server 192.168.1.3 weight=2;  # 20% of traffic
}

Best for: Servers with different capacities.

Least Connections

Route to server with fewest active connections:

1
2
3
4
5
6

upstream backend {
    least_conn;
    server 192.168.1.1;
    server 192.168.1.2;
    server 192.168.1.3;
}

Best for: Long-lived connections, varying request complexity.

IP Hash

Same client always hits same server:

1
2
3
4
5

upstream backend {
    ip_hash;
    server 192.168.1.1;
    server 192.168.1.2;
}

Best for: Session affinity without external session storage.

Least Response Time

Route to fastest responding server:

1
2
3
4
5

upstream backend {
    least_time header;  # Based on response header time
    server 192.168.1.1;
    server 192.168.1.2;
}

Best for: Performance-critical applications.

Health Checks

Don’t send traffic to dead servers:

1
2
3
4

upstream backend {
    server 192.168.1.1 max_fails=3 fail_timeout=30s;
    server 192.168.1.2 max_fails=3 fail_timeout=30s;
}

Active health checks (Nginx Plus/HAProxy):

1
2
3
4
5
6
7

upstream backend {
    zone backend 64k;
    server 192.168.1.1;
    server 192.168.1.2;

    health_check interval=5s passes=2 fails=3;
}

Layer 4 vs Layer 7

Layer 4 (Transport)

Routes based on IP and port:

Client → L4 LB → Server (TCP connection passed through)

• Faster (no packet inspection)
• Protocol agnostic
• Less intelligent routing

Layer 7 (Application)

Routes based on HTTP content:

Client → L7 LB → Server (HTTP request inspected)

1
2
3
4
5
6
7

location /api {
    proxy_pass http://api_servers;
}

location /static {
    proxy_pass http://static_servers;
}

• Content-based routing
• SSL termination
• Request modification

SSL Termination

Offload encryption to load balancer:

1
2
3
4
5
6
7
8
9

server {
    listen 443 ssl;
    ssl_certificate /etc/ssl/cert.pem;
    ssl_certificate_key /etc/ssl/key.pem;

    location / {
        proxy_pass http://backend;  # Plain HTTP to backends
    }
}

Common Setups

Nginx

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

http {
    upstream app {
        least_conn;
        server app1:8080;
        server app2:8080;
        server app3:8080;
    }

    server {
        listen 80;

        location / {
            proxy_pass http://app;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
        }
    }
}

HAProxy

frontend http_front
    bind *:80
    default_backend http_back

backend http_back
    balance roundrobin
    option httpchk GET /health
    server app1 192.168.1.1:8080 check
    server app2 192.168.1.2:8080 check

Cloud Load Balancers

• AWS: ALB (Layer 7), NLB (Layer 4)
• GCP: HTTP(S) LB, TCP/UDP LB
• Azure: Application Gateway, Load Balancer

Choose based on your traffic patterns and requirements.