LUNAROPS · OPERATIONAL UPLINK 100% UPTIME 1,247d POSTS 893 JEFF.MOON@LUNAROPS.DEV UTC --:--:--

Feature Flags: Safe Deployments, Dark Launches, and Rolling Rollouts

feature-flagsdevopsdeploymentci-cdunleashlaunchdarklyprogressive-deliverytesting

Every deployment is a risk event. Something that worked in staging breaks in production. A new checkout flow confuses users. A database query that ran fast on 1,000 rows crawls on 10 million. The classic response is to slow down deployments — review more, test more, deploy less often. Feature flags take the opposite approach: deploy constantly, but control what users actually see through configuration rather than code.

This post covers the full lifecycle of feature flags, from a minimal PostgreSQL toggle table to production-grade platforms like Unleash and LaunchDarkly, with working code examples in Python, Go, and TypeScript.


What Are Feature Flags and Why Do They Matter?

A feature flag (also called a feature toggle, feature switch, or feature gate — they all mean the same thing) is a conditional in your code that gates access to a feature based on runtime configuration rather than compiled behavior.

The core idea is simple:

1
2
3
4
if feature_flags.is_enabled("new-checkout-flow"):
    return new_checkout()
else:
    return legacy_checkout()

That conditional is evaluated at runtime against a configuration store. Flip the configuration, and every user hitting that code path gets the new behavior — no deployment required.

The Deployment Risk Problem

The traditional mental model is: deploy = release. Code lands in production, users get it. This creates enormous incentive to batch changes together (“let’s ship these 10 things at once”) and to slow down deployments with heavyweight review processes.

Feature flags break this coupling:

  • Deploy means: land code in production. The new code path exists but runs for nobody.
  • Release means: turn the flag on. Users start seeing the feature.

These two events can be separated by minutes, hours, or weeks. The code sits dark in production until you decide to light it up.

Use Cases

Kill switch — The highest-value use case. Something is broken in production. With a kill switch, you flip the flag off and the feature is gone in seconds — no incident war room, no emergency rollback, no 3am deploy. The blast radius of any bug is bounded by how quickly you can flip a flag.

Canary release — Enable a feature for 1% of users, watch your error rate and latency dashboards, increase to 5%, 25%, 50%, then 100%. If anything looks wrong, drop it back to 0% immediately. This is how every serious internet company ships.

A/B testing — Show version A to 50% of users and version B to the other 50%. Measure conversion rate, click-through, revenue. Let data drive product decisions instead of opinions.

Dark launch — Run the new code path in production alongside the old one, but only return results from the old path to users. The new code runs silently, its output compared against the old code in the background. You get production load testing and result validation before any user sees the new behavior.

Beta programs — Enable a feature for specific user accounts before general availability. Your design partners, internal employees, and power users get early access; everyone else sees nothing different.

Ops flags — Tune system behavior without a deploy. Rate limits, cache TTLs, connection pool sizes, timeout thresholds — these are operational parameters that have no business being baked into compiled binaries.

The Trunk-Based Development Connection

Feature flags are what make trunk-based development practical at scale. Without them, incomplete features have to live on long-running branches, causing merge conflicts and integration pain. With flags, you merge incomplete work to main every day — the new code ships to production with the flag off, and the flag goes on only when the feature is complete and you’re ready.

Facebook has shipped features to 1% of users first for well over a decade. GitHub used flags extensively to ship GitHub Actions, gradually expanding access from private beta to GA over months — all from the same codebase.


Types of Feature Flags

Not all flags are the same. Mixing these categories in your head leads to flags that never get cleaned up.

Release Flags (Temporary)

Release flags hide incomplete or not-yet-released features during development. They exist to enable continuous integration of partial work. They are always temporary — once the feature is fully released to 100% of users, the flag should be removed along with the old code path.

Typical lifespan: 1-8 weeks. If a release flag is still around after three months, something has gone wrong.

Experiment Flags (Temporary)

Experiment flags drive A/B tests and multivariate experiments. They are temporary because once the experiment concludes and a winner is chosen, the losing variant is deleted and the flag goes away.

Statistical significance matters here. An experiment flag that gets removed after one day of data is not producing useful signal. Most teams need at least a week of data per variant, accounting for day-of-week variation, before drawing conclusions.

Ops Flags (Permanent)

Ops flags control operational behavior — they are your circuit breakers, your emergency throttles, your configuration surface. A flag that sets the maximum number of concurrent requests to an upstream service is an ops flag. It is expected to live forever as part of the system’s operational interface.

These flags should be documented alongside your runbooks. When an incident happens at 2am, the on-call engineer needs to know: “flip this flag to shed load.”

Permission Flags (Permanent)

Permission flags enable features for specific user segments: premium subscribers, internal employees, beta participants, specific enterprise accounts. They are permanent because the segmentation logic is permanent.

The Flag Debt Problem

Here is the danger: flags accumulate. Teams create them, forget to remove them, and six months later nobody knows what’s on or off or why. This is called flag debt, and it is genuinely hazardous.

The most cited example is Knight Capital Group in 2012. A deployment flag controlling which code path the trading system used was accidentally re-enabled on a production server. The system executed orders at high speed using logic that should have been long dead — $440 million in losses in 45 minutes. A leftover deployment flag, improperly managed.

The fix is simple but requires discipline: set a removal date when you create a release or experiment flag. Put it in the flag description. Schedule a calendar reminder. Treat flag removal as part of the feature’s definition of done.


A Simple Toggle Table: Rolling Your Own

For small teams, simple use cases, or environments where you can’t send feature evaluation data to a third-party SaaS, a database table and a few dozen lines of code are often all you need.

PostgreSQL Schema

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
CREATE TABLE feature_flags (
  name        TEXT PRIMARY KEY,
  enabled     BOOLEAN NOT NULL DEFAULT false,
  description TEXT,
  rollout_pct SMALLINT DEFAULT 100 CHECK (rollout_pct BETWEEN 0 AND 100),
  updated_at  TIMESTAMPTZ DEFAULT now()
);

-- Create a trigger to auto-update updated_at
CREATE OR REPLACE FUNCTION update_updated_at()
RETURNS TRIGGER AS $$
BEGIN
  NEW.updated_at = now();
  RETURN NEW;
END;
$$ LANGUAGE plpgsql;

CREATE TRIGGER feature_flags_updated_at
  BEFORE UPDATE ON feature_flags
  FOR EACH ROW EXECUTE FUNCTION update_updated_at();

-- Seed some flags
INSERT INTO feature_flags (name, enabled, description, rollout_pct) VALUES
  ('new-checkout-flow',     false, 'Redesigned checkout — ticket ENG-4421', 100),
  ('recommendation-engine', false, 'ML-based recs — remove by 2026-04-30',  10),
  ('dark-mode',             true,  'UI dark mode toggle — ops flag',        100);

The rollout_pct column enables percentage-based rollouts: when set to 10, only 10% of users see the feature. The key trick is making that 10% deterministic — the same user must always land in the same bucket, otherwise users experience flickering behavior.

Python Implementation with Redis Caching

Hitting the database on every flag check is too slow for high-throughput services. Cache flag state in Redis with a short TTL.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
import hashlib
import json
import redis
import psycopg2
from typing import Optional
from functools import lru_cache

r = redis.Redis(host="localhost", port=6379, decode_responses=True)
conn = psycopg2.connect("postgresql://app:secret@localhost/appdb")

FLAG_CACHE_TTL = 60  # seconds


def _fetch_flag_from_db(flag_name: str) -> Optional[dict]:
    with conn.cursor() as cur:
        cur.execute(
            "SELECT enabled, rollout_pct FROM feature_flags WHERE name = %s",
            (flag_name,)
        )
        row = cur.fetchone()
        if row is None:
            return None
        return {"enabled": row[0], "rollout_pct": row[1]}


def _get_flag(flag_name: str) -> Optional[dict]:
    cache_key = f"flag:{flag_name}"
    cached = r.get(cache_key)

    if cached is not None:
        return json.loads(cached)

    flag = _fetch_flag_from_db(flag_name)
    if flag is None:
        # Cache negative lookups too, briefly
        r.setex(cache_key, 10, json.dumps(None))
        return None

    r.setex(cache_key, FLAG_CACHE_TTL, json.dumps(flag))
    return flag


def _user_bucket(flag_name: str, user_id: str) -> int:
    """Consistent hashing: same flag + same user always lands in the same bucket (0-99)."""
    key = f"{flag_name}:{user_id}".encode()
    return int(hashlib.sha256(key).hexdigest(), 16) % 100


def is_enabled(flag_name: str, user_id: Optional[str] = None) -> bool:
    """
    Check whether a feature flag is enabled.

    If user_id is provided and the flag has a rollout_pct < 100, consistent
    hashing ensures each user gets a stable result — no flickering.
    """
    flag = _get_flag(flag_name)

    if flag is None:
        return False  # Safe default: unknown flags are off

    if not flag["enabled"]:
        return False

    pct = flag["rollout_pct"]
    if pct >= 100:
        return True
    if pct <= 0:
        return False

    if user_id is None:
        # No user context: treat as a global flag at the percentage threshold
        # This is non-deterministic — prefer always passing user_id
        return False

    bucket = _user_bucket(flag_name, user_id)
    return bucket < pct


def invalidate_flag(flag_name: str) -> None:
    """Call this from a webhook or admin action when a flag changes."""
    r.delete(f"flag:{flag_name}")

The _user_bucket function is the critical piece. By hashing the flag name and user ID together with SHA-256 and taking the result modulo 100, we get a stable integer in [0, 99] for each (flag, user) pair. Set rollout_pct to 10, and the 10% of users whose bucket falls in [0, 9] always see the flag enabled — consistently, across requests, across sessions, across restarts.

Go Implementation

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
package flags

import (
    "context"
    "crypto/sha256"
    "encoding/binary"
    "encoding/json"
    "fmt"
    "log"
    "time"

    "github.com/go-redis/redis/v9"
    "github.com/jackc/pgx/v5/pgxpool"
)

type Flag struct {
    Enabled    bool `json:"enabled"`
    RolloutPct int  `json:"rollout_pct"`
}

type Client struct {
    db    *pgxpool.Pool
    cache *redis.Client
    ttl   time.Duration
}

func NewClient(db *pgxpool.Pool, cache *redis.Client) *Client {
    return &Client{db: db, cache: cache, ttl: 60 * time.Second}
}

func (c *Client) getFlag(ctx context.Context, name string) (*Flag, error) {
    key := fmt.Sprintf("flag:%s", name)

    cached, err := c.cache.Get(ctx, key).Result()
    if err == nil {
        var flag Flag
        if err := json.Unmarshal([]byte(cached), &flag); err == nil {
            return &flag, nil
        }
    }

    var flag Flag
    err = c.db.QueryRow(ctx,
        `SELECT enabled, rollout_pct FROM feature_flags WHERE name = $1`,
        name,
    ).Scan(&flag.Enabled, &flag.RolloutPct)
    if err != nil {
        return nil, err // flag not found
    }

    data, _ := json.Marshal(flag)
    c.cache.SetEx(ctx, key, string(data), c.ttl)

    return &flag, nil
}

// userBucket returns a stable integer in [0, 99] for a (flagName, userID) pair.
func userBucket(flagName, userID string) int {
    h := sha256.New()
    h.Write([]byte(fmt.Sprintf("%s:%s", flagName, userID)))
    sum := h.Sum(nil)
    n := binary.BigEndian.Uint64(sum[:8])
    return int(n % 100)
}

// IsEnabled returns true if the named flag is active for the given user.
// Pass an empty userID to check whether the flag is globally enabled.
func (c *Client) IsEnabled(ctx context.Context, flagName, userID string) bool {
    flag, err := c.getFlag(ctx, flagName)
    if err != nil {
        log.Printf("feature flag %q lookup failed: %v", flagName, err)
        return false // safe default
    }

    if !flag.Enabled {
        return false
    }
    if flag.RolloutPct >= 100 {
        return true
    }
    if flag.RolloutPct <= 0 {
        return false
    }
    if userID == "" {
        return false
    }

    return userBucket(flagName, userID) < flag.RolloutPct
}

TypeScript/Node Implementation

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
import { createClient } from "redis";
import { Pool } from "pg";
import { createHash } from "crypto";

const pool = new Pool({ connectionString: process.env.DATABASE_URL });
const redis = createClient({ url: process.env.REDIS_URL });
await redis.connect();

const FLAG_TTL_SECONDS = 60;

interface FlagRecord {
  enabled: boolean;
  rollout_pct: number;
}

async function getFlag(flagName: string): Promise<FlagRecord | null> {
  const cacheKey = `flag:${flagName}`;
  const cached = await redis.get(cacheKey);

  if (cached !== null) {
    return JSON.parse(cached) as FlagRecord;
  }

  const result = await pool.query<FlagRecord>(
    "SELECT enabled, rollout_pct FROM feature_flags WHERE name = $1",
    [flagName]
  );

  if (result.rows.length === 0) {
    await redis.setEx(cacheKey, 10, JSON.stringify(null));
    return null;
  }

  const flag = result.rows[0];
  await redis.setEx(cacheKey, FLAG_TTL_SECONDS, JSON.stringify(flag));
  return flag;
}

function userBucket(flagName: string, userId: string): number {
  const hash = createHash("sha256")
    .update(`${flagName}:${userId}`)
    .digest("hex");
  // Take first 8 bytes as a big-endian uint64 and mod 100
  const n = BigInt(`0x${hash.slice(0, 16)}`);
  return Number(n % 100n);
}

export async function isEnabled(
  flagName: string,
  userId?: string
): Promise<boolean> {
  const flag = await getFlag(flagName);

  if (!flag) return false;
  if (!flag.enabled) return false;
  if (flag.rollout_pct >= 100) return true;
  if (flag.rollout_pct <= 0) return false;
  if (!userId) return false;

  return userBucket(flagName, userId) < flag.rollout_pct;
}

When Simple Is Enough

A toggle table works well when:

  • You have fewer than ~50 active flags
  • You don’t need per-flag audit logs or a management UI
  • Your team is small enough that everyone knows what flags exist
  • You have privacy constraints that prevent sending evaluation data to third-party SaaS

When you outgrow it: no audit log, no activation strategies, no SDK ecosystem, no metrics on how often each flag is evaluated. That is when you reach for Unleash or LaunchDarkly.


Unleash: The Open-Source Feature Flag Platform

Unleash is a mature, self-hostable open-source feature flag service. It has been in production since 2014, has SDKs for every major language and framework, and provides everything a toggle table lacks: a management UI, audit logging, activation strategies, and impression metrics.

Self-Hosting with Docker Compose

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
services:
  unleash:
    image: unleashorg/unleash-server:latest
    ports:
      - "4242:4242"
    environment:
      DATABASE_URL: postgres://unleash:unleash@db/unleash
      INIT_FRONTEND_API_TOKENS: "default:development.unleash-insecure-frontend-api-token"
    depends_on:
      db:
        condition: service_healthy
    restart: unless-stopped

  db:
    image: postgres:16
    environment:
      POSTGRES_USER: unleash
      POSTGRES_PASSWORD: unleash
      POSTGRES_DB: unleash
    volumes:
      - unleash_db:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U unleash"]
      interval: 5s
      timeout: 5s
      retries: 5
    restart: unless-stopped

volumes:
  unleash_db:

Run docker compose up -d and the Unleash UI is available at http://localhost:4242. Default credentials: admin / unleash4all.

Unleash uses a polling architecture: SDK clients periodically fetch the current flag configuration (every 15 seconds by default) and cache it locally. Flag evaluation happens entirely in process — no network request per evaluation. This means sub-millisecond flag checks and resilience to Unleash server outages.

Activation Strategies

Activation strategies are Unleash’s killer feature. Each flag can have multiple strategies, any of which can enable the flag for a given request context.

Default (on/off) — Simple boolean. Enable for everyone or nobody.

UserIDs — Provide a comma-separated list of user IDs. The flag is enabled only for those exact users. Perfect for internal dogfooding.

Gradual Rollout — Percentage-based with consistent hashing. Set to 10% and exactly 10% of your users see the feature. Increment it over time as confidence grows. Unleash hashes by userId, sessionId, or random — choose userId for persistent per-user assignment.

IPs — Enable for requests from specific IP addresses or CIDR ranges. Useful for office-only features or staging environments.

Custom Strategies — Define your own context fields and evaluation logic. Examples: “enable for accounts on the Enterprise plan”, “enable in the EU region”, “enable for accounts created before 2025-01-01”.

The Unleash Context

Every flag evaluation receives a context object. The context is what activation strategies use to make decisions:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
{
  "userId": "user-12345",
  "sessionId": "session-abc",
  "remoteAddress": "203.0.113.42",
  "properties": {
    "accountId": "acct-9876",
    "plan": "enterprise",
    "region": "eu-west-1"
  }
}

Populate the context with as much information as you can — you will want to target flags by plan or region later, and if that data is not in the context, you cannot use it in activation strategies without a redeploy.

SDK Examples

Python:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
from UnleashClient import UnleashClient

client = UnleashClient(
    url="http://localhost:4242/api",
    app_name="my-service",
    custom_headers={"Authorization": "default:development.my-api-token"},
)
client.initialize_client()

context = {
    "userId": user.id,
    "properties": {
        "plan": user.plan,
        "region": user.region,
    }
}

if client.is_enabled("new-checkout-flow", context):
    return new_checkout(user)
else:
    return legacy_checkout(user)

Node.js:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
import { initialize, isEnabled } from "unleash-client";

const unleash = initialize({
  url: "http://localhost:4242/api",
  appName: "my-service",
  customHeaders: { Authorization: "default:development.my-api-token" },
});

await new Promise<void>((resolve) => unleash.on("synchronized", resolve));

const context = {
  userId: user.id,
  properties: { plan: user.plan, region: user.region },
};

if (isEnabled("new-checkout-flow", context)) {
  return newCheckout(user);
} else {
  return legacyCheckout(user);
}

Go:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
import (
    "github.com/Unleash/unleash-client-go/v3"
    unleashContext "github.com/Unleash/unleash-client-go/v3/context"
)

err := unleash.Initialize(
    unleash.WithUrl("http://localhost:4242/api"),
    unleash.WithAppName("my-service"),
    unleash.WithCustomHeaders(http.Header{
        "Authorization": {"default:development.my-api-token"},
    }),
)

ctx := &unleashContext.Context{
    UserId: user.ID,
    Properties: map[string]string{
        "plan":   user.Plan,
        "region": user.Region,
    },
}

if unleash.IsEnabled("new-checkout-flow", unleash.WithContext(ctx)) {
    return newCheckout(user)
}
return legacyCheckout(user)

Feature Flag Variants

Unleash supports variants, which extend flags from boolean to multi-value. Instead of just on/off, a flag can return different payloads per variant — the foundation of proper A/B testing.

1
2
3
4
5
6
variant = client.get_variant("checkout-button-color", context)

if variant["enabled"]:
    color = variant["payload"]["value"]  # "blue", "green", or "red"
else:
    color = "blue"  # default

Variants have configurable weights (e.g., 33%/33%/34% for three variants) and support payload types: strings, JSON, CSV, or numbers. Unleash tracks impression data per variant — how many times each was served — giving you the raw numbers for your experiment analysis.


LaunchDarkly: The Enterprise Standard

LaunchDarkly is the hosted feature flag platform and market leader. Where Unleash is self-hosted open source, LaunchDarkly is managed SaaS with a commercial model.

Pricing starts at approximately $10/seat/month, with a free tier supporting up to 1,000 Monthly Active Users. For teams that do not want to operate their own flag infrastructure — and can accept sending flag evaluation context to a third party — LaunchDarkly removes all operational overhead.

What Makes LaunchDarkly Different

Targeting rules — LaunchDarkly’s rule builder is the most powerful in the market. You can write rules like: “enable for users where plan = 'enterprise' AND country IN ['US', 'CA', 'GB']” with AND/OR logic, string contains/starts with, numeric comparisons, and semantic versioning comparisons. No custom strategy code required.

Experiments — Built-in A/B testing with statistical significance tracking. LaunchDarkly calculates p-values and confidence intervals from impression and conversion data, flagging when results reach significance.

Data Export — Stream all flag evaluation events to your data warehouse (Snowflake, BigQuery, Redshift) for custom analysis.

Integrations — Native integrations with Jira (link flags to tickets), Slack (flag change notifications), Datadog (flag change events overlaid on metrics), PagerDuty, and dozens more.

SDK streaming — LaunchDarkly SDKs maintain a persistent SSE connection to the LaunchDarkly streaming service. Flag changes propagate in milliseconds rather than polling intervals.

Python SDK Setup

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
import ldclient
from ldclient.config import Config

ldclient.set_config(Config("sdk-key-your-key-here"))
client = ldclient.get()

# Wait for the SDK to initialize (downloads all flag rules locally)
if not client.is_initialized():
    raise RuntimeError("LaunchDarkly SDK failed to initialize")

# Build a context — richer context enables richer targeting rules
context = ldclient.Context.builder(user_id) \
    .set("email", user.email) \
    .set("plan", user.plan) \
    .set("country", user.country) \
    .set("accountId", user.account_id) \
    .build()

# Flag evaluation is synchronous and in-process — no network call
if client.variation("new-checkout-flow", context, False):
    return new_checkout(user)
else:
    return legacy_checkout(user)

# Multi-variate flag: returns a string, number, JSON, or boolean
button_color = client.variation("checkout-button-color", context, "blue")

The second argument to variation is the default value — returned if the SDK has not initialized, the flag does not exist, or there is any evaluation error. This default is your safety net: always choose the safe, proven behavior.

Targeting Rules in Practice

LaunchDarkly evaluates rules top-down, returning the first matching rule’s variation. A typical production flag might look like:

  1. If email ends with @yourcompany.com → Serve true (internal dogfood)
  2. If plan is one of ["enterprise", "business"] → Serve true (paying customers first)
  3. If userId is in rollout group (15% gradual rollout) → Serve true
  4. Default rule → Serve false

This maps naturally to a real progressive rollout: internal team first, then paying customers, then gradual expansion to free users.

LaunchDarkly vs Unleash

Concern Unleash (self-hosted) LaunchDarkly (SaaS)
Cost Hosting + ops time ~$10/seat/month
Data privacy All data stays in your infra Evaluation context sent to LD
Ops overhead You maintain the service Zero
Targeting power Good (custom strategies) Excellent (visual rule builder)
Experiment analysis Basic impression counts Full statistical significance
SDK quality Good, community-maintained Excellent, first-party
Streaming updates Polling (15s default) SSE (milliseconds)

Use Unleash when: you have ops capacity, privacy requirements prevent third-party data sharing, or budget is tight. Use LaunchDarkly when: you want a fully managed service, need rich targeting rules without custom code, or are running rigorous product experiments.


Implementation Patterns in Real Applications

Where to Check Flags

Check flags at the boundary of your application — in the HTTP handler, the queue consumer, the scheduled job entry point — not deep in business logic or utility functions.

The closer a flag check is to user-facing code, the clearer it is what behavior it controls. A flag buried five layers deep in a utility function is a flag nobody understands.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
# Bad: flag checked deep in business logic
def calculate_shipping_cost(order):
    base = order.weight * 0.5
    if feature_flags.is_enabled("new-shipping-algorithm"):  # ← wrong layer
        base = new_algorithm(order)
    return base + tax(base)

# Good: flag checked at the entry point
def handle_checkout_request(request):
    user = get_user(request)
    if use_new_checkout(user.id):
        return new_checkout_handler(request)
    else:
        return legacy_checkout_handler(request)

The Flag Wrapper Pattern

Do not scatter is_enabled("flag-name-as-string") calls throughout your codebase. String literals are invisible to your IDE, refactoring tools, and static analysis. Instead, wrap flag checks behind named functions:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
# Bad: string literals everywhere
if feature_flags.is_enabled("new-checkout-v2"):
    ...
# (and in 15 other files)

# Good: one place, one name, one string
def use_new_checkout(user_id: str) -> bool:
    """Enable for new checkout flow — remove after 2026-04-30 (ENG-4421)."""
    return feature_flags.is_enabled("new-checkout-v2", user_id)

# All callers use the wrapper
if use_new_checkout(user.id):
    ...

The wrapper is searchable, documents its own removal date, and can be updated in one place if the flag name changes.

Backend vs Frontend Flags

Server-side evaluation is more secure — users cannot inspect flag rules or force themselves into a variant by manipulating client-side state. Server-rendered content reflects flag state before the HTML leaves the server. Use server-side evaluation for security-sensitive flags, pricing changes, and access control.

Client-side evaluation moves flag logic to the browser or mobile app. LaunchDarkly and Unleash both have frontend SDKs that bootstrap flag state on page load. This enables faster rendering and client-side A/B tests, but flag rules must be safe to expose publicly.

For SPAs, the classic problem is a flash of wrong content: the page renders with the old experience for a moment before the flag SDK initializes and updates the UI. The fix is to bootstrap initial flag state into the server-rendered HTML:

1
2
3
4
5
6
7
<!-- Server injects current flag state into the page -->
<script>
  window.__FEATURE_FLAGS__ = {
    "new-checkout-flow": true,
    "dark-mode": false
  };
</script>

The frontend SDK reads window.__FEATURE_FLAGS__ immediately, then subscribes to streaming updates for changes.

Testing with Flags

Never use a real flag service in unit tests. Inject a mock evaluator:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
# Production code: flag evaluator is injected
class CheckoutService:
    def __init__(self, flags: FlagEvaluator):
        self.flags = flags

    def process(self, user):
        if self.flags.is_enabled("new-checkout-flow", user.id):
            return new_checkout(user)
        return legacy_checkout(user)

# Unit test: inject a stub
def test_new_checkout_when_flag_enabled():
    flags = StubFlags({"new-checkout-flow": True})
    service = CheckoutService(flags)
    result = service.process(test_user)
    assert result == expected_new_result

def test_legacy_checkout_when_flag_disabled():
    flags = StubFlags({"new-checkout-flow": False})
    service = CheckoutService(flags)
    result = service.process(test_user)
    assert result == expected_legacy_result

Test both states of every flag. It is a common failure mode to test only the happy path (flag on) and discover the flag-off path is broken only when you try to roll back.

Avoiding Flag Spaghetti

Never check a flag inside another flag’s code branch:

1
2
3
4
5
6
# Do not do this
if use_new_checkout():
    if use_new_payment_processor():  # ← flag inside flag
        ...
    else:
        ...

Nested flags create exponential test matrices and make behavior nearly impossible to reason about. If two features always go together, they should share one flag. If they are independent, they belong at the same level of the code.


Operational Patterns: The Lifecycle of a Flag

Naming Conventions

Consistent naming makes flags searchable and self-documenting. Two common conventions:

  • feature-<team>-<description>: feature-checkout-new-payment-flow
  • release-<ticket-id>-<description>: release-eng4421-new-checkout-redesign

Prefix ops flags distinctly: ops-rate-limit-upstream-api, ops-circuit-breaker-payments.

Progressive Rollout Procedure

The standard progression for a release flag:

  1. Deploy with flag off — Code ships to production. Zero users affected. Verify the deploy is healthy.
  2. Enable for internal users — Flag on for employees. 10-50 users. Find obvious bugs.
  3. 1% rollout — Watch error rates, latency p99, and business metrics for 30-60 minutes.
  4. 5% rollout — Statistically meaningful signal. Watch for another hour.
  5. 25% rollout — At this point you are fairly confident. Verify business metrics are not degrading.
  6. 50% rollout — Roughly even split. Compare old vs new directly.
  7. 100% rollout — Everyone on the new path.
  8. Delete old code path — Remove the conditional and the legacy branch.
  9. Delete the flag — Remove from the database/platform and any SDKs.

Each step should have an explicit green light from your metrics. If anything looks wrong at any stage, drop back to the previous percentage immediately — no deploy required.

The Kill Switch Procedure

A kill switch flag is a permanent ops flag used as a circuit breaker. When you see error rates spiking:

  1. Open your feature flag management interface (bookmarked, accessible without VPN if possible)
  2. Toggle the flag off — behavior reverts to safe baseline in seconds
  3. Write a brief incident note: when you toggled, what changed, what you observed
  4. Investigate root cause with the system stable

The kill switch is only useful if you have rehearsed using it. Make sure every on-call engineer knows where flags live and how to toggle them.

Flag Cleanup

Schedule monthly flag reviews. Any release or experiment flag older than 90 days with rollout_pct < 100 is a problem: either complete the rollout or revert and delete the flag. A flag that exists at 50% for three months means half your users have a different experience indefinitely — that is flag debt turned into technical debt.

Automate this with a cron job that queries your flags table for old release flags and posts a Slack message to the owning team.

Monitoring Flag Changes

Every flag change in production should generate a notification. Unleash and LaunchDarkly both have audit logs. Wire them to your monitoring stack:

  • Datadog: LaunchDarkly’s integration adds flag changes as events overlaid on your dashboards — instantly correlate a metric spike with a flag toggle.
  • Slack: Webhook on flag changes → #ops-flags channel. The on-call engineer sees it in real time.
  • PagerDuty: For kill switch toggles, alert the team immediately.

Advanced Patterns

Dark Launching

Dark launching runs new code in production alongside old code, but only returns the old code’s results to users. The new code executes silently, its output compared for correctness and performance in the background.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
def get_user_recommendations(user_id: str) -> list[Recommendation]:
    # Always use the proven result
    result = legacy_recommendations(user_id)

    if use_new_recommendations(user_id):
        try:
            new_result = new_recommendations(user_id)
            # Compare results asynchronously — never block the response
            asyncio.create_task(
                compare_and_log(result, new_result, user_id)
            )
        except Exception as e:
            # The shadow call must never fail the real request
            log.warning("dark launch error: %s", e)

    return result


async def compare_and_log(
    old: list[Recommendation],
    new: list[Recommendation],
    user_id: str
) -> None:
    old_ids = {r.id for r in old}
    new_ids = {r.id for r in new}
    overlap = len(old_ids & new_ids) / max(len(old_ids), 1)
    metrics.histogram("recommendations.dark_launch.overlap", overlap, tags=[f"user:{user_id}"])
    if overlap < 0.5:
        log.info("low overlap for user %s: old=%s new=%s", user_id, old_ids, new_ids)

Dark launching is how you validate a new algorithm or service against production load before any user experiences it. It is particularly valuable for search ranking changes, recommendation engines, and pricing logic.

Database Migrations with Flags

Backwards-incompatible database migrations combined with feature flags enable zero-downtime schema changes:

  1. Deploy migration — Add the new column (nullable, no default required). Old code ignores it.
  2. Backfill — Populate the new column for existing rows asynchronously.
  3. Deploy code with flag — New code path reads from the new column; old path reads from old column. Flag off.
  4. Ramp up flag — Gradually shift traffic to the new schema path, verifying data integrity.
  5. 100% flag — All traffic uses the new schema.
  6. Remove old column and code — Clean up once confident.

This decomposition means a schema change that would previously require a maintenance window can be done with zero downtime and full rollback capability at each step.

Feature Flags in Microservices

When a flag gates behavior that spans multiple services, you need to propagate flag evaluation results across service boundaries. The two approaches:

Evaluate once at the edge — The API gateway or BFF evaluates all flags for a request and injects results into downstream HTTP headers or gRPC metadata: X-Feature-New-Checkout: true. Downstream services read the header rather than evaluating the flag themselves. This is simpler and ensures consistent behavior within a single request.

Each service evaluates independently — Every service has the flag SDK and evaluates flags against the same user context. This works if your flag platform has low enough latency (in-process evaluation like LaunchDarkly SDKs), but requires that each service has access to user context.

OpenFeature: The CNCF Standard

OpenFeature is a CNCF project defining a vendor-neutral interface for feature flag SDKs. Instead of writing code against LaunchDarkly’s SDK directly, you write against the OpenFeature interface — then swap the provider (LaunchDarkly, Unleash, your toggle table) without changing application code.

1
2
3
4
5
6
7
8
9
import { OpenFeature } from "@openfeature/server-sdk";
import { LaunchDarklyProvider } from "@openfeature/launchdarkly-provider";

// Configure once at startup
OpenFeature.setProvider(new LaunchDarklyProvider("sdk-key"));
const client = OpenFeature.getClient("my-service");

// Application code is provider-agnostic
const enabled = await client.getBooleanValue("new-checkout-flow", false, context);

OpenFeature is worth considering for any new greenfield project. Vendor lock-in on feature flags is real — switching from LaunchDarkly to Unleash after three years of ldclient.variation() calls is painful. OpenFeature eliminates that migration cost.


Pitfalls and Anti-Patterns

Flag explosion — Teams create flags for every change, and within a year the system has 500 active flags that nobody fully understands. Establish a flag creation process: document owner, type, and removal date. Run quarterly audits.

Testing only the happy path — You test the flag-on state thoroughly and forget to test flag-off. A production incident reveals that the legacy code path has been silently broken for weeks. Always run both paths through CI.

Long-lived release flags — A release flag that has been at 50% for four months is a liability. It means your codebase has two active production code paths with diverging test coverage. Set a calendar reminder at flag creation for 90 days: “complete rollout or revert.”

Flags in the wrong layer — Feature flags do not belong in database repositories, utility libraries, or deep domain services. They belong at the edges of your system. Putting them deep in shared code means you cannot test or reason about the behavior from the outside.

The stale flag problem — An SDK checks a flag that no longer exists in the configuration store. Always return a safe default. Never throw an error or return an ambiguous null — make the default the safe, backwards-compatible behavior:

1
2
3
4
5
def is_enabled(flag_name: str, default: bool = False) -> bool:
    flag = get_flag(flag_name)
    if flag is None:
        return default  # flag deleted? fall back to safe default
    ...

Skipping caching — Checking a database on every HTTP request for flag state will crater performance. Caching is not optional — it is the mechanism that makes the whole system fast. Use a 30-60 second TTL in Redis or an in-memory LRU cache, and invalidate on flag changes via webhooks.

No polling or streaming — If your flag service is unreachable and you have no local cache of flag state, every evaluation fails. Ensure your SDK has an in-memory snapshot of the last known state, so a brief flag service outage does not take down your application.


Summary

Feature flags are one of the highest-leverage operational tools available to a software team. The pattern is simple — a conditional backed by runtime configuration — but applied consistently it transforms how you deploy and release software.

Start with a toggle table if your needs are modest. Self-host Unleash when you need a management UI, audit log, and activation strategies without a SaaS bill. Adopt LaunchDarkly when you need enterprise targeting, statistical experiments, and fully managed infrastructure.

Regardless of which platform you choose, the discipline matters more than the tool:

  • Set removal dates when creating release and experiment flags
  • Check flags at the boundaries of your system, not deep in shared code
  • Test both states of every flag in CI
  • Monitor flag changes as operational events
  • Run progressive rollouts — 1%, 5%, 25%, 50%, 100% — with observability at each step

Deploy constantly. Release carefully. Flags give you the ability to do both.

Comments