Serverless in Production: Lambda Patterns That Hold Up

Serverless is a billing model as much as it is an architecture. AWS Lambda charges for execution time in 1-millisecond increments and for the number of invocations, with no charge when the function is idle. That model suits workloads with uneven, bursty, or low-volume traffic better than it suits anything that runs continuously at high throughput — where EC2 or containers become cheaper.

But “Lambda is cheap and scales automatically” is where most teams stop thinking, and it is where most Lambda production problems originate. Cold starts affect user-facing latency more than benchmarks suggest. The concurrency model has hard edges that cause silent request drops. API Gateway adds cost and complexity that HTTP APIs partially address. Event-driven architectures built on SQS and SNS require careful handling of partial failures and poison messages. Step Functions are the right tool for coordinating multi-step workflows, but Standard vs Express Workflow is not an aesthetic choice.

This post covers the mechanics of each, the failure modes, and the patterns that survive contact with production traffic.

The Execution Environment Lifecycle

Understanding cold starts requires understanding how Lambda manages execution environments.

When a function is invoked and no warm execution environment is available, Lambda:

1. Allocates a Firecracker microVM with the requested memory
2. Downloads and mounts the function package (ZIP or container image)
3. Starts the language runtime (Python interpreter, JVM, Node.js, etc.)
4. Runs your initialization code (everything outside the handler function)
5. Runs your handler

Steps 1–4 are the INIT phase — the cold start. Step 5 is the INVOKE phase — the billable execution. After the handler returns, the execution environment stays warm for an indeterminate period (typically 5–15 minutes of inactivity). The next invocation to hit this environment skips the INIT phase entirely.

Cold invocation timeline:
│◄──────── INIT (cold start) ─────────►│◄── INVOKE ──►│
│ microVM + runtime + init code (100ms–10s) │ your handler │
                                             ↑
                                      This is what the user waits for

Warm invocation timeline:
│◄── INVOKE ──►│
│ your handler │

As of August 2025, AWS bills for the INIT phase in addition to the INVOKE phase. For functions with heavy initialization (loading ML models, warming connection pools, importing large libraries), this shift meaningfully increases Lambda spend. Measure your INIT duration in CloudWatch with the Init Duration metric before assuming Lambda is cheap.

Concurrency and scaling

Lambda scales by creating new execution environments in parallel — one environment per concurrent invocation. The default account limit is 1,000 concurrent executions across all functions in a region. A burst of 1,000 simultaneous requests spins up 1,000 execution environments, each going through the INIT phase simultaneously.

Lambda applies a burst limit during rapid scale-up: 3,000 immediate concurrency in us-east-1 and us-west-2 (500 in other regions), then +500 per minute until the account limit is reached. A sudden traffic spike that requires 5,000 concurrent executions in us-east-1 takes at least 4 minutes to fully scale — requests during that window are throttled (429 errors).

Cold Starts: Causes and Solutions

What drives cold start duration

The most impactful optimization is almost always moving initialization outside the handler:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

# Wrong: connection pool created on every cold start inside handler
def handler(event, context):
    conn = psycopg2.connect(os.environ["DB_URL"])  # ~100ms each cold start
    result = conn.execute("SELECT ...")
    conn.close()
    return result

# Right: connection pool created once at module load; reused across warm invocations
import psycopg2
from psycopg2 import pool

# This runs during INIT, not during INVOKE
_db_pool = psycopg2.pool.SimpleConnectionPool(
    minconn=1,
    maxconn=5,
    dsn=os.environ["DB_URL"]
)

def handler(event, context):
    conn = _db_pool.getconn()
    try:
        result = conn.execute("SELECT ...")
        return result
    finally:
        _db_pool.putconn(conn)

The connection pool is created once per execution environment, not once per invocation. Warm invocations skip the connection overhead entirely.

The same pattern applies to SDK clients, loaded models, configuration files, and anything else that is expensive to initialize:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

import boto3
import json

# Initialize at module level — runs once per execution environment
_ssm = boto3.client("ssm")
_config = json.loads(
    _ssm.get_parameter(Name="/myapp/config", WithDecryption=True)["Parameter"]["Value"]
)
_s3 = boto3.client("s3")

def handler(event, context):
    # _config, _ssm, _s3 are all already initialized
    ...

SnapStart

SnapStart takes a snapshot of the initialized execution environment and restores it for subsequent invocations, eliminating the runtime startup and initialization time. Originally Java-only, it now supports Java 11+, .NET 8 with Native AOT, and Python in limited preview.

For Java functions, SnapStart reduces cold start time by 80–90%. Enable it per function version:

1
2
3
4
5

aws lambda put-function-configuration \
  --function-name my-java-function \
  --snap-start ApplyOn=PublishedVersions

aws lambda publish-version --function-name my-java-function

SnapStart has one critical constraint: your initialization code must be idempotent with respect to the snapshot/restore cycle. The restored environment has the same state as the snapshot, including any random seeds, timestamps captured at init time, and open sockets. AWS provides CacheUtils hooks for Java to regenerate non-idempotent state at restore time:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

import com.amazonaws.services.lambda.crac.Core;
import com.amazonaws.services.lambda.crac.Resource;

public class Handler implements RequestHandler<Map<String,String>, String>, Resource {
    public Handler() {
        Core.getGlobalContext().register(this);
        // Expensive initialization here
        connectToDatabase();
        loadModel();
    }

    @Override
    public void beforeCheckpoint(Context<? extends Resource> context) {
        // Called before snapshot is taken — close connections that can't survive
        db.close();
    }

    @Override
    public void afterRestore(Context<? extends Resource> context) {
        // Called after restore — re-establish connections
        connectToDatabase();
    }
}

Provisioned concurrency

Provisioned concurrency pre-initializes a specified number of execution environments and keeps them warm permanently. Invocations hitting provisioned environments have no cold start — single-digit millisecond overhead.

1
2
3
4

aws lambda put-provisioned-concurrency-config \
  --function-name my-function \
  --qualifier prod \
  --provisioned-concurrent-executions 10

The cost is significant: provisioned concurrency is billed per environment-hour regardless of whether the environments are being used. 10 provisioned concurrent executions at 1,024 MB memory in us-east-1 cost roughly $130/month — in addition to normal invocation charges. Compare that to the cold start frequency and latency impact before enabling it.

When provisioned concurrency is worth it:

• User-facing APIs where P99 latency is a product requirement
• Functions with >500ms cold start times that cannot be reduced by code optimization
• Traffic patterns with predictable peak windows (use Application Auto Scaling to schedule provisioned concurrency up before peak and down afterward)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

# Scale provisioned concurrency on a schedule
aws application-autoscaling register-scalable-target \
  --service-namespace lambda \
  --resource-id function:my-function:prod \
  --scalable-dimension lambda:function:ProvisionedConcurrency \
  --min-capacity 0 \
  --max-capacity 50

aws application-autoscaling put-scheduled-action \
  --service-namespace lambda \
  --resource-id function:my-function:prod \
  --scalable-dimension lambda:function:ProvisionedConcurrency \
  --scheduled-action-name scale-up-for-peak \
  --schedule "cron(0 8 * * ? *)" \
  --scalable-target-action MinCapacity=10,MaxCapacity=10

aws application-autoscaling put-scheduled-action \
  --service-namespace lambda \
  --resource-id function:my-function:prod \
  --scalable-dimension lambda:function:ProvisionedConcurrency \
  --scheduled-action-name scale-down-after-peak \
  --schedule "cron(0 22 * * ? *)" \
  --scalable-target-action MinCapacity=0,MaxCapacity=0

Concurrency Controls

Reserved concurrency

Reserved concurrency sets the maximum number of concurrent executions for a specific function, carved out from the account pool. It serves two purposes:

Throttle protection for downstream services: a Lambda processing database writes should not be allowed to open 1,000 simultaneous DB connections. Setting reserved concurrency to 50 caps database connections from that function at 50.

Guaranteed capacity: a reserved concurrency of 100 guarantees that 100 concurrent executions are always available for that function, regardless of what other functions are doing.

1
2
3

aws lambda put-function-concurrency \
  --function-name db-writer \
  --reserved-concurrent-executions 50

Setting reserved concurrency to 0 effectively disables the function — useful for emergency shutoff without deleting the function.

Throttling behavior

When a function hits its concurrency limit (reserved or account-wide), Lambda throttles — it returns a 429 ThrottlingException. What happens next depends on how the function was invoked:

Synchronous throttles are visible to users as errors. For synchronous invocations, the caller must handle 429s. Asynchronous invocations absorb throttles via retry — but six hours of retry delay is not “absorbed” from a user perspective; it is silent failure with delayed processing.

Event-Driven Patterns

SQS as a buffer

SQS decouples producers from Lambda consumers. The Lambda Event Source Mapping (ESM) polls SQS automatically — you do not write polling code. Lambda scales consumers up to the queue’s available messages.

1
2
3
4
5
6
7
8

import json
import boto3

def handler(event, context):
    for record in event["Records"]:
        body = json.loads(record["body"])
        process_order(body)
        # If this raises, the entire batch is returned to the queue

Batch processing and partial failures: by default, if your handler raises an exception for any record in a batch, the entire batch goes back to the queue and every record is reprocessed. This causes duplicate processing of records that succeeded before the failure.

Enable partial batch response to report individual record failures:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

def handler(event, context):
    failures = []

    for record in event["Records"]:
        try:
            body = json.loads(record["body"])
            process_order(body)
        except Exception as e:
            # Report this specific message ID as failed
            failures.append({"itemIdentifier": record["messageId"]})

    # Only failed records return to the queue; successful ones are deleted
    return {"batchItemFailures": failures}

Enable this in the ESM configuration:

1
2
3

aws lambda update-event-source-mapping \
  --uuid <esm-uuid> \
  --function-response-types ReportBatchItemFailures

Dead letter queues: messages that fail repeatedly (after the queue’s maxReceiveCount) are moved to the DLQ. Set up a DLQ for every SQS queue that drives Lambda processing:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

resource "aws_sqs_queue" "orders" {
  name                       = "orders"
  visibility_timeout_seconds = 300   # must be >= Lambda timeout
  redrive_policy = jsonencode({
    deadLetterTargetArn = aws_sqs_queue.orders_dlq.arn
    maxReceiveCount     = 5
  })
}

resource "aws_sqs_queue" "orders_dlq" {
  name                       = "orders-dlq"
  message_retention_seconds  = 1209600  # 14 days
}

Alert on DLQ depth — messages in the DLQ represent business logic failures requiring human attention.

SNS fan-out

SNS delivers one event to multiple SQS queues simultaneously. The canonical pattern for broadcasting a domain event to multiple consuming services:

Order placed
      │
      ▼
   SNS Topic
   (order-events)
      │
  ┌───┴───┐
  ▼       ▼
SQS     SQS
(billing) (inventory)
  │       │
  ▼       ▼
Lambda  Lambda

Each consuming service has its own SQS queue with its own DLQ, retry policy, and concurrency settings. A failure in the billing Lambda does not affect the inventory Lambda.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

resource "aws_sns_topic" "order_events" {
  name = "order-events"
}

resource "aws_sns_topic_subscription" "billing" {
  topic_arn = aws_sns_topic.order_events.arn
  protocol  = "sqs"
  endpoint  = aws_sqs_queue.billing.arn

  filter_policy = jsonencode({
    event_type = ["order.placed", "order.cancelled"]
  })
}

resource "aws_sns_topic_subscription" "inventory" {
  topic_arn = aws_sns_topic.order_events.arn
  protocol  = "sqs"
  endpoint  = aws_sqs_queue.inventory.arn
}

SNS subscription filter policies limit which message types each queue receives. Without filters, every queue receives every event — as event volume grows, this adds unnecessary processing and cost.

EventBridge for event routing

EventBridge is a more capable event bus than SNS for routing. It supports content-based routing via rule patterns, multiple event sources (AWS services, SaaS providers, your own applications), schema registry, and event replay. Use EventBridge when:

• Events from multiple AWS services need to trigger the same Lambda
• You need content-based routing with complex matching rules
• You want decoupled routing that can be updated without changing producers

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18

resource "aws_cloudwatch_event_rule" "high_value_orders" {
  name        = "high-value-orders"
  event_bus_name = "default"

  event_pattern = jsonencode({
    source      = ["myapp.orders"]
    detail-type = ["OrderPlaced"]
    detail = {
      amount = [{ numeric = [">", 1000] }]
    }
  })
}

resource "aws_cloudwatch_event_target" "fraud_check" {
  rule      = aws_cloudwatch_event_rule.high_value_orders.name
  target_id = "FraudCheckLambda"
  arn       = aws_lambda_function.fraud_check.arn
}

Step Functions

Step Functions orchestrates multi-step workflows where each step is a Lambda function (or any AWS service action). It handles retries, error catching, branching, parallel execution, and wait states — all defined in a state machine, not in application code.

Standard vs Express Workflows

Standard Workflows are the default. Express Workflows are cheaper at high volume and appropriate for short-lived pipelines (ETL transformations, event processing pipelines under 5 minutes).

Defining a state machine

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57

{
  "Comment": "Order processing workflow",
  "StartAt": "ValidateOrder",
  "States": {
    "ValidateOrder": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:us-east-1:123:function:validate-order",
      "Retry": [
        {
          "ErrorEquals": ["Lambda.ServiceException", "Lambda.AWSLambdaException"],
          "IntervalSeconds": 2,
          "MaxAttempts": 3,
          "BackoffRate": 2
        }
      ],
      "Catch": [
        {
          "ErrorEquals": ["ValidationError"],
          "Next": "RejectOrder",
          "ResultPath": "$.error"
        }
      ],
      "Next": "ReserveInventory"
    },
    "ReserveInventory": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:us-east-1:123:function:reserve-inventory",
      "Next": "ProcessPayment"
    },
    "ProcessPayment": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:us-east-1:123:function:process-payment",
      "Catch": [
        {
          "ErrorEquals": ["PaymentDeclinedError"],
          "Next": "ReleaseInventory"
        }
      ],
      "Next": "FulfillOrder"
    },
    "ReleaseInventory": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:us-east-1:123:function:release-inventory",
      "Next": "RejectOrder"
    },
    "FulfillOrder": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:us-east-1:123:function:fulfill-order",
      "End": true
    },
    "RejectOrder": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:us-east-1:123:function:reject-order",
      "End": true
    }
  }
}

The state machine handles the compensating transaction (ReleaseInventory when payment fails) declaratively. The Lambda functions contain only business logic — no orchestration code, no retry loops, no saga coordination.

Wait for human approval

Standard Workflows support pausing indefinitely pending an external signal:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

"WaitForApproval": {
  "Type": "Task",
  "Resource": "arn:aws:states:::lambda:invoke.waitForTaskToken",
  "Parameters": {
    "FunctionName": "send-approval-email",
    "Payload": {
      "taskToken.$": "$$.Task.Token",
      "orderId.$": "$.orderId"
    }
  },
  "TimeoutSeconds": 86400,
  "Next": "ProcessApproval"
}

The approval email Lambda receives the taskToken. When the approver clicks approve/reject, your API calls:

1
2
3
4
5
6
7
8

import boto3
sfn = boto3.client("stepfunctions")

# Approve
sfn.send_task_success(taskToken=token, output=json.dumps({"approved": True}))

# Reject
sfn.send_task_failure(taskToken=token, error="Rejected", cause="Fraud suspected")

The execution resumes from the WaitForApproval state with the response. Without Step Functions, implementing this pattern requires a database to store the token, a cron job to check status, and significant custom code.

API Gateway: REST vs HTTP vs Function URLs

REST API vs HTTP API

Most Lambda-backed APIs should use HTTP API, not REST API. HTTP API is 70% cheaper ($1.00/million vs $3.50/million requests), has lower latency, and supports all common Lambda proxy integration patterns. REST API adds value for specific features: WAF integration, request validation, usage plans, API keys, canary deployments, and x-ray integration at the API layer. If you do not need those features, HTTP API is the correct choice.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

# HTTP API (preferred for most Lambda backends)
resource "aws_apigatewayv2_api" "main" {
  name          = "my-api"
  protocol_type = "HTTP"

  cors_configuration {
    allow_headers = ["Content-Type", "Authorization"]
    allow_methods = ["GET", "POST", "PUT", "DELETE"]
    allow_origins = ["https://app.example.com"]
  }
}

resource "aws_apigatewayv2_integration" "lambda" {
  api_id             = aws_apigatewayv2_api.main.id
  integration_type   = "AWS_PROXY"
  integration_uri    = aws_lambda_function.api.invoke_arn
  payload_format_version = "2.0"
}

resource "aws_apigatewayv2_route" "default" {
  api_id    = aws_apigatewayv2_api.main.id
  route_key = "$default"
  target    = "integrations/${aws_apigatewayv2_integration.lambda.id}"
}

The hidden API Gateway costs

API Gateway’s per-request pricing is only part of the cost:

• Data transfer out: $0.09/GB. An API returning 10 KB responses at 10M requests/month transfers 100 GB — adding ~$9/month on top of request charges.
• CloudWatch Logs: if you enable full execution logging (the default in many setups), API Gateway logs every request body and response. At $0.30/GB ingested, high-volume APIs generate hundreds of GB of logs monthly. Set log level to ERROR in production and use sampled access logs instead.
• Caching: API Gateway’s built-in cache is $0.02–$0.038/hour per cache size tier. A 0.5 GB cache runs ~$14/month — often cheaper than the Lambda invocations it avoids, but easy to forget when estimating costs.

Lambda Function URLs

Function URLs provide a dedicated HTTPS endpoint for a Lambda function without API Gateway. They support IAM authentication or unauthenticated access, streaming responses, and CORS configuration:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

resource "aws_lambda_function_url" "main" {
  function_name      = aws_lambda_function.api.function_name
  qualifier          = "prod"
  authorization_type = "NONE"

  cors {
    allow_credentials = false
    allow_headers     = ["Content-Type"]
    allow_methods     = ["GET", "POST"]
    allow_origins     = ["https://app.example.com"]
    max_age           = 86400
  }
}

Function URLs have no per-request charge beyond normal Lambda invocation pricing. For simple Lambda-backed APIs without advanced API Gateway features (custom domains, throttling, usage plans), Function URLs eliminate a cost layer entirely. The trade-off: no custom domain (you get a lambda-url.us-east-1.on.aws URL unless you front it with CloudFront), no built-in caching, and no WAF integration.

Packaging: ZIP vs Layers vs Container Images

ZIP packages

The default: a ZIP containing your code and dependencies, up to 50 MB compressed (250 MB uncompressed). Fast to deploy and update.

1
2
3
4
5
6
7
8

# Python: install dependencies to a local directory, zip together
pip install -r requirements.txt -t package/
cd package && zip -r ../function.zip .
cd .. && zip function.zip lambda_function.py

aws lambda update-function-code \
  --function-name my-function \
  --zip-file fileb://function.zip

Lambda layers

Layers separate shared dependencies from function code. A layer is a ZIP deployed independently and attached to functions at runtime. Multiple functions sharing a layer means the layer counts toward each function’s 250 MB limit, but only deploys once.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

# Create a layer for shared dependencies
mkdir python
pip install psycopg2-binary boto3 -t python/
zip -r psycopg2-layer.zip python/

aws lambda publish-layer-version \
  --layer-name psycopg2 \
  --zip-file fileb://psycopg2-layer.zip \
  --compatible-runtimes python3.12

# Attach to a function
aws lambda update-function-configuration \
  --function-name my-function \
  --layers arn:aws:lambda:us-east-1:123:layer:psycopg2:1

Layers are most useful for: shared internal libraries across multiple functions, large dependencies that change less frequently than the function code (keeps function deploys fast), and binary extensions.

Container images

Container images (up to 10 GB) allow you to use any runtime, any system dependency, and any toolchain. The function code is the container’s CMD or ENTRYPOINT:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

FROM public.ecr.aws/lambda/python:3.12

# Copy requirements and install
COPY requirements.txt .
RUN pip install -r requirements.txt

# Copy function code
COPY lambda_function.py .

CMD ["lambda_function.handler"]

1
2
3
4
5
6
7
8
9

# Build and push to ECR
aws ecr get-login-password | docker login --username AWS --password-stdin 123.dkr.ecr.us-east-1.amazonaws.com
docker build -t my-function .
docker tag my-function:latest 123.dkr.ecr.us-east-1.amazonaws.com/my-function:latest
docker push 123.dkr.ecr.us-east-1.amazonaws.com/my-function:latest

aws lambda update-function-code \
  --function-name my-function \
  --image-uri 123.dkr.ecr.us-east-1.amazonaws.com/my-function:latest

Container images have longer cold starts than ZIP packages — the image must be pulled and mounted. AWS caches images at the execution environment level so warm invocations are not affected, but the initial cold start after deployment is longer. For image-heavy models (ML inference), the extra startup time is typically worth the flexibility.

Idempotency

Lambda with asynchronous invocation (S3 events, SNS, async invocation API) retries on failure. Step Functions retries Lambda invocations on transient errors. SQS delivers at-least-once. This means your Lambda handlers will be invoked multiple times for the same event in normal production operation. If your handler is not idempotent, you get duplicate side effects — duplicate charges, duplicate emails, duplicate database rows.

The practical approach: generate a deterministic idempotency key from the event, and check/set it in a fast store (DynamoDB with a TTL is the standard choice) at the start of the handler.

AWS Powertools for Lambda provides an idempotency decorator:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

from aws_lambda_powertools.utilities.idempotency import (
    idempotent,
    DynamoDBPersistenceLayer,
    IdempotencyConfig,
)

persistence_layer = DynamoDBPersistenceLayer(table_name="IdempotencyTable")
config = IdempotencyConfig(
    event_key_jmespath="body.orderId",  # derive key from this field
    expires_after_seconds=3600,
)

@idempotent(config=config, persistence_store=persistence_layer)
def handler(event, context):
    # This body runs exactly once per unique orderId, even if invoked multiple times
    charge_customer(event["body"]["orderId"])
    return {"statusCode": 200}

The DynamoDB table stores the idempotency key, the result, and a TTL. Duplicate invocations within the TTL window return the cached result without re-executing the handler body.

Memory Tuning and the CPU Relationship

Lambda does not have a separate CPU configuration. CPU is allocated proportionally to memory — 1,769 MB gets exactly 1 vCPU. Below 1,769 MB, you get a fraction of a vCPU. Above, you get more than one (up to approximately 6 vCPUs at 10,240 MB).

This means increasing memory beyond what your function needs can reduce execution time (and therefore cost) by providing more CPU. The Lambda Power Tuning open-source tool runs your function at multiple memory configurations and plots cost vs. performance:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

# Deploy Lambda Power Tuning
git clone https://github.com/alexcasalboni/aws-lambda-power-tuning
cd aws-lambda-power-tuning
sam deploy --guided

# Run tuning (invoke the state machine)
aws stepfunctions start-execution \
  --state-machine-arn arn:aws:states:us-east-1:123:stateMachine:powerTuningStateMachine \
  --input '{
    "lambdaARN": "arn:aws:lambda:us-east-1:123:function:my-function",
    "powerValues": [128, 256, 512, 1024, 2048, 3008],
    "num": 50,
    "payload": {"key": "value"},
    "parallelInvocation": true,
    "strategy": "cost"
  }'

The output is a URL to a visualization showing cost and duration at each memory setting. For CPU-bound functions, the optimal cost point is often at 1,769 MB (1 full vCPU) even if the function only uses 400 MB of memory — the extra CPU reduces execution time enough to offset the higher per-ms billing rate.

Observability

Structured logging

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

import json
import logging

logger = logging.getLogger()
logger.setLevel(logging.INFO)

def handler(event, context):
    logger.info(json.dumps({
        "event": "order.processing.started",
        "orderId": event["orderId"],
        "requestId": context.aws_request_id,
        "functionVersion": context.function_version,
    }))

CloudWatch Logs Insights queries structured JSON logs efficiently:

fields @timestamp, event, orderId, @duration
| filter event = "order.processing.failed"
| sort @timestamp desc
| limit 20

Key metrics to monitor

X-Ray tracing

Enable active tracing to get end-to-end traces across Lambda invocations and downstream AWS service calls:

1
2
3
4
5
6

resource "aws_lambda_function" "api" {
  # ...
  tracing_config {
    mode = "Active"
  }
}

In your handler, instrument SDK calls:

1
2
3
4
5
6
7
8
9

from aws_xray_sdk.core import xray_recorder, patch_all
patch_all()  # instruments boto3, requests, httplib, etc.

def handler(event, context):
    with xray_recorder.in_subsegment("validate-order"):
        validate(event)

    with xray_recorder.in_subsegment("write-db"):
        save_to_database(event)

X-Ray traces show the full invocation timeline including cold start duration, downstream calls, and where time is actually spent.

Honest Trade-offs

Lambda is not always cheap. The free tier (1M requests/month, 400,000 GB-seconds) covers small workloads. Beyond that, cost depends heavily on invocation volume, duration, and memory. A high-throughput API processing 100M requests/month at 100ms average duration with 512 MB memory costs ~$400/month in Lambda alone — before API Gateway, data transfer, and CloudWatch. An EC2 t3.medium at ~$30/month handles significant throughput. The break-even point is often lower than teams expect.

The INIT phase billing change. As of August 2025, initialization code that was previously “free” is now billed. Functions with heavy initialization (loading ML models, establishing many connections) saw Lambda bill increases of 10–50%. Measure before assuming Lambda is cheap.

Cold starts affect more users than benchmarks suggest. A 1-second cold start on a function with 0.1% cold start rate feels invisible in aggregate metrics but hits 1 in 1,000 users with a bad experience. At P99 cold starts, real-user latency looks fine in P50 dashboards. Instrument InitDuration separately and track it by percentile.

SQS at-least-once is an architectural constraint, not a footnote. Every Lambda that processes SQS messages must be idempotent. Most are not, which means most SQS Lambda pipelines have silent duplicate processing bugs waiting for a deployment failure or concurrency spike to surface them.

Step Functions state machine definition is infrastructure. Changes to the state machine require careful versioning — executions in flight when you deploy a new definition continue running against the old definition. For long-running workflows (days to weeks), this means supporting multiple versions of the state machine simultaneously. Plan for this before building workflows that span business days.

The operational model changes. Lambda removes the concern of server provisioning and patching but adds the concern of function proliferation, invocation model variety (synchronous, asynchronous, stream, event source), and distributed system failure modes. A microservices architecture on Lambda can have 50–100 functions with different concurrency limits, different error handling, and different scaling behaviors. That is a lot of operational surface to understand before something breaks at 2 AM.