LUNAROPS · OPERATIONAL UPLINK 100% UPTIME 1,247d POSTS 893 JEFF.MOON@LUNAROPS.DEV UTC --:--:--

Bash Scripting Patterns That Hold Up in Production

bashshellscriptinglinuxdevopsautomation

Most Bash scripts start the same way: someone needs to automate one thing, writes ten lines, it works, and then three months later it’s running in CI, being sourced by other scripts, and nobody remembers what it does. This guide covers the patterns that keep scripts maintainable, debuggable, and safe when that happens.

Start Every Script the Same Way

These four lines should open every non-trivial script:

1
2
3
#!/usr/bin/env bash
set -euo pipefail
IFS=$'\n\t'

What each does

#!/usr/bin/env bash — Uses env to find bash rather than hardcoding /bin/bash. More portable across macOS, Linux, and NixOS where bash may live elsewhere.

set -e — Exit immediately if any command returns a non-zero exit code. Without this, errors silently pass and subsequent commands run on bad state.

set -u — Treat unset variables as errors. Catches typos in variable names before they cause data loss (rm -rf "$DIIR/" vs rm -rf "$DIR/").

set -o pipefail — Normally, a pipeline’s exit code is just the last command’s. With pipefail, the pipeline fails if any command in it fails. Without it, failing_command | tee output.log always succeeds.

IFS=$'\n\t' — Changes the Internal Field Separator from space/tab/newline to just tab/newline. Prevents word splitting on filenames with spaces.

1
2
3
4
5
#!/usr/bin/env bash
set -euo pipefail
IFS=$'\n\t'

# The rest of your script...

Script Structure

A well-structured script is readable from top to bottom like a document:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
#!/usr/bin/env bash
set -euo pipefail
IFS=$'\n\t'

# ── Constants ──────────────────────────────────────────────────────────────────
readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
readonly SCRIPT_NAME="$(basename "$0")"
readonly LOG_FILE="/var/log/${SCRIPT_NAME%.sh}.log"

# ── Defaults ───────────────────────────────────────────────────────────────────
VERBOSE=false
DRY_RUN=false
OUTPUT_DIR="/tmp/output"

# ── Functions ──────────────────────────────────────────────────────────────────
usage() { ... }
log()   { ... }
cleanup() { ... }

# ── Argument parsing ───────────────────────────────────────────────────────────
parse_args() { ... }

# ── Main logic ─────────────────────────────────────────────────────────────────
main() { ... }

# ── Entry point ────────────────────────────────────────────────────────────────
parse_args "$@"
main

Keep main at the bottom and call it last. This means all functions are defined before they’re invoked, which avoids a class of “function not found” errors.

SCRIPT_DIR — The Right Way

Scripts that reference other files relative to themselves often break when called from a different directory. This idiom always gives you the real directory the script lives in:

1
readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"

Use ${BASH_SOURCE[0]} instead of $0$0 breaks when the script is sourced. Then reference other files as "$SCRIPT_DIR/lib/helpers.sh" instead of "./lib/helpers.sh".


Logging

A consistent logging function is more useful than scattered echo calls. It lets you control verbosity, add timestamps, and route to a log file — all in one place.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
# Log levels
readonly LOG_LEVEL_DEBUG=0
readonly LOG_LEVEL_INFO=1
readonly LOG_LEVEL_WARN=2
readonly LOG_LEVEL_ERROR=3

CURRENT_LOG_LEVEL=${LOG_LEVEL_INFO}

log() {
    local level="$1"
    local message="$2"
    local timestamp
    timestamp="$(date '+%Y-%m-%d %H:%M:%S')"
    local level_name

    case "$level" in
        0) level_name="DEBUG" ;;
        1) level_name="INFO " ;;
        2) level_name="WARN " ;;
        3) level_name="ERROR" ;;
        *) level_name="?????" ;;
    esac

    if [[ "$level" -ge "$CURRENT_LOG_LEVEL" ]]; then
        local line="[${timestamp}] [${level_name}] ${message}"

        # Color output to terminal
        if [[ -t 2 ]]; then
            case "$level" in
                2) line="\033[33m${line}\033[0m" ;;   # Yellow for WARN
                3) line="\033[31m${line}\033[0m" ;;   # Red for ERROR
            esac
        fi

        echo -e "$line" >&2

        # Also write to log file if defined
        if [[ -n "${LOG_FILE:-}" ]]; then
            echo "[${timestamp}] [${level_name}] ${message}" >> "$LOG_FILE"
        fi
    fi
}

# Convenience wrappers
log_debug() { log "$LOG_LEVEL_DEBUG" "$*"; }
log_info()  { log "$LOG_LEVEL_INFO"  "$*"; }
log_warn()  { log "$LOG_LEVEL_WARN"  "$*"; }
log_error() { log "$LOG_LEVEL_ERROR" "$*"; }

Usage:

1
2
3
log_info  "Starting backup of $SOURCE_DIR"
log_warn  "Destination is almost full (${DISK_USAGE}% used)"
log_error "Failed to connect to $HOST after $MAX_RETRIES attempts"

Error Handling

The die Function

When you need to exit with an error message:

1
2
3
4
die() {
    log_error "$*"
    exit 1
}
1
2
[[ -f "$CONFIG_FILE" ]] || die "Config file not found: $CONFIG_FILE"
[[ $EUID -eq 0 ]] || die "This script must be run as root"

Checking Command Success

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
# Bad — error is silently swallowed
cp "$src" "$dst"
if [[ $? -ne 0 ]]; then
    die "Copy failed"
fi

# Good — direct test
if ! cp "$src" "$dst"; then
    die "Failed to copy $src to $dst"
fi

# Also fine for simple cases
cp "$src" "$dst" || die "Failed to copy $src to $dst"

Traps — Cleanup on Exit

trap runs a function when the script exits, whether normally, on error, or on a signal. Always clean up temp files and half-written state.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
# Create temp directory safely
TMPDIR_WORK=""

cleanup() {
    local exit_code=$?
    if [[ -n "$TMPDIR_WORK" && -d "$TMPDIR_WORK" ]]; then
        rm -rf "$TMPDIR_WORK"
        log_debug "Cleaned up temp dir: $TMPDIR_WORK"
    fi
    exit "$exit_code"
}

trap cleanup EXIT INT TERM

# Now it's safe to create temp files — they'll always be removed
TMPDIR_WORK="$(mktemp -d)"

Traps to know:

Signal Triggered When
EXIT Script exits for any reason
INT User presses Ctrl+C
TERM Script receives SIGTERM (e.g. kill)
ERR Any command returns non-zero (with set -e)

Disabling set -e Locally

Sometimes a command is expected to fail:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
# ping returns non-zero if host is unreachable — that's our check
if ping -c1 -W1 "$HOST" &>/dev/null; then
    log_info "$HOST is reachable"
else
    log_warn "$HOST is not reachable"
fi

# Or use the explicit override
set +e
some_command_that_may_fail
exit_code=$?
set -e
process_exit_code "$exit_code"

Argument Parsing

Simple Positional Arguments

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
usage() {
    cat <<EOF
Usage: $SCRIPT_NAME [OPTIONS] SOURCE DESTINATION

Copy SOURCE to DESTINATION with optional compression.

Options:
  -c, --compress     Compress output with gzip
  -v, --verbose      Enable verbose logging
  -n, --dry-run      Show what would be done without doing it
  -h, --help         Show this help message

Examples:
  $SCRIPT_NAME /data/backups /mnt/nas/backups
  $SCRIPT_NAME --compress --verbose /home/user /backup/user
EOF
}

parse_args() {
    local positional=()

    while [[ $# -gt 0 ]]; do
        case "$1" in
            -c|--compress)
                COMPRESS=true
                shift
                ;;
            -v|--verbose)
                VERBOSE=true
                CURRENT_LOG_LEVEL=$LOG_LEVEL_DEBUG
                shift
                ;;
            -n|--dry-run)
                DRY_RUN=true
                shift
                ;;
            -h|--help)
                usage
                exit 0
                ;;
            -*)
                die "Unknown option: $1"
                ;;
            *)
                positional+=("$1")
                shift
                ;;
        esac
    done

    # Validate positional arguments
    if [[ ${#positional[@]} -ne 2 ]]; then
        usage
        die "Expected exactly 2 arguments, got ${#positional[@]}"
    fi

    SOURCE="${positional[0]}"
    DESTINATION="${positional[1]}"
}

Options with Values

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
while [[ $# -gt 0 ]]; do
    case "$1" in
        -o|--output)
            [[ $# -gt 1 ]] || die "--output requires an argument"
            OUTPUT_DIR="$2"
            shift 2
            ;;
        --output=*)
            OUTPUT_DIR="${1#--output=}"
            shift
            ;;
        # ...
    esac
done

Using getopts (POSIX)

For scripts that must run on POSIX sh as well as bash:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
while getopts ":hvcno:" opt; do
    case "$opt" in
        h) usage; exit 0 ;;
        v) VERBOSE=true ;;
        c) COMPRESS=true ;;
        n) DRY_RUN=true ;;
        o) OUTPUT_DIR="$OPTARG" ;;
        :) die "Option -$OPTARG requires an argument" ;;
        ?) die "Unknown option: -$OPTARG" ;;
    esac
done
shift $((OPTIND - 1))

# Remaining args are positional
SOURCE="${1:-}"
DESTINATION="${2:-}"

Variable Handling

Default Values

1
2
3
4
5
6
7
# Use default if variable is unset or empty
OUTPUT_DIR="${OUTPUT_DIR:-/tmp/output}"
RETRIES="${RETRIES:-3}"
TIMEOUT="${TIMEOUT:-30}"

# Use default only if unset (not if empty)
NAME="${NAME-default}"

Required Variables

1
2
: "${DATABASE_URL:?DATABASE_URL must be set}"
: "${API_KEY:?API_KEY must be set}"

The : is a no-op, but ${VAR:?message} causes the script to exit with the message if VAR is unset or empty.

String Operations

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
FILE="/var/log/app.log.2024-01-15"

# Remove prefix
echo "${FILE#/var/log/}"      # app.log.2024-01-15

# Remove suffix
echo "${FILE%.log*}"          # /var/log/app

# Extract filename
echo "${FILE##*/}"            # app.log.2024-01-15

# Extract directory
echo "${FILE%/*}"             # /var/log

# String replacement
echo "${FILE/log/LOG}"        # /var/LOG/app.log.2024-01-15 (first)
echo "${FILE//log/LOG}"       # /var/LOG/app.LOG.2024-01-15 (all)

# Substring
echo "${FILE:9:3}"            # log (offset 9, length 3)

# Length
echo "${#FILE}"               # 27

# Uppercase / lowercase (bash 4+)
VAR="Hello World"
echo "${VAR^^}"               # HELLO WORLD
echo "${VAR,,}"               # hello world

Arrays

Bash arrays are underused and solve many problems cleanly.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
# Declare
files=()
servers=("web01" "web02" "db01")

# Add elements
files+=("/etc/nginx/nginx.conf")
files+=("/etc/nginx/conf.d/app.conf")

# Length
echo "${#servers[@]}"         # 3

# Access element
echo "${servers[0]}"          # web01

# Iterate
for server in "${servers[@]}"; do
    echo "Processing $server"
done

# Slice
echo "${servers[@]:1:2}"      # web02 db01

# Check if array contains value
contains() {
    local needle="$1"
    shift
    local item
    for item in "$@"; do
        [[ "$item" == "$needle" ]] && return 0
    done
    return 1
}

if contains "db01" "${servers[@]}"; then
    echo "db01 is in the list"
fi

Associative Arrays (bash 4+)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
declare -A config
config["host"]="localhost"
config["port"]="5432"
config["dbname"]="myapp"

echo "${config[host]}"

# Iterate keys and values
for key in "${!config[@]}"; do
    echo "$key = ${config[$key]}"
done

Functions

Return Values

Bash functions return exit codes (0–255), not values. Return data through stdout or a variable:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
# Via stdout (subprocess — slower but clean)
get_timestamp() {
    date '+%Y%m%d_%H%M%S'
}
TIMESTAMP="$(get_timestamp)"

# Via a named variable (faster, no subshell)
get_timestamp_into() {
    local -n _result_ref="$1"   # nameref — bash 4.3+
    _result_ref="$(date '+%Y%m%d_%H%M%S')"
}
get_timestamp_into TIMESTAMP

Local Variables

Always use local inside functions to avoid polluting the global namespace:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
process_file() {
    local file="$1"
    local line_count
    local tmp_file

    tmp_file="$(mktemp)"
    line_count="$(wc -l < "$file")"

    log_info "Processing $file ($line_count lines)"
    # ...
}

Dry Run Pattern

A --dry-run flag is invaluable for scripts that modify state. The cleanest implementation wraps execution in a helper:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
DRY_RUN=false

run() {
    if [[ "$DRY_RUN" == true ]]; then
        log_info "[DRY RUN] $*"
    else
        "$@"
    fi
}

# Usage — wrap any destructive command
run rm -rf "$OLD_DIR"
run systemctl restart nginx
run aws s3 sync "$LOCAL_DIR" "s3://$BUCKET/"
run docker compose up -d --build

Input Validation Patterns

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
# Validate integer
is_integer() {
    [[ "$1" =~ ^-?[0-9]+$ ]]
}

# Validate IP address
is_valid_ip() {
    local ip="$1"
    local regex='^([0-9]{1,3}\.){3}[0-9]{1,3}$'
    if [[ "$ip" =~ $regex ]]; then
        local IFS='.'
        read -ra octets <<< "$ip"
        for octet in "${octets[@]}"; do
            [[ "$octet" -le 255 ]] || return 1
        done
        return 0
    fi
    return 1
}

# Validate that a directory exists and is writable
require_writable_dir() {
    local dir="$1"
    [[ -d "$dir" ]] || die "Directory does not exist: $dir"
    [[ -w "$dir" ]] || die "Directory is not writable: $dir"
}

# Validate required commands exist
require_commands() {
    local missing=()
    for cmd in "$@"; do
        command -v "$cmd" &>/dev/null || missing+=("$cmd")
    done
    if [[ ${#missing[@]} -gt 0 ]]; then
        die "Required commands not found: ${missing[*]}"
    fi
}

# At the top of main():
require_commands curl jq aws docker
require_writable_dir "$OUTPUT_DIR"

Retry Logic

Network calls and flaky operations benefit from automatic retries:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
retry() {
    local max_attempts="${1}"
    local delay="${2}"
    local description="${3}"
    shift 3

    local attempt=1
    while true; do
        if "$@"; then
            return 0
        fi

        if [[ "$attempt" -ge "$max_attempts" ]]; then
            log_error "Failed after $max_attempts attempts: $description"
            return 1
        fi

        log_warn "Attempt $attempt/$max_attempts failed for: $description. Retrying in ${delay}s..."
        sleep "$delay"
        ((attempt++))
        delay=$((delay * 2))  # Exponential backoff
    done
}

# Usage
retry 5 2 "health check" curl -sf "http://localhost:8080/health"
retry 3 10 "database migration" run_migration

Locking — Prevent Concurrent Runs

Scripts run by cron can overlap if they take longer than expected:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
LOCK_FILE="/var/run/${SCRIPT_NAME%.sh}.lock"

acquire_lock() {
    if ! mkdir "$LOCK_FILE" 2>/dev/null; then
        local pid
        pid="$(cat "$LOCK_FILE/pid" 2>/dev/null || echo "unknown")"
        die "Another instance is running (PID: $pid). Lock: $LOCK_FILE"
    fi
    echo $$ > "$LOCK_FILE/pid"
    log_debug "Acquired lock: $LOCK_FILE"
}

release_lock() {
    rm -rf "$LOCK_FILE"
    log_debug "Released lock: $LOCK_FILE"
}

# Add to cleanup
cleanup() {
    local exit_code=$?
    release_lock
    [[ -n "${TMPDIR_WORK:-}" ]] && rm -rf "$TMPDIR_WORK"
    exit "$exit_code"
}

trap cleanup EXIT INT TERM
acquire_lock

Using mkdir for locking is atomic — two processes cannot both successfully mkdir the same path simultaneously.


Portability Tips

Pattern Portable? Notes
#!/usr/bin/env bash Yes Safer than #!/bin/bash
[[ vs [ bash only [[ is safer, avoid in /bin/sh scripts
$(...) vs backticks Prefer $(...) Nestable, readable
local in functions bash/zsh Not POSIX, avoid in sh scripts
(( )) arithmetic bash only Use $(( )) for POSIX
{a..z} brace expansion bash only Not in sh
read -r POSIX Always use -r to disable backslash processing
printf vs echo Prefer printf echo behavior varies across implementations
1
2
3
# Safer printf instead of echo for arbitrary strings
printf '%s\n' "$potentially_tricky_string"
printf 'Name: %s, Count: %d\n' "$name" "$count"

Debugging

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
# Trace every command as it executes (most useful debugging tool)
set -x

# Or enable just for a section
set -x
some_complex_logic
set +x

# Run the whole script in trace mode
bash -x ./myscript.sh

# Print each line before executing (like set -x but before expansion)
set -v

# Check syntax without running
bash -n ./myscript.sh

# Shellcheck — static analysis (install separately)
shellcheck ./myscript.sh

shellcheck catches a huge class of common mistakes and is worth running on every script before committing. Install with apt install shellcheck or brew install shellcheck.


Putting It All Together

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
#!/usr/bin/env bash
set -euo pipefail
IFS=$'\n\t'

readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
readonly SCRIPT_NAME="$(basename "$0")"

# Defaults
VERBOSE=false
DRY_RUN=false
OUTPUT_DIR="/tmp/deploy-output"

# Logging
log_info()  { echo "[INFO]  $*" >&2; }
log_warn()  { echo "[WARN]  $*" >&2; }
log_error() { echo "[ERROR] $*" >&2; }
die()       { log_error "$*"; exit 1; }

run() {
    if [[ "$DRY_RUN" == true ]]; then
        log_info "[DRY RUN] $*"
    else
        "$@"
    fi
}

cleanup() {
    local code=$?
    [[ -n "${TMPDIR_WORK:-}" ]] && rm -rf "$TMPDIR_WORK"
    exit "$code"
}
trap cleanup EXIT INT TERM

usage() {
    cat <<EOF
Usage: $SCRIPT_NAME [OPTIONS] ENVIRONMENT

Deploy the application to ENVIRONMENT (staging|production).

Options:
  -o, --output DIR   Output directory (default: $OUTPUT_DIR)
  -n, --dry-run      Show what would be done
  -v, --verbose      Verbose output
  -h, --help         Show this help
EOF
}

parse_args() {
    local positional=()
    while [[ $# -gt 0 ]]; do
        case "$1" in
            -o|--output) OUTPUT_DIR="$2"; shift 2 ;;
            -n|--dry-run) DRY_RUN=true; shift ;;
            -v|--verbose) VERBOSE=true; shift ;;
            -h|--help) usage; exit 0 ;;
            -*) die "Unknown option: $1" ;;
            *) positional+=("$1"); shift ;;
        esac
    done
    [[ ${#positional[@]} -eq 1 ]] || { usage; die "Expected exactly 1 argument"; }
    ENVIRONMENT="${positional[0]}"
    [[ "$ENVIRONMENT" =~ ^(staging|production)$ ]] || die "Invalid environment: $ENVIRONMENT"
}

main() {
    require_commands docker git curl

    log_info "Deploying to $ENVIRONMENT"
    [[ "$DRY_RUN" == true ]] && log_warn "Dry run mode — no changes will be made"

    TMPDIR_WORK="$(mktemp -d)"

    run docker compose pull
    run docker compose up -d --build
    run curl -sf "http://localhost:8080/health" || die "Health check failed after deploy"

    log_info "Deploy to $ENVIRONMENT complete"
}

require_commands() {
    local missing=()
    for cmd in "$@"; do command -v "$cmd" &>/dev/null || missing+=("$cmd"); done
    [[ ${#missing[@]} -eq 0 ]] || die "Missing commands: ${missing[*]}"
}

parse_args "$@"
main

Quick Reference

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
# Header
#!/usr/bin/env bash
set -euo pipefail
IFS=$'\n\t'

# Safe temp dir
TMPDIR_WORK="$(mktemp -d)"
trap 'rm -rf "$TMPDIR_WORK"' EXIT

# Script directory
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"

# Default value
VAR="${VAR:-default}"

# Required value
: "${VAR:?VAR must be set}"

# Check command exists
command -v docker &>/dev/null || die "docker not found"

# Silent redirect
cmd > /dev/null 2>&1

# Capture exit code without triggering set -e
cmd || exit_code=$?

# Array iteration (handles spaces correctly)
for item in "${array[@]}"; do ...

# Conditional with multiple commands
if cmd1 && cmd2; then ...

# Arithmetic
count=$(( count + 1 ))
(( count++ ))

Bash scripting rewards the same habits as any other programming: consistent structure, explicit error handling, and testing your assumptions. A script with proper error handling, a cleanup trap, and a --dry-run flag is one you can hand to someone else and trust in production.

Comments