Most Bash scripts start the same way: someone needs to automate one thing, writes ten lines, it works, and then three months later it’s running in CI, being sourced by other scripts, and nobody remembers what it does. This guide covers the patterns that keep scripts maintainable, debuggable, and safe when that happens.
Start Every Script the Same Way
These four lines should open every non-trivial script:
1
2
3
|
#!/usr/bin/env bash
set -euo pipefail
IFS=$'\n\t'
|
What each does
#!/usr/bin/env bash — Uses env to find bash rather than hardcoding /bin/bash. More portable across macOS, Linux, and NixOS where bash may live elsewhere.
set -e — Exit immediately if any command returns a non-zero exit code. Without this, errors silently pass and subsequent commands run on bad state.
set -u — Treat unset variables as errors. Catches typos in variable names before they cause data loss (rm -rf "$DIIR/" vs rm -rf "$DIR/").
set -o pipefail — Normally, a pipeline’s exit code is just the last command’s. With pipefail, the pipeline fails if any command in it fails. Without it, failing_command | tee output.log always succeeds.
IFS=$'\n\t' — Changes the Internal Field Separator from space/tab/newline to just tab/newline. Prevents word splitting on filenames with spaces.
1
2
3
4
5
|
#!/usr/bin/env bash
set -euo pipefail
IFS=$'\n\t'
# The rest of your script...
|
Script Structure
A well-structured script is readable from top to bottom like a document:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
|
#!/usr/bin/env bash
set -euo pipefail
IFS=$'\n\t'
# ── Constants ──────────────────────────────────────────────────────────────────
readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
readonly SCRIPT_NAME="$(basename "$0")"
readonly LOG_FILE="/var/log/${SCRIPT_NAME%.sh}.log"
# ── Defaults ───────────────────────────────────────────────────────────────────
VERBOSE=false
DRY_RUN=false
OUTPUT_DIR="/tmp/output"
# ── Functions ──────────────────────────────────────────────────────────────────
usage() { ... }
log() { ... }
cleanup() { ... }
# ── Argument parsing ───────────────────────────────────────────────────────────
parse_args() { ... }
# ── Main logic ─────────────────────────────────────────────────────────────────
main() { ... }
# ── Entry point ────────────────────────────────────────────────────────────────
parse_args "$@"
main
|
Keep main at the bottom and call it last. This means all functions are defined before they’re invoked, which avoids a class of “function not found” errors.
SCRIPT_DIR — The Right Way
Scripts that reference other files relative to themselves often break when called from a different directory. This idiom always gives you the real directory the script lives in:
1
|
readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
Use ${BASH_SOURCE[0]} instead of $0 — $0 breaks when the script is sourced. Then reference other files as "$SCRIPT_DIR/lib/helpers.sh" instead of "./lib/helpers.sh".
Logging
A consistent logging function is more useful than scattered echo calls. It lets you control verbosity, add timestamps, and route to a log file — all in one place.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
|
# Log levels
readonly LOG_LEVEL_DEBUG=0
readonly LOG_LEVEL_INFO=1
readonly LOG_LEVEL_WARN=2
readonly LOG_LEVEL_ERROR=3
CURRENT_LOG_LEVEL=${LOG_LEVEL_INFO}
log() {
local level="$1"
local message="$2"
local timestamp
timestamp="$(date '+%Y-%m-%d %H:%M:%S')"
local level_name
case "$level" in
0) level_name="DEBUG" ;;
1) level_name="INFO " ;;
2) level_name="WARN " ;;
3) level_name="ERROR" ;;
*) level_name="?????" ;;
esac
if [[ "$level" -ge "$CURRENT_LOG_LEVEL" ]]; then
local line="[${timestamp}] [${level_name}] ${message}"
# Color output to terminal
if [[ -t 2 ]]; then
case "$level" in
2) line="\033[33m${line}\033[0m" ;; # Yellow for WARN
3) line="\033[31m${line}\033[0m" ;; # Red for ERROR
esac
fi
echo -e "$line" >&2
# Also write to log file if defined
if [[ -n "${LOG_FILE:-}" ]]; then
echo "[${timestamp}] [${level_name}] ${message}" >> "$LOG_FILE"
fi
fi
}
# Convenience wrappers
log_debug() { log "$LOG_LEVEL_DEBUG" "$*"; }
log_info() { log "$LOG_LEVEL_INFO" "$*"; }
log_warn() { log "$LOG_LEVEL_WARN" "$*"; }
log_error() { log "$LOG_LEVEL_ERROR" "$*"; }
|
Usage:
1
2
3
|
log_info "Starting backup of $SOURCE_DIR"
log_warn "Destination is almost full (${DISK_USAGE}% used)"
log_error "Failed to connect to $HOST after $MAX_RETRIES attempts"
|
Error Handling
The die Function
When you need to exit with an error message:
1
2
3
4
|
die() {
log_error "$*"
exit 1
}
|
1
2
|
[[ -f "$CONFIG_FILE" ]] || die "Config file not found: $CONFIG_FILE"
[[ $EUID -eq 0 ]] || die "This script must be run as root"
|
Checking Command Success
1
2
3
4
5
6
7
8
9
10
11
12
13
|
# Bad — error is silently swallowed
cp "$src" "$dst"
if [[ $? -ne 0 ]]; then
die "Copy failed"
fi
# Good — direct test
if ! cp "$src" "$dst"; then
die "Failed to copy $src to $dst"
fi
# Also fine for simple cases
cp "$src" "$dst" || die "Failed to copy $src to $dst"
|
Traps — Cleanup on Exit
trap runs a function when the script exits, whether normally, on error, or on a signal. Always clean up temp files and half-written state.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
|
# Create temp directory safely
TMPDIR_WORK=""
cleanup() {
local exit_code=$?
if [[ -n "$TMPDIR_WORK" && -d "$TMPDIR_WORK" ]]; then
rm -rf "$TMPDIR_WORK"
log_debug "Cleaned up temp dir: $TMPDIR_WORK"
fi
exit "$exit_code"
}
trap cleanup EXIT INT TERM
# Now it's safe to create temp files — they'll always be removed
TMPDIR_WORK="$(mktemp -d)"
|
Traps to know:
| Signal |
Triggered When |
EXIT |
Script exits for any reason |
INT |
User presses Ctrl+C |
TERM |
Script receives SIGTERM (e.g. kill) |
ERR |
Any command returns non-zero (with set -e) |
Disabling set -e Locally
Sometimes a command is expected to fail:
1
2
3
4
5
6
7
8
9
10
11
12
13
|
# ping returns non-zero if host is unreachable — that's our check
if ping -c1 -W1 "$HOST" &>/dev/null; then
log_info "$HOST is reachable"
else
log_warn "$HOST is not reachable"
fi
# Or use the explicit override
set +e
some_command_that_may_fail
exit_code=$?
set -e
process_exit_code "$exit_code"
|
Argument Parsing
Simple Positional Arguments
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
|
usage() {
cat <<EOF
Usage: $SCRIPT_NAME [OPTIONS] SOURCE DESTINATION
Copy SOURCE to DESTINATION with optional compression.
Options:
-c, --compress Compress output with gzip
-v, --verbose Enable verbose logging
-n, --dry-run Show what would be done without doing it
-h, --help Show this help message
Examples:
$SCRIPT_NAME /data/backups /mnt/nas/backups
$SCRIPT_NAME --compress --verbose /home/user /backup/user
EOF
}
parse_args() {
local positional=()
while [[ $# -gt 0 ]]; do
case "$1" in
-c|--compress)
COMPRESS=true
shift
;;
-v|--verbose)
VERBOSE=true
CURRENT_LOG_LEVEL=$LOG_LEVEL_DEBUG
shift
;;
-n|--dry-run)
DRY_RUN=true
shift
;;
-h|--help)
usage
exit 0
;;
-*)
die "Unknown option: $1"
;;
*)
positional+=("$1")
shift
;;
esac
done
# Validate positional arguments
if [[ ${#positional[@]} -ne 2 ]]; then
usage
die "Expected exactly 2 arguments, got ${#positional[@]}"
fi
SOURCE="${positional[0]}"
DESTINATION="${positional[1]}"
}
|
Options with Values
1
2
3
4
5
6
7
8
9
10
11
12
13
14
|
while [[ $# -gt 0 ]]; do
case "$1" in
-o|--output)
[[ $# -gt 1 ]] || die "--output requires an argument"
OUTPUT_DIR="$2"
shift 2
;;
--output=*)
OUTPUT_DIR="${1#--output=}"
shift
;;
# ...
esac
done
|
Using getopts (POSIX)
For scripts that must run on POSIX sh as well as bash:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
|
while getopts ":hvcno:" opt; do
case "$opt" in
h) usage; exit 0 ;;
v) VERBOSE=true ;;
c) COMPRESS=true ;;
n) DRY_RUN=true ;;
o) OUTPUT_DIR="$OPTARG" ;;
:) die "Option -$OPTARG requires an argument" ;;
?) die "Unknown option: -$OPTARG" ;;
esac
done
shift $((OPTIND - 1))
# Remaining args are positional
SOURCE="${1:-}"
DESTINATION="${2:-}"
|
Variable Handling
Default Values
1
2
3
4
5
6
7
|
# Use default if variable is unset or empty
OUTPUT_DIR="${OUTPUT_DIR:-/tmp/output}"
RETRIES="${RETRIES:-3}"
TIMEOUT="${TIMEOUT:-30}"
# Use default only if unset (not if empty)
NAME="${NAME-default}"
|
Required Variables
1
2
|
: "${DATABASE_URL:?DATABASE_URL must be set}"
: "${API_KEY:?API_KEY must be set}"
|
The : is a no-op, but ${VAR:?message} causes the script to exit with the message if VAR is unset or empty.
String Operations
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
|
FILE="/var/log/app.log.2024-01-15"
# Remove prefix
echo "${FILE#/var/log/}" # app.log.2024-01-15
# Remove suffix
echo "${FILE%.log*}" # /var/log/app
# Extract filename
echo "${FILE##*/}" # app.log.2024-01-15
# Extract directory
echo "${FILE%/*}" # /var/log
# String replacement
echo "${FILE/log/LOG}" # /var/LOG/app.log.2024-01-15 (first)
echo "${FILE//log/LOG}" # /var/LOG/app.LOG.2024-01-15 (all)
# Substring
echo "${FILE:9:3}" # log (offset 9, length 3)
# Length
echo "${#FILE}" # 27
# Uppercase / lowercase (bash 4+)
VAR="Hello World"
echo "${VAR^^}" # HELLO WORLD
echo "${VAR,,}" # hello world
|
Arrays
Bash arrays are underused and solve many problems cleanly.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
|
# Declare
files=()
servers=("web01" "web02" "db01")
# Add elements
files+=("/etc/nginx/nginx.conf")
files+=("/etc/nginx/conf.d/app.conf")
# Length
echo "${#servers[@]}" # 3
# Access element
echo "${servers[0]}" # web01
# Iterate
for server in "${servers[@]}"; do
echo "Processing $server"
done
# Slice
echo "${servers[@]:1:2}" # web02 db01
# Check if array contains value
contains() {
local needle="$1"
shift
local item
for item in "$@"; do
[[ "$item" == "$needle" ]] && return 0
done
return 1
}
if contains "db01" "${servers[@]}"; then
echo "db01 is in the list"
fi
|
Associative Arrays (bash 4+)
1
2
3
4
5
6
7
8
9
10
11
|
declare -A config
config["host"]="localhost"
config["port"]="5432"
config["dbname"]="myapp"
echo "${config[host]}"
# Iterate keys and values
for key in "${!config[@]}"; do
echo "$key = ${config[$key]}"
done
|
Functions
Return Values
Bash functions return exit codes (0–255), not values. Return data through stdout or a variable:
1
2
3
4
5
6
7
8
9
10
11
12
|
# Via stdout (subprocess — slower but clean)
get_timestamp() {
date '+%Y%m%d_%H%M%S'
}
TIMESTAMP="$(get_timestamp)"
# Via a named variable (faster, no subshell)
get_timestamp_into() {
local -n _result_ref="$1" # nameref — bash 4.3+
_result_ref="$(date '+%Y%m%d_%H%M%S')"
}
get_timestamp_into TIMESTAMP
|
Local Variables
Always use local inside functions to avoid polluting the global namespace:
1
2
3
4
5
6
7
8
9
10
11
|
process_file() {
local file="$1"
local line_count
local tmp_file
tmp_file="$(mktemp)"
line_count="$(wc -l < "$file")"
log_info "Processing $file ($line_count lines)"
# ...
}
|
Dry Run Pattern
A --dry-run flag is invaluable for scripts that modify state. The cleanest implementation wraps execution in a helper:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
|
DRY_RUN=false
run() {
if [[ "$DRY_RUN" == true ]]; then
log_info "[DRY RUN] $*"
else
"$@"
fi
}
# Usage — wrap any destructive command
run rm -rf "$OLD_DIR"
run systemctl restart nginx
run aws s3 sync "$LOCAL_DIR" "s3://$BUCKET/"
run docker compose up -d --build
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
|
# Validate integer
is_integer() {
[[ "$1" =~ ^-?[0-9]+$ ]]
}
# Validate IP address
is_valid_ip() {
local ip="$1"
local regex='^([0-9]{1,3}\.){3}[0-9]{1,3}$'
if [[ "$ip" =~ $regex ]]; then
local IFS='.'
read -ra octets <<< "$ip"
for octet in "${octets[@]}"; do
[[ "$octet" -le 255 ]] || return 1
done
return 0
fi
return 1
}
# Validate that a directory exists and is writable
require_writable_dir() {
local dir="$1"
[[ -d "$dir" ]] || die "Directory does not exist: $dir"
[[ -w "$dir" ]] || die "Directory is not writable: $dir"
}
# Validate required commands exist
require_commands() {
local missing=()
for cmd in "$@"; do
command -v "$cmd" &>/dev/null || missing+=("$cmd")
done
if [[ ${#missing[@]} -gt 0 ]]; then
die "Required commands not found: ${missing[*]}"
fi
}
# At the top of main():
require_commands curl jq aws docker
require_writable_dir "$OUTPUT_DIR"
|
Retry Logic
Network calls and flaky operations benefit from automatic retries:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
|
retry() {
local max_attempts="${1}"
local delay="${2}"
local description="${3}"
shift 3
local attempt=1
while true; do
if "$@"; then
return 0
fi
if [[ "$attempt" -ge "$max_attempts" ]]; then
log_error "Failed after $max_attempts attempts: $description"
return 1
fi
log_warn "Attempt $attempt/$max_attempts failed for: $description. Retrying in ${delay}s..."
sleep "$delay"
((attempt++))
delay=$((delay * 2)) # Exponential backoff
done
}
# Usage
retry 5 2 "health check" curl -sf "http://localhost:8080/health"
retry 3 10 "database migration" run_migration
|
Locking — Prevent Concurrent Runs
Scripts run by cron can overlap if they take longer than expected:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
|
LOCK_FILE="/var/run/${SCRIPT_NAME%.sh}.lock"
acquire_lock() {
if ! mkdir "$LOCK_FILE" 2>/dev/null; then
local pid
pid="$(cat "$LOCK_FILE/pid" 2>/dev/null || echo "unknown")"
die "Another instance is running (PID: $pid). Lock: $LOCK_FILE"
fi
echo $$ > "$LOCK_FILE/pid"
log_debug "Acquired lock: $LOCK_FILE"
}
release_lock() {
rm -rf "$LOCK_FILE"
log_debug "Released lock: $LOCK_FILE"
}
# Add to cleanup
cleanup() {
local exit_code=$?
release_lock
[[ -n "${TMPDIR_WORK:-}" ]] && rm -rf "$TMPDIR_WORK"
exit "$exit_code"
}
trap cleanup EXIT INT TERM
acquire_lock
|
Using mkdir for locking is atomic — two processes cannot both successfully mkdir the same path simultaneously.
Portability Tips
| Pattern |
Portable? |
Notes |
#!/usr/bin/env bash |
Yes |
Safer than #!/bin/bash |
[[ vs [ |
bash only |
[[ is safer, avoid in /bin/sh scripts |
$(...) vs backticks |
Prefer $(...) |
Nestable, readable |
local in functions |
bash/zsh |
Not POSIX, avoid in sh scripts |
(( )) arithmetic |
bash only |
Use $(( )) for POSIX |
{a..z} brace expansion |
bash only |
Not in sh |
read -r |
POSIX |
Always use -r to disable backslash processing |
printf vs echo |
Prefer printf |
echo behavior varies across implementations |
1
2
3
|
# Safer printf instead of echo for arbitrary strings
printf '%s\n' "$potentially_tricky_string"
printf 'Name: %s, Count: %d\n' "$name" "$count"
|
Debugging
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
|
# Trace every command as it executes (most useful debugging tool)
set -x
# Or enable just for a section
set -x
some_complex_logic
set +x
# Run the whole script in trace mode
bash -x ./myscript.sh
# Print each line before executing (like set -x but before expansion)
set -v
# Check syntax without running
bash -n ./myscript.sh
# Shellcheck — static analysis (install separately)
shellcheck ./myscript.sh
|
shellcheck catches a huge class of common mistakes and is worth running on every script before committing. Install with apt install shellcheck or brew install shellcheck.
Putting It All Together
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
|
#!/usr/bin/env bash
set -euo pipefail
IFS=$'\n\t'
readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
readonly SCRIPT_NAME="$(basename "$0")"
# Defaults
VERBOSE=false
DRY_RUN=false
OUTPUT_DIR="/tmp/deploy-output"
# Logging
log_info() { echo "[INFO] $*" >&2; }
log_warn() { echo "[WARN] $*" >&2; }
log_error() { echo "[ERROR] $*" >&2; }
die() { log_error "$*"; exit 1; }
run() {
if [[ "$DRY_RUN" == true ]]; then
log_info "[DRY RUN] $*"
else
"$@"
fi
}
cleanup() {
local code=$?
[[ -n "${TMPDIR_WORK:-}" ]] && rm -rf "$TMPDIR_WORK"
exit "$code"
}
trap cleanup EXIT INT TERM
usage() {
cat <<EOF
Usage: $SCRIPT_NAME [OPTIONS] ENVIRONMENT
Deploy the application to ENVIRONMENT (staging|production).
Options:
-o, --output DIR Output directory (default: $OUTPUT_DIR)
-n, --dry-run Show what would be done
-v, --verbose Verbose output
-h, --help Show this help
EOF
}
parse_args() {
local positional=()
while [[ $# -gt 0 ]]; do
case "$1" in
-o|--output) OUTPUT_DIR="$2"; shift 2 ;;
-n|--dry-run) DRY_RUN=true; shift ;;
-v|--verbose) VERBOSE=true; shift ;;
-h|--help) usage; exit 0 ;;
-*) die "Unknown option: $1" ;;
*) positional+=("$1"); shift ;;
esac
done
[[ ${#positional[@]} -eq 1 ]] || { usage; die "Expected exactly 1 argument"; }
ENVIRONMENT="${positional[0]}"
[[ "$ENVIRONMENT" =~ ^(staging|production)$ ]] || die "Invalid environment: $ENVIRONMENT"
}
main() {
require_commands docker git curl
log_info "Deploying to $ENVIRONMENT"
[[ "$DRY_RUN" == true ]] && log_warn "Dry run mode — no changes will be made"
TMPDIR_WORK="$(mktemp -d)"
run docker compose pull
run docker compose up -d --build
run curl -sf "http://localhost:8080/health" || die "Health check failed after deploy"
log_info "Deploy to $ENVIRONMENT complete"
}
require_commands() {
local missing=()
for cmd in "$@"; do command -v "$cmd" &>/dev/null || missing+=("$cmd"); done
[[ ${#missing[@]} -eq 0 ]] || die "Missing commands: ${missing[*]}"
}
parse_args "$@"
main
|
Quick Reference
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
|
# Header
#!/usr/bin/env bash
set -euo pipefail
IFS=$'\n\t'
# Safe temp dir
TMPDIR_WORK="$(mktemp -d)"
trap 'rm -rf "$TMPDIR_WORK"' EXIT
# Script directory
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# Default value
VAR="${VAR:-default}"
# Required value
: "${VAR:?VAR must be set}"
# Check command exists
command -v docker &>/dev/null || die "docker not found"
# Silent redirect
cmd > /dev/null 2>&1
# Capture exit code without triggering set -e
cmd || exit_code=$?
# Array iteration (handles spaces correctly)
for item in "${array[@]}"; do ...
# Conditional with multiple commands
if cmd1 && cmd2; then ...
# Arithmetic
count=$(( count + 1 ))
(( count++ ))
|
Bash scripting rewards the same habits as any other programming: consistent structure, explicit error handling, and testing your assumptions. A script with proper error handling, a cleanup trap, and a --dry-run flag is one you can hand to someone else and trust in production.
Comments