If you work with APIs, cloud CLIs, or any modern infrastructure tooling, you deal with JSON constantly. jq is to JSON what awk is to text — a purpose-built language for slicing, filtering, and transforming structured data at the command line. This guide goes from first principles to the patterns you’ll actually reach for in production.
Installation
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
|
# Debian / Ubuntu
apt-get install -y jq
# RHEL / Rocky / Fedora
dnf install -y jq
# macOS
brew install jq
# Alpine
apk add jq
# Check version
jq --version
# jq-1.7.1
|
The Mental Model
jq is a filter language. Every jq program is a filter that takes input JSON, transforms it, and produces output JSON. You chain filters together with the pipe | operator, exactly like a Unix pipeline.
1
2
3
4
5
6
7
8
9
|
# Basic invocation
echo '{"name":"alice","age":30}' | jq '.name'
# "alice"
# From a file
jq '.name' data.json
# From a URL (with curl)
curl -s https://api.example.com/users/1 | jq '.'
|
jq outputs valid, pretty-printed JSON by default. Use -r (raw) to strip quotes from strings for use in shell scripts.
Basic Filters
Identity — .
The simplest filter. Returns its input unchanged, but pretty-prints it:
1
2
3
4
5
|
echo '{"a":1,"b":2}' | jq '.'
# {
# "a": 1,
# "b": 2
# }
|
Field Access — .field
1
2
3
4
5
6
7
8
9
10
11
12
|
echo '{"user":{"name":"alice","role":"admin"}}' | jq '.user'
# {
# "name": "alice",
# "role": "admin"
# }
jq '.user.name'
# "alice"
# Optional field (no error if field missing)
jq '.user.email?'
# null
|
Array Index — .[N]
1
2
3
4
5
6
7
8
|
echo '[10,20,30,40]' | jq '.[0]'
# 10
jq '.[-1]' # Last element
# 40
jq '.[1:3]' # Slice (index 1 up to but not including 3)
# [20, 30]
|
Array / Object Iterator — .[]
Explodes an array or object into a stream of values:
1
2
3
4
5
6
7
8
|
echo '[1,2,3]' | jq '.[]'
# 1
# 2
# 3
echo '{"a":1,"b":2}' | jq '.[]'
# 1
# 2
|
Pipes and Chaining
Pipe | passes the output of one filter as input to the next:
1
2
3
4
|
echo '{"users":[{"name":"alice"},{"name":"bob"}]}' \
| jq '.users[] | .name'
# "alice"
# "bob"
|
This is the core pattern: navigate to a collection, explode it, then extract a field.
Constructing Output
Arrays — [expr]
Wrap an expression in [] to collect its output into an array:
1
2
3
4
5
6
|
echo '{"users":[{"name":"alice","age":30},{"name":"bob","age":25}]}' \
| jq '[.users[] | .name]'
# [
# "alice",
# "bob"
# ]
|
Objects — {key: expr}
Build new objects from pieces of the input:
1
2
3
4
5
6
|
echo '{"name":"alice","age":30,"email":"alice@example.com","role":"admin"}' \
| jq '{username: .name, isAdmin: (.role == "admin")}'
# {
# "username": "alice",
# "isAdmin": true
# }
|
When the key name matches the field name, use shorthand:
1
2
|
jq '{name, age}'
# { "name": "alice", "age": 30 }
|
Combining Both
1
2
|
# Reshape an array of objects
jq '[.users[] | {name, admin: (.role == "admin")}]'
|
Useful Built-in Functions
length
1
2
3
4
5
6
7
8
|
echo '[1,2,3,4,5]' | jq 'length'
# 5
echo '"hello"' | jq 'length'
# 5
echo '{"a":1,"b":2}' | jq 'length'
# 2
|
keys and values
1
2
3
4
5
6
7
8
|
echo '{"b":2,"a":1,"c":3}' | jq 'keys'
# ["a","b","c"] (sorted alphabetically)
jq 'values'
# [1,2,3]
jq 'keys_unsorted'
# ["b","a","c"] (original order)
|
has and in
1
2
3
4
5
|
echo '{"name":"alice"}' | jq 'has("name")'
# true
jq 'has("email")'
# false
|
type
1
2
3
4
5
6
|
echo '"hello"' | jq 'type' # "string"
echo '42' | jq 'type' # "number"
echo 'null' | jq 'type' # "null"
echo '[]' | jq 'type' # "array"
echo '{}' | jq 'type' # "object"
echo 'true' | jq 'type' # "boolean"
|
select — Conditional Filtering
select(expr) passes through values where expr is true, drops the rest:
1
2
3
4
5
6
7
8
9
|
echo '[1,2,3,4,5,6]' | jq '.[] | select(. > 3)'
# 4
# 5
# 6
# Filter objects in an array
echo '[{"name":"alice","age":30},{"name":"bob","age":17}]' \
| jq '[.[] | select(.age >= 18)]'
# [{"name":"alice","age":30}]
|
map and map_values
map(f) is shorthand for [.[] | f]:
1
2
3
4
5
6
7
8
9
|
echo '[1,2,3,4]' | jq 'map(. * 2)'
# [2,4,6,8]
echo '[{"name":"alice"},{"name":"bob"}]' | jq 'map(.name)'
# ["alice","bob"]
# map_values applies to object values
echo '{"a":1,"b":2,"c":3}' | jq 'map_values(. + 10)'
# {"a":11,"b":12,"c":13}
|
select + map Together
1
2
3
4
5
|
# Get names of all admin users
jq '[.users[] | select(.role == "admin") | .name]'
# Equivalent using map + select
jq '.users | map(select(.role == "admin")) | map(.name)'
|
sort_by and group_by
1
2
3
4
5
6
|
echo '[{"name":"charlie","age":25},{"name":"alice","age":30},{"name":"bob","age":25}]' \
| jq 'sort_by(.age, .name)'
# [{"name":"bob","age":25},{"name":"charlie","age":25},{"name":"alice","age":30}]
jq 'group_by(.age)'
# [[{"name":"bob","age":25},{"name":"charlie","age":25}],[{"name":"alice","age":30}]]
|
unique and unique_by
1
2
3
4
5
6
|
echo '[1,2,2,3,3,3]' | jq 'unique'
# [1,2,3]
echo '[{"id":1,"v":"a"},{"id":1,"v":"b"},{"id":2,"v":"c"}]' \
| jq 'unique_by(.id)'
# [{"id":1,"v":"a"},{"id":2,"v":"c"}]
|
min_by and max_by
1
2
3
4
5
6
|
echo '[{"name":"alice","score":85},{"name":"bob","score":92},{"name":"carol","score":78}]' \
| jq 'max_by(.score)'
# {"name":"bob","score":92}
jq '[max_by(.score), min_by(.score)] | map(.name)'
# ["bob","carol"]
|
flatten
1
2
3
4
5
|
echo '[[1,2],[3,[4,5]]]' | jq 'flatten'
# [1,2,3,4,5]
jq 'flatten(1)' # One level only
# [1,2,3,[4,5]]
|
add
Reduces an array by addition — works on numbers, strings, arrays, and objects:
1
2
3
4
5
6
7
8
|
echo '[1,2,3,4]' | jq 'add'
# 10
echo '["a","b","c"]' | jq 'add'
# "abc"
echo '[{"a":1},{"b":2},{"a":3}]' | jq 'add'
# {"a":3,"b":2} (later values overwrite earlier ones)
|
String Operations
String Interpolation — "\(expr)"
1
2
3
|
echo '{"name":"alice","role":"admin"}' \
| jq '"User \(.name) has role: \(.role)"'
# "User alice has role: admin"
|
split and join
1
2
3
4
5
|
echo '"a,b,c,d"' | jq 'split(",")'
# ["a","b","c","d"]
echo '["a","b","c"]' | jq 'join(", ")'
# "a, b, c"
|
ltrimstr, rtrimstr, startswith, endswith
1
2
3
4
5
6
7
8
|
echo '"hello-world"' | jq 'ltrimstr("hello-")'
# "world"
echo '"image:latest"' | jq 'rtrimstr(":latest")'
# "image"
echo '"api/v1/users"' | jq 'startswith("api/")'
# true
|
ascii_downcase and ascii_upcase
1
2
|
echo '"Hello World"' | jq 'ascii_downcase'
# "hello world"
|
test — Regex Matching
1
2
3
4
5
6
7
|
echo '"alice@example.com"' | jq 'test("^[a-z]+@")'
# true
# Filter array by regex
echo '["alice","bob123","carol"]' \
| jq '[.[] | select(test("[0-9]"))]'
# ["bob123"]
|
capture — Named Regex Groups
1
2
|
echo '"2026-03-25"' | jq 'capture("(?<year>[0-9]{4})-(?<month>[0-9]{2})-(?<day>[0-9]{2})")'
# {"year":"2026","month":"03","day":"25"}
|
Conditionals and Logic
if-then-else
1
2
3
4
5
6
|
echo '42' | jq 'if . > 40 then "big" else "small" end'
# "big"
echo '[{"name":"alice","score":85},{"name":"bob","score":45}]' \
| jq '[.[] | {name, result: (if .score >= 60 then "pass" else "fail" end)}]'
# [{"name":"alice","result":"pass"},{"name":"bob","result":"fail"}]
|
// — Alternative Operator
Returns the right side if the left is false or null:
1
2
3
4
5
|
echo '{"name":"alice"}' | jq '.email // "no email"'
# "no email"
echo 'null' | jq '. // "default"'
# "default"
|
This is the idiomatic way to set defaults for missing fields.
and, or, not
1
2
3
4
5
6
|
echo '{"active":true,"role":"admin"}' \
| jq '.active and (.role == "admin")'
# true
jq '.active | not'
# false
|
reduce — Aggregation
reduce iterates over a stream, accumulating a value:
1
2
3
4
5
6
7
8
9
10
11
12
13
|
# Sum all values
echo '[1,2,3,4,5]' | jq 'reduce .[] as $x (0; . + $x)'
# 15
# Build a lookup map from an array
echo '[{"id":"u1","name":"alice"},{"id":"u2","name":"bob"}]' \
| jq 'reduce .[] as $item ({}; . + {($item.id): $item.name})'
# {"u1":"alice","u2":"bob"}
# Find the maximum
echo '[3,1,4,1,5,9,2,6]' \
| jq 'reduce .[] as $x (0; if $x > . then $x else . end)'
# 9
|
path and getpath / setpath — Deep Manipulation
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
|
# Get the path to a value
echo '{"a":{"b":{"c":42}}}' | jq 'path(.a.b.c)'
# ["a","b","c"]
# Get value at a path
jq 'getpath(["a","b","c"])'
# 42
# Set value at a path
jq 'setpath(["a","b","c"]; 99)'
# {"a":{"b":{"c":99}}}
# Delete a path
jq 'delpaths([["a","b","c"]])'
# {"a":{"b":{}}}
|
env — Reading Environment Variables
1
2
3
4
5
6
7
|
export MY_USER="alice"
echo '{}' | jq --arg user "$MY_USER" '{user: $user}'
# {"user":"alice"}
# Or use env object directly
echo '{}' | jq '{user: env.MY_USER}'
# {"user":"alice"}
|
Passing Values In — --arg, --argjson, --slurpfile
1
2
3
4
5
6
7
8
9
10
11
12
13
14
|
# Pass a string
jq --arg name "alice" '.users[] | select(.name == $name)'
# Pass a number (argjson parses as JSON)
jq --argjson threshold 80 '.scores[] | select(. > $threshold)'
# Pass a boolean
jq --argjson debug true 'if $debug then . else empty end'
# Pass a file as a JSON value
jq --slurpfile config config.json '.settings + $config[0]'
# Pass a raw string from a file
jq --rawfile template template.txt '{body: $template}'
|
Output Flags
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
|
# Raw output — strip quotes (essential for shell scripting)
echo '"hello world"' | jq -r '.'
# hello world (no quotes)
# Compact output — single line
echo '{"a":1,"b":2}' | jq -c '.'
# {"a":1,"b":2}
# Raw input — treat each line of input as a string
printf 'alice\nbob\ncarol\n' | jq -R '{"name": .}'
# {"name":"alice"}
# {"name":"bob"}
# {"name":"carol"}
# Slurp — read all input into a single array
echo -e '1\n2\n3' | jq -s '.'
# [1,2,3]
# Null input — don't read stdin
jq -n '{generated: true, date: "2026-03-25"}'
# Tab-indented output
jq --tab '.'
# Indent N spaces
jq --indent 2 '.'
# Join output (no newlines between outputs)
echo '[1,2,3]' | jq -j '.[] | tostring + ","'
# 1,2,3,
|
Real-World Recipes
AWS CLI
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
|
# List all EC2 instance IDs and their states
aws ec2 describe-instances \
| jq -r '.Reservations[].Instances[] | "\(.InstanceId)\t\(.State.Name)"'
# Get private IPs of all running instances
aws ec2 describe-instances \
--filters Name=instance-state-name,Values=running \
| jq -r '.Reservations[].Instances[].PrivateIpAddress'
# List S3 buckets created after a date
aws s3api list-buckets \
| jq -r '.Buckets[] | select(.CreationDate > "2025-01-01") | .Name'
# Get all IAM users with their last login
aws iam list-users \
| jq -r '.Users[] | [.UserName, (.PasswordLastUsed // "never")] | @tsv'
# Find security groups with 0.0.0.0/0 ingress
aws ec2 describe-security-groups \
| jq -r '.SecurityGroups[] |
select(.IpPermissions[].IpRanges[]?.CidrIp == "0.0.0.0/0") |
"\(.GroupId)\t\(.GroupName)"'
|
Azure CLI
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
|
# List all VMs with their sizes and locations
az vm list \
| jq -r '.[] | [.name, .location, .hardwareProfile.vmSize] | @tsv'
# Get all resource groups and their tags
az group list \
| jq -r '.[] | "\(.name): \(.tags // {} | to_entries | map("\(.key)=\(.value)") | join(","))"'
# Find all storage accounts without HTTPS enforcement
az storage account list \
| jq -r '.[] | select(.enableHttpsTrafficOnly == false) | .name'
# List all AKS clusters and their Kubernetes versions
az aks list \
| jq -r '.[] | "\(.name)\t\(.kubernetesVersion)\t\(.location)"'
|
kubectl
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
|
# List all pods and their status
kubectl get pods -A -o json \
| jq -r '.items[] | [.metadata.namespace, .metadata.name, .status.phase] | @tsv'
# Find pods not in Running state
kubectl get pods -A -o json \
| jq -r '.items[] | select(.status.phase != "Running") |
"\(.metadata.namespace)/\(.metadata.name): \(.status.phase)"'
# Get container image versions for a deployment
kubectl get deployment myapp -o json \
| jq -r '.spec.template.spec.containers[] | "\(.name): \(.image)"'
# List all services with their ClusterIP and ports
kubectl get services -A -o json \
| jq -r '.items[] | "\(.metadata.namespace)\t\(.metadata.name)\t\(.spec.clusterIP)\t\([.spec.ports[]?.port | tostring] | join(","))"'
# Find all pods using a specific image
kubectl get pods -A -o json \
| jq -r --arg img "nginx" \
'.items[] | select(.spec.containers[].image | startswith($img)) |
"\(.metadata.namespace)/\(.metadata.name)"'
# Get resource requests and limits
kubectl get pods -A -o json | jq -r '
.items[] |
.metadata.name as $pod |
.spec.containers[] |
"\($pod)\t\(.name)\t\(.resources.requests.cpu // "-")\t\(.resources.requests.memory // "-")"'
|
GitHub API / REST APIs
1
2
3
4
5
6
7
8
9
10
11
|
# Get open PRs with author and title
curl -s "https://api.github.com/repos/owner/repo/pulls" \
| jq -r '.[] | "\(.number)\t\(.user.login)\t\(.title)"'
# Count open issues by label
curl -s "https://api.github.com/repos/owner/repo/issues" \
| jq 'group_by(.labels[0].name) | map({label: .[0].labels[0].name, count: length})'
# Get release asset download URLs
curl -s "https://api.github.com/repos/owner/repo/releases/latest" \
| jq -r '.assets[] | "\(.name)\t\(.browser_download_url)"'
|
Docker
1
2
3
4
5
6
7
8
9
10
11
12
13
14
|
# List containers with name, image, and status
docker inspect $(docker ps -q) \
| jq -r '.[] | "\(.Name)\t\(.Config.Image)\t\(.State.Status)"'
# Get exposed ports for all running containers
docker inspect $(docker ps -q) \
| jq -r '.[] | .Name as $name |
.NetworkSettings.Ports | to_entries[] |
select(.value != null) |
"\($name): \(.key) -> \(.value[0].HostPort)"'
# Find containers with no resource limits set
docker inspect $(docker ps -q) \
| jq -r '.[] | select(.HostConfig.Memory == 0) | .Name'
|
Processing Multiple Files
1
2
3
4
5
6
7
8
9
10
|
# Process all JSON files in a directory
jq '.name' *.json
# Slurp multiple files into one array
jq -s '.' file1.json file2.json file3.json
# Process files and include filename context
for f in configs/*.json; do
jq -r --arg file "$f" '"\($file): \(.version)"' "$f"
done
|
Streaming Large Files
For very large JSON files that don’t fit in memory, use --stream:
1
2
3
4
5
|
# Stream a large file — outputs path/value pairs
jq --stream 'select(length == 2)' large.json | head -20
# Extract just the top-level keys without loading the whole thing
jq -n --stream 'first(inputs | select(length == 2 and .[0][0] == "users") | .[1])'
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
|
# TSV output (great for pasting into spreadsheets)
echo '[{"name":"alice","age":30},{"name":"bob","age":25}]' \
| jq -r '.[] | [.name, .age] | @tsv'
# alice 30
# bob 25
# CSV output
jq -r '.[] | [.name, .age] | @csv'
# "alice",30
# "bob",25
# Base64 encode
echo '"hello world"' | jq -r '@base64'
# aGVsbG8gd29ybGQ=
# Base64 decode
echo '"aGVsbG8gd29ybGQ="' | jq -r '@base64d'
# hello world
# URL encode
echo '"hello world & more"' | jq -r '@uri'
# hello%20world%20%26%20more
# HTML encode
echo '"<script>alert(1)</script>"' | jq -r '@html'
# <script>alert(1)</script>
# sh — shell-safe quoting
echo '"user input; rm -rf /"' | jq -r '@sh'
# 'user input; rm -rf /'
|
Using @sh Safely in Scripts
When passing jq output to shell commands, always use @sh or -r with proper quoting:
1
2
3
4
5
6
7
8
9
10
|
# UNSAFE — breaks on spaces/special chars
name=$(echo "$json" | jq '.name')
echo $name
# SAFE — use -r and quote the variable
name=$(echo "$json" | jq -r '.name')
echo "$name"
# SAFE — use @sh for values that go into eval or complex commands
eval "$(echo "$json" | jq -r '@sh "name=\(.name) age=\(.age)"')"
|
Defining Functions
1
2
3
4
5
6
7
|
jq '
def double: . * 2;
def clamp(lo; hi): if . < lo then lo elif . > hi then hi else . end;
[1,2,3,4,5] | map(double) | map(clamp(4; 8))
'
# [4,4,6,8,8]
|
Functions can be stored in a file (~/.jq or a .jq file) and included:
1
2
3
4
5
6
7
8
|
# ~/.jq
def to_gib: . / 1073741824 | (. * 10 | round) / 10;
def humanize_bytes:
if . >= 1073741824 then "\(to_gib) GiB"
elif . >= 1048576 then "\(. / 1048576 | round) MiB"
elif . >= 1024 then "\(. / 1024 | round) KiB"
else "\(.) B"
end;
|
1
2
3
4
5
|
echo '1073741824' | jq 'humanize_bytes'
# "1 GiB"
echo '5368709120' | jq 'humanize_bytes'
# "5 GiB"
|
Practical Script Patterns
Generate a Config File from JSON
1
2
3
4
5
6
7
8
9
10
|
#!/usr/bin/env bash
# Generate nginx upstream block from service registry
services=$(curl -s http://registry/services)
echo "$services" | jq -r '
.[] |
"upstream \(.name) {",
(.instances[] | " server \(.host):\(.port);"),
"}"
'
|
Validate JSON Structure
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
|
validate_config() {
local file="$1"
local errors=0
# Check required fields exist
for field in name version environment; do
if ! jq -e ".$field" "$file" > /dev/null 2>&1; then
echo "ERROR: Missing required field: $field" >&2
((errors++))
fi
done
# Check environment is valid
env=$(jq -r '.environment' "$file")
if [[ ! "$env" =~ ^(dev|staging|production)$ ]]; then
echo "ERROR: Invalid environment: $env" >&2
((errors++))
fi
return $errors
}
|
Merge JSON Objects
1
2
3
4
5
|
# Merge two JSON files, second wins on conflicts
jq -s '.[0] * .[1]' base.json overrides.json
# Deep merge all JSON files in a directory
jq -s 'reduce .[] as $x ({}; . * $x)' configs/*.json
|
Patch a JSON File In-Place
1
2
3
4
5
6
|
# Update a field in a JSON file
tmp=$(mktemp)
jq '.version = "2.0.1"' package.json > "$tmp" && mv "$tmp" package.json
# Add an element to an array
jq '.dependencies += ["newpkg"]' config.json | sponge config.json
|
1
2
3
4
5
6
7
8
9
10
|
# Pretty-print a stream of newline-delimited JSON logs (NDJSON)
tail -f app.log | jq -c 'select(.level == "error") | {ts: .timestamp, msg: .message}'
# Count errors by service in the last 1000 lines
tail -1000 app.log | jq -rs '
[.[] | select(.level == "error")] |
group_by(.service) |
map({service: .[0].service, count: length}) |
sort_by(.count) | reverse[]
'
|
Quick Reference
. Identity (pretty-print)
.foo Field access
.foo.bar Nested field
.foo? Optional field (no error if missing)
.[0] Array index
.[-1] Last element
.[2:5] Slice
.[] Iterate array/object
| Pipe
, Produce multiple outputs
( expr ) Grouping
[expr] Collect into array
{key: expr} Build object
{name} Shorthand: {name: .name}
select(bool) Filter — pass through if true
map(f) Apply f to each element
map_values(f) Apply f to each object value
length Count
keys / values Object keys / values
has("key") Check field existence
type Get type as string
to_entries [{key,value}] pairs
from_entries Reverse of to_entries
with_entries(f) map over {key,value} pairs
sort / sort_by(f) Sort
group_by(f) Group into arrays
unique / unique_by Deduplicate
flatten Flatten nested arrays
add Sum / concatenate array
first / last First / last of stream
min_by / max_by Extremes
if C then A else B Conditional
. // default Alternative (null/false fallback)
not Boolean negation
reduce .[] as $x (init; expr) Fold
@tsv / @csv / @base64 / @uri / @sh Output formats
-r Raw string output
-c Compact output
-n No input
-s Slurp into array
-R Raw string input
--arg name val Bind string variable
--argjson name val Bind JSON variable
jq rewards investment. The first twenty minutes of learning pays off every time you reach for it instead of opening Python for a one-liner. The next step is keeping ~/.jq stocked with your own helper functions — once you’ve written humanize_bytes or to_epoch once, you never write it again.
Comments