At small scale any log store works; at scale the invoice decides, not the feature list. The four dominant logging backends make fundamentally different storage bets that fix their cost-per-GB and what queries are even fast. A deep dive on the index-everything camp versus the cheap-storage camp, and the ingest discipline that beats all of them.
Logging
-
Modern Logging Architecture: Loki, Splunk, Elasticsearch, ClickHouse, and the Cost-Per-GB That Decides Everything -
systemd Deep Dive A thorough guide to systemd: unit file anatomy for Service, Timer, Socket, and Target units, dependency ordering, socket activation, journald log management, journalctl filtering, systemd-analyze for boot profiling, drop-in overrides, and user-mode systemd instances.
-
auditd: Linux's Syscall Logger A practical guide to Linux's audit daemon — how auditd logs security-relevant events at the syscall level, how to write rules that capture what compliance and threat detection need, and how to turn its verbose output into something usable.
-
ClickHouse for Observability: Logs, Metrics, and Traces at Scale How ClickHouse's columnar storage architecture makes it the ideal backend for logs, metrics, and traces at scale — replacing Elasticsearch with a fraction of the resources.
-
Designing for Observability: Building Applications You Can Actually Debug A practical guide to designing applications that are easy to debug in production — structured logging with trace IDs, meaningful metrics, health endpoints, graceful degradation, and the patterns that separate systems you can reason about from ones you can only guess at.
-
OpenTelemetry: One Instrumentation to Rule Them All A deep dive into OpenTelemetry: the data model for traces, metrics, and logs; auto-instrumentation for Go, Python, and Node.js; building a Collector pipeline; and exporting to Jaeger, Prometheus, and Loki.
-
Loki for Log Aggregation: Ship, Query, and Correlate Your Logs A complete technical guide to Grafana Loki — covering architecture, log shipping with Promtail and Alloy, LogQL queries, Grafana integration, alerting rules, and label strategy for homelabbers and DevOps engineers adding logs to their observability stack.
-
Effective Log Management Strategies Logs are cheap to produce and expensive to keep, and most teams get the economics backwards. A practical guide to log management as a discipline: structured logging, log levels that mean something, the cost-and-cardinality model, what to sample or drop before ingest, retention and tiering, PII redaction, and how logs fit alongside metrics and traces.