ClickHouse columnar storage, the MergeTree engine family, primary key vs sorting key, materialized views for pre-aggregation, query profiling with EXPLAIN, replication with ClickHouse Keeper, distributed tables for sharding, and when to choose ClickHouse over PostgreSQL.
Data-Engineering
-
ClickHouse for Analytics Workloads -
Kafka Connect Deep Dive A comprehensive guide to Kafka Connect — connector architecture, Debezium CDC, Single Message Transforms, exactly-once delivery, schema evolution with Schema Registry, and running at scale on Kubernetes with Strimzi.
-
Delta Lake vs Iceberg vs Hudi: The Table Format Shootout A clear-eyed comparison of the three open lakehouse table formats — Delta Lake, Apache Iceberg, and Hudi — covering ACID transactions, schema evolution, time travel, and which engine ecosystems back each one.
-
Apache Airflow Deep Dive: DAGs, Executors, and Production Realities A production-focused guide to Apache Airflow — how DAGs, schedulers, and executors really work, where Airflow shines and where it hurts, and what you need to know to run it reliably rather than inherit it and suffer.
-
Apache Spark Fundamentals: RDDs, DataFrames, Catalyst, and When Spark Still Wins How Apache Spark actually works under the hood — RDDs, DataFrames, the Catalyst optimizer, and the execution model — and an honest look at when Spark still wins versus DuckDB, Trino, Polars, and Flink.
-
Apache Iceberg: Table Format for the Data Lakehouse How Apache Iceberg brings ACID transactions, schema evolution, hidden partitioning, and time travel to object storage — with Spark, Trino, DuckDB, and Flink.
-
Building Internal Data Platforms: The Modern Data Stack in Practice A practical guide to building an internal data platform — combining dbt, Airflow, data warehouses, data contracts, and data quality checks into a coherent system your engineers and analysts actually want to use.
-
Change Data Capture with Debezium: Streaming Database Changes in Real Time A comprehensive guide to Change Data Capture with Debezium—covering how CDC works at the database level, setting up connectors for PostgreSQL and MySQL, routing events through Kafka, handling schema evolution, transforming records with SMTs, and building production-grade CDC pipelines.
-
dbt Core in Practice: Data Modeling That Doesn't Rot A complete hands-on guide to dbt Core — project structure, sources and staging layers, testing data quality, documentation, incremental models, and running dbt in production with Airflow.
-
DuckDB for Analytics Engineers: In-Process OLAP That Replaces Your Spark Cluster A comprehensive guide to DuckDB—covering its columnar vectorized execution engine, querying Parquet and S3 directly, replacing heavy Spark jobs for medium data workloads, Python and R integration, performance tuning, and when to use DuckDB vs alternatives.
-
Real-Time Streaming with Apache Flink: Stateful Stream Processing at Scale A deep dive into Apache Flink — stateful stream processing, windowing, exactly-once semantics, checkpointing, and deploying production Flink jobs on Kubernetes.
-
Apache Kafka Deep Dive A comprehensive guide to Apache Kafka — how it works internally, producers and consumers, partitions and consumer groups, exactly-once semantics, schema management, and running Kafka reliably in Kubernetes with Strimzi.
-
Data Pipeline Patterns: ETL vs ELT, Streaming, Batch Processing, and Orchestration A practical guide to data pipeline architecture — ETL vs ELT trade-offs, streaming with Kafka and Flink, batch transformation with dbt, and orchestration with Airflow and Dagster.