How Polars — a Rust-backed, Arrow-native DataFrame library — achieves real multi-core performance, a query optimizer, and streaming execution that pandas structurally cannot match, and how to migrate to it without losing your mind.
Data Engineering
-
Polars: The DataFrame That Replaced pandas -
Ray: Distributed Python Without the Pain A deep technical look at Ray's core primitives, scheduling model, high-level ML libraries, KubeRay on Kubernetes, and honest trade-offs versus Dask and Spark.
-
Trino: One Query Engine Over Everything Trino is a distributed MPP SQL engine that queries data where it lives — across object storage, relational databases, and streaming systems — without loading it into a central warehouse first. This guide covers the architecture, connectors, optimizer, failure model, and honest operational realities.
-
Delta Lake vs Iceberg vs Hudi: The Table Format Shootout -
Apache Airflow Deep Dive: DAGs, Executors, and Production Realities -
Apache Spark Fundamentals: RDDs, DataFrames, Catalyst, and When Spark Still Wins -
dbt Core in Practice: Data Modeling That Doesn't Rot A complete hands-on guide to dbt Core — project structure, sources and staging layers, testing data quality, documentation, incremental models, and running dbt in production with Airflow.