Distributed SQL: CockroachDB, TiDB, and YugabyteDB

June 3, 2026 11 min read Data & Databases

distributed-sqlcockroachdbtidbyugabytedbdatabasesdistributed-systems

Contents

For two decades the database world told you to pick two of three: SQL with its joins and transactions, horizontal scale across many machines, or strong consistency that never lies to you. Relational databases gave you SQL and consistency on one big box; the NoSQL wave gave you scale by throwing out SQL and relaxing consistency. The thing nobody could sell you was all three at once — a database you could shard across a dozen machines, lose one to a backhoe, and still run BEGIN; ... COMMIT; against, with serializable guarantees, as if nothing happened.

Distributed SQL — the category also called NewSQL — is the claim that you no longer have to choose. CockroachDB, TiDB, and YugabyteDB are three open-source databases that horizontally scale writes across nodes, survive machine and even whole-region failures with zero data loss, and still speak SQL over the Postgres or MySQL wire protocol. They are genuinely impressive, they descend from the same Google research, and they are the right answer far less often than their marketing implies. This post is about how they pull off the trick, where the three differ, the tax you pay for the magic, and the section every honest comparison buries: when boring single-node Postgres is still the correct, adult choice.

The shared blueprint: ranges, Raft, and the Spanner lineage

The three databases differ in important ways, but underneath they are remarkably alike, because they all descend from the same paper: Google’s Spanner. CockroachDB was founded by ex-Google engineers who wanted an open-source Spanner; YugabyteDB’s founders came from Facebook’s Cassandra and HBase teams with the same goal; TiDB’s PingCAP built to the Spanner/F1 design. So the core architecture rhymes across all three.

The shared mechanism, in one picture: a logical table is chopped into contiguous chunks of key space — CockroachDB calls them ranges (around 512 MB each) — and each chunk is independently replicated by its own Raft consensus group across multiple nodes.

   one logical table (sorted by key)
   ┌──────────┬──────────┬──────────┬──────────┐
   │ range 1  │ range 2  │ range 3  │   ...     │   each ~512 MB
   └────┬─────┴────┬─────┴────┬─────┴───────────┘
        │          │          │   each range = one independent Raft group
        ▼          ▼          ▼
   range 2 replicated across nodes:
        nodeA [leader]    ─┐
        nodeB [follower]   ├─  a write commits when a MAJORITY acks it
        nodeC [follower]  ─┘   (so any one node can die with no data loss)

Two ideas from elsewhere on this blog are the entire foundation here. The replication is Raft consensus: each range elects a leader, the leader proposes every write to its followers, and the write commits only once a majority has durably accepted it — which is exactly why a node (or with enough replicas, a whole region) can vanish without losing a committed transaction. And the guarantees they offer sit on the CAP and consistency spectrum at the strong end: these are CP systems that choose consistency over availability when a partition forces the choice, using Raft majorities and (in Spanner’s case, hardware clocks; in these, hybrid logical clocks) to provide serializable or near-serializable isolation. Storage underneath is an LSM-tree engine in the RocksDB lineage (CockroachDB’s Pebble, TiKV, YugabyteDB’s DocDB). Same bones, three bodies.

How the three actually differ

If the blueprint is shared, the architecture on top of it is where you choose. The cleanest way to see it:

	CockroachDB	TiDB	YugabyteDB
Architecture	Monolithic: SQL directly on a distributed KV store, no external deps	Disaggregated: stateless SQL nodes (TiDB) + storage (TiKV) + placement driver (PD)	Postgres query layer reused on a distributed DocDB storage layer
Wire compatibility	PostgreSQL	MySQL	PostgreSQL (YSQL) and Cassandra (YCQL)
Default isolation	Serializable (strongest)	Snapshot / repeatable-read (serializable optional)	Serializable or snapshot
Standout feature	First-class multi-region SQL & survival goals	HTAP: TiFlash columnar replicas for analytics on live data	Reuses real PostgreSQL code for deep PG compatibility; dual API
Written in	Go	Rust (TiKV) + Go (TiDB)	C++

The personalities that fall out of that:

CockroachDB feels the most like “Postgres that happens to span data centers.” It is one tightly-integrated binary with no moving parts to assemble — the SQL layer sits straight on the distributed key-value store. It defaults to serializable isolation, the strongest there is, which is the safest behavior and also the reason its writes carry a touch more coordination overhead than the others. Its multi-region story (next section) is the most mature and SQL-native of the three.
TiDB splits the system into pieces on purpose: stateless TiDB servers parse SQL and plan queries, TiKV stores the data as Raft-replicated key-value, and the Placement Driver (PD) manages metadata and rebalancing. The headline differentiator is HTAP — TiDB can keep a columnar replica (TiFlash) of your data alongside the row store, so you run transactional and analytical queries on the same live data without an ETL pipeline. It speaks the MySQL protocol, which matters if you are migrating off MySQL.
YugabyteDB made the boldest compatibility bet: instead of reimplementing a Postgres-like SQL layer, YSQL reuses the actual PostgreSQL query-processing code on top of its distributed DocDB storage. That buys unusually deep Postgres feature compatibility. It also exposes a second, Cassandra-compatible API (YCQL) for teams that want a wide-column model on the same cluster.

Multi-region survival — and the latency tax nobody puts on the slide

The reason you tolerate this much machinery is survival: the database keeps serving, with no data loss, through failures that would take a single Postgres offline. But survival is not free, and the cost is the most important thing to understand before you adopt any of these.

The clearest framing is CockroachDB’s survival goals, an explicit SQL-level choice:

Zone survival — the cluster tolerates losing an availability zone. Achievable within a single region.
Region survival — the cluster tolerates losing an entire region with zero data loss. This requires at least three regions, because a Raft majority has to survive one whole region disappearing.

ALTER DATABASE app SET PRIMARY REGION "us-east1";
ALTER DATABASE app ADD REGION "us-west1";
ALTER DATABASE app ADD REGION "europe-west1";
ALTER DATABASE app SURVIVE REGION FAILURE;          -- the strong goal

ALTER TABLE users SET LOCALITY REGIONAL BY ROW;     -- pin each row to a home region

Here is the tax. Raft commits a write only when a majority of replicas acknowledge it. If those replicas are spread across regions for region survival, every write must make a cross-region round trip to reach majority — and cross-region latency is tens to hundreds of milliseconds, set by the speed of light, not by your hardware. So the stronger your survival goal, the higher your write latency. A database configured to survive a region failure pays a real, physics-bounded latency penalty on every write that needs that guarantee. You cannot buy your way out of it; it is the cost of agreement across distance.

The escape hatch is data locality, and it is where distributed SQL earns its keep for the right workload. Features like CockroachDB’s REGIONAL BY ROW pin each row to a home region, so a European user’s data lives in europe-west1 and their writes commit at local latency, while still being globally accessible and surviving a regional outage. This is also the compliance story — data residency (GDPR-style “EU data stays in the EU”) becomes a table property rather than an application nightmare. YugabyteDB offers analogous geo-partitioning; the principle is universal: keep the Raft quorum for a piece of data physically close to the users who write it. Get locality right and the latency tax mostly disappears for local access; get it wrong and every write pays the cross-planet toll.

Online schema changes: ALTER without the lock

A second thing distributed SQL gets right, and a real pain point on traditional databases, is online schema changes. On a big single-node database, an ALTER TABLE that rewrites or adds a constraint can lock the table or hold it for an uncomfortable window — the whole reason the migrations-without-downtime playbook exists.

CockroachDB (and the others, to varying degrees) implement Google’s F1 online schema change protocol: a schema change rolls through a series of intermediate, mutually-compatible states (DELETE ONLY, WRITE ONLY, then public), with index backfills happening asynchronously in the background. At no point do two nodes hold incompatible views of the schema, and the table keeps serving reads and writes throughout. You ALTER TABLE ... ADD COLUMN on a multi-terabyte distributed table and the application never notices. This is genuinely better than the single-node experience, and it is one of the under-sold reasons teams adopt these systems — operating schema evolution on a live, large database stops being a scheduled-downtime event.

The multi-region caveat is real, though: a schema change in a region-surviving deployment still has to coordinate across regions, so it inherits the same latency physics as everything else. Online, yes; instantaneous across a planet, no.

The honest head-to-head

No universal winner; it depends on what you are optimizing and what you are migrating from.

If you want…	Reach for
The most Postgres-like single-binary experience and the best multi-region SQL	CockroachDB
The strongest default safety (serializable isolation)	CockroachDB
Transactional + analytical (HTAP) on one live dataset	TiDB (TiFlash)
To migrate off MySQL with minimal app changes	TiDB
The deepest PostgreSQL feature compatibility	YugabyteDB (reuses PG code)
A Cassandra-style wide-column API alongside SQL	YugabyteDB (YCQL)
Raw write throughput	TiDB tends to lead; CockroachDB trades some throughput for serializable isolation

All three are mature, open-source, Kubernetes-friendly, and production-proven at real companies. The choice is rarely “which is best” and almost always “which fits my existing wire protocol, my consistency requirements, and whether I need HTAP.” Migrating from MySQL? Start with TiDB. Want serializable-by-default and the cleanest geo-SQL? CockroachDB. Want maximal Postgres compatibility or a dual API? YugabyteDB.

When boring single-node Postgres still wins

This is the section the vendors will not write, so here it is plainly: most applications do not need distributed SQL, and adopting it when you do not is a costly mistake.

A single, well-tuned PostgreSQL instance on modern hardware handles a staggering amount of load — tens of thousands of transactions per second, terabytes of data — and with a streaming replica and failover (the Postgres replication-and-HA setup) it tolerates node failure too. Against that baseline, distributed SQL asks you to pay three real prices:

Latency. Every write that needs a quorum pays a cross-node (and possibly cross-region) round trip. A single-node Postgres commit is a local fsync; a distributed commit is a network negotiation. For a low-latency, single-region OLTP app, distributed SQL is often slower per transaction, not faster.
Operational complexity. You are now running a distributed system — more nodes, rebalancing, clock management, multi-region topology, and a much larger surface to understand when something is wrong. That is real headcount and real on-call weight.
Cost. Three-plus nodes (or three-plus regions) running continuously is a bigger bill than one box and a replica.

You should reach for distributed SQL when you have a specific requirement that single-node Postgres genuinely cannot meet:

Write throughput beyond what one machine can serve — you have actually saturated a big Postgres primary’s write path, not merely worried that you might.
Always-on with automatic survival of node or region failure, where even a brief failover window or any data loss is unacceptable.
Geo-distribution for latency or compliance — global users needing local write latency, or data-residency laws that require rows to physically live in specific regions.
Horizontal scale you would otherwise build by hand with sharding logic in your application — distributed SQL does that sharding for you, transparently.

If none of those is a hard requirement you can articulate, the correct architecture is almost certainly a single Postgres with a read replica, and the engineering energy you would have spent operating a distributed database is better spent elsewhere. The mature move is to start boring and migrate to distributed SQL when a real ceiling forces it — these databases speak the Postgres/MySQL wire protocol precisely so that migration stays possible later. Scale is a problem you earn; do not pre-pay for it.

The verdict

Distributed SQL is a genuine engineering achievement: CockroachDB, TiDB, and YugabyteDB really do deliver SQL, horizontal scale, and survive-a-node-dying strong consistency together, dissolving a trade-off that defined databases for twenty years. They do it with the same Spanner-derived recipe — key-space ranges, each its own Raft group — and they differ mostly in what sits on top: CockroachDB the integrated Postgres-like with serializable safety and the best geo-SQL, TiDB the disaggregated MySQL-compatible HTAP system, YugabyteDB the deep-Postgres-compatibility dual-API option. Choose among them by wire protocol, consistency needs, and whether you want HTAP.

But the most useful thing this post can tell you is to check, honestly, whether you need any of them at all. The latency tax and operational weight are real and physics-bounded, and a single Postgres serves the overwhelming majority of applications with room to spare. Reach for distributed SQL when you have a concrete requirement it uniquely satisfies — write scale past one node, region-failure survival, or geo-partitioned compliance — and reach for boring Postgres, gratefully, every other time.