Trino: One Query Engine Over Everything

The fundamental premise of a data warehouse is that you should move data into one place before you query it. Extract it, transform it, load it — and then run your analytics on the copy. That model made sense when data lived in one or two operational databases and the warehouse was simply a denormalized, indexed replica optimized for reads. It makes progressively less sense as your data landscape fractures: Parquet files on S3, Iceberg tables managed by a lakehouse catalog, PostgreSQL transactional databases, Apache Kafka topics, MongoDB collections, and Elasticsearch indices — each owned by a different team, each updated on a different cadence. ETL-ing all of it into a warehouse is expensive in pipeline engineering, expensive in storage duplication, and introduces latency between the source and the place where analysts can actually query it.

Trino is the answer to that problem, with real teeth behind it. It is a distributed, massively parallel processing (MPP) SQL query engine — not a database. It has no storage of its own. It queries data where it lives, through a connector model that speaks directly to dozens of source systems. A single Trino query can JOIN a dimension table in Postgres against a billion-row fact table in Parquet files on S3 through an Apache Iceberg catalog, return results in seconds, and never write a byte to any intermediate store. That capability is genuinely useful, and also genuinely hard to operate at production scale. This post covers both sides.

Architecture: Coordinator, Workers, and Discovery

Trino follows a classical MPP coordinator-worker topology. The coordinator is the brain: it accepts incoming SQL over JDBC or the HTTP API, parses and validates the query, runs the cost-based optimizer to produce a physical query plan, and then divides that plan into a DAG of stages. It does not process data itself. The workers do the computation: they pull data from connectors, execute operators (filters, projections, joins, aggregations), and stream intermediate results to other workers or back to the coordinator.

                         Client (JDBC / REST)
                                 |
                          ┌──────┴──────┐
                          │ Coordinator │
                          │  - Parse    │
                          │  - Plan     │
                          │  - Schedule │
                          └──────┬──────┘
              ┌──────────────────┼──────────────────┐
              │                  │                  │
        ┌─────┴─────┐     ┌──────┴────┐      ┌──────┴────┐
        │  Worker 1 │     │  Worker 2 │      │  Worker 3 │
        └─────┬─────┘     └─────┬─────┘      └─────┬─────┘
              │                 │                   │
     ┌────────┼──────┐  ┌───────┼──────┐  ┌────────┼──────┐
     │        │      │  │       │      │  │        │      │
  Iceberg  Postgres Kafka  Iceberg  Postgres  Iceberg   MongoDB
  (S3)      (JDBC) (topic) (S3)    (JDBC)    (S3)

Workers register themselves with the coordinator through the discovery service — a lightweight HTTP endpoint (embedded in the coordinator in typical deployments) that workers ping on startup and heartbeat against continuously. The coordinator uses the registry of live workers to schedule tasks and detect failures.

From SQL to Distributed Execution

When you submit a query, the execution pipeline is:

1. Parse and analyze — The coordinator’s SQL parser validates syntax and semantics, resolves table and column references against the connector metadata API, and checks types.
2. Logical planning — The query is turned into a relational algebra tree: scan, filter, project, join, aggregate nodes.
3. Cost-based optimization — The optimizer rewrites the logical plan: join reordering, predicate pushdown into connectors, join distribution strategy selection. Statistics from connector metadata drive this step.
4. Physical plan / stage DAG — The optimizer divides the plan into stages at exchange boundaries. Each stage corresponds to a set of tasks distributed across workers. Data flows between stages via exchanges — HTTP-based streaming buffers, not files on disk (in the default execution mode).
5. Task scheduling — The coordinator assigns tasks to workers based on data locality hints provided by the connector (e.g. which HDFS node holds a given split) and current worker load.
6. Pipelined execution — Workers execute their tasks in a pipelined fashion. A split is the smallest unit of work: for the Hive/Iceberg connector, one split maps to one Parquet file (or a row group within it). Workers process splits, emit result pages (columnar batches of rows), and stream those pages to the downstream stage without waiting for the current stage to complete. This pipelining is what gives Trino low latency on interactive queries — results start flowing to the client before all data has been scanned.

The coordinator polls each task for status and collects the final output stage’s pages, streaming them back to the client.

The Connector Model: The SPI

Everything that makes Trino useful operationally is the Service Provider Interface (SPI) — the connector API. Connectors implement a handful of Java interfaces: ConnectorMetadata (list schemas, tables, columns, statistics), ConnectorSplitManager (enumerate splits for a table scan), ConnectorRecordSetProvider or ConnectorPageSourceProvider (read data for a given split), and optionally ConnectorPageSinkProvider (write data). If a connector implements the write interface, Trino can INSERT INTO and CREATE TABLE AS SELECT through it.

The result is a broad connector ecosystem:

The Iceberg and Delta Lake connectors deserve special mention. They consume the table format’s metadata natively — reading Iceberg’s manifest files to skip data files at the manifest level before a single byte of Parquet is touched, and understanding Delta’s transaction log for snapshot isolation. You get ACID-safe reads of your lakehouse without any additional tooling.

A Federated JOIN in Practice

The canonical Trino use case: a fact table in Iceberg on S3, a dimension table in Postgres.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

-- catalog "iceberg" points to your Iceberg/S3 catalog
-- catalog "pg" points to your PostgreSQL instance
SELECT
    c.customer_name,
    c.region,
    SUM(o.total_amount)  AS revenue,
    COUNT(*)             AS order_count
FROM iceberg.sales.orders AS o
JOIN pg.crm.customers    AS c
    ON o.customer_id = c.id
WHERE o.order_date >= DATE '2026-01-01'
  AND c.region = 'EMEA'
GROUP BY c.customer_name, c.region
ORDER BY revenue DESC
LIMIT 100;

Trino pushes the region = 'EMEA' predicate down to Postgres, fetching only matching customer rows. The order_date >= DATE '2026-01-01' filter is pushed into the Iceberg connector, which uses partition metadata and bloom filters to skip irrelevant Parquet files. The result sets from both connectors are streamed into a distributed hash join on the workers — the smaller Postgres result set is broadcast to all workers as the build side; each worker probes it with its slice of the Iceberg scan. No ETL pipeline, no staging tables, no intermediate storage.

The Cost-Based Optimizer

Trino’s optimizer is a rule-based and cost-based hybrid. Rule-based rewrites happen first: predicate pushdown, projection elimination, constant folding. Then the cost-based optimizer (CBO) uses table statistics — row counts, column NDV (number of distinct values), min/max histograms, and null fractions — to make decisions that depend on data distribution:

Join reordering: With three or more joined tables, the optimizer enumerates join orders and estimates the cost of each based on intermediate cardinalities. Picking the wrong join order on a large fact table can turn a 10-second query into a query that never finishes.

Join distribution strategy: For each join, the optimizer chooses between:

• Broadcast join — the smaller (build) table is replicated to every worker. Each worker probes the full copy. Efficient when one side is small enough to fit in worker memory.
• Partitioned (repartitioned) join — both sides are repartitioned by the join key across workers. Each worker receives a partition of both sides. Scales to arbitrarily large tables, but more expensive due to the repartition shuffle.

Dynamic filtering: At runtime, after the build side of a join is loaded into memory, Trino constructs a bloom filter or range filter over the join key and pushes it back into the probe-side scan. For the Hive/Iceberg connector, this means partitions or files that cannot possibly match any join key are skipped entirely before they are read. Dynamic filtering can reduce scan volume by orders of magnitude on selective lookups against large fact tables.

For any of this to work, statistics must exist and be current. Run ANALYZE iceberg.sales.orders after large loads. For Iceberg tables, Trino can also read statistics from Iceberg’s Puffin statistics files if they are written by a compatible engine. Stale or missing statistics cause the CBO to fall back to heuristic estimates, which often produces suboptimal join orders and unnecessarily expensive shuffle operations.

Memory, Failures, and Fault-Tolerant Execution

This is where Trino’s operational reality diverges sharply from its marketing materials.

Classic Execution Mode

In the default (classic) execution mode, all intermediate query state lives in JVM heap memory on the coordinator and workers. There is no intermediate materialization to disk between stages. A stage’s output pages are buffered in the producing worker’s output buffer and consumed by the downstream stage as fast as the consumer can pull them. When everything goes right, this pipelining is extremely efficient — low latency, no disk I/O bottleneck.

When things go wrong, the consequences are total. A worker process killed by the Linux OOM killer, a JVM GC pause that causes a heartbeat timeout, a hardware failure — any of these causes the coordinator to declare the worker lost and kill every query that had tasks on that worker. There is no partial recovery; the client receives an error and must resubmit. On long-running queries over large datasets — think multi-hour ETL jobs or complex analytical queries scanning terabytes — this failure model means queries routinely fail before they complete on clusters where workers are memory-pressured or where spot/preemptible instances are used.

Trino does have a legacy spill-to-disk mechanism (spill-enabled=true) for specific operators (large hash joins, sorts), but the documentation is blunt about it: “Spill to disk is a legacy functionality of Trino. Consider using fault-tolerant execution.” Spill works at the operator level and does not protect against node failures.

Fault-Tolerant Execution (Project Tardigrade)

Trino’s fault-tolerant execution (FTE) mode, introduced under the name Project Tardigrade and now a first-class feature, fundamentally changes the failure model. The key mechanism is the exchange spool:

Instead of streaming output pages directly between stages over HTTP, FTE materializes each stage’s output to a exchange spool — either a local file system or, in production, an object store (S3, GCS, HDFS, or compatible). The spool is managed by a pluggable Exchange Manager.

The retry policy is configured per cluster:

• retry-policy=QUERY — if a query fails for any reason, the entire query is retried from scratch. Simple but wasteful.
• retry-policy=TASK — if an individual task fails, only that task is retried by picking up its inputs from the exchange spool. This is the correct mode for long ETL jobs on clusters with spot instances.

1
2
3
4

# config.properties — coordinator and workers
retry-policy=TASK
exchange.base-directories=/mnt/trino-exchange-spill
# For S3 exchange spooling, configure an exchange-manager.properties file

The cost is throughput: materializing exchanges to object storage adds latency and I/O compared to pure in-memory streaming. FTE is not for interactive dashboards. It is for long-running ETL and batch analytical jobs on ephemeral infrastructure where tolerating a 5-minute task retry is far preferable to failing a 4-hour job at the 3:50 mark.

When Trino Wins and When It Does Not

Use Trino when:

You want to avoid ETL copies. If the source system already has the data in a usable format (Parquet on S3, an Iceberg table, a Postgres database), querying it directly eliminates the pipeline, the storage cost, and the latency of a nightly load.

You need to query a lakehouse without leaving it. Trino with the Iceberg or Delta Lake connector is the standard SQL-on-lakehouse compute engine for teams not locked into Spark. See Apache Iceberg and the data lakehouse and Delta Lake vs Iceberg vs Hudi.

You need true federation. The Postgres-JOIN-Iceberg query above is not a toy demo; it is a legitimate pattern used by teams that want a single SQL layer over an operational database and a historical fact store without moving either.

Your data scale exceeds single-node tools. For datasets measured in terabytes or petabytes, DuckDB and single-process tools top out. Trino distributes across dozens or hundreds of workers and has been run at multi-petabyte scale at Meta, Netflix, and Lyft.

Ad-hoc exploration over raw lake data. Data scientists and analysts who need to explore unstructured lake data before it is formalized into a warehouse are well-served by a Trino cluster with broad connector coverage.

Do not use Trino when:

You need low-latency serving at high concurrency. Trino is an analytical engine. Query planning alone takes hundreds of milliseconds. It is not suitable as a backend for user-facing dashboards with thousands of concurrent queries and sub-100ms requirements. ClickHouse, Apache Pinot, or a caching layer in front of a warehouse are better fits.

Your data is hot and well-modeled in a warehouse. If you have already loaded, cleaned, and modeled your core metrics into Redshift, BigQuery, or Snowflake, re-querying those sources through Trino adds network hops and loses the warehouse’s internal optimizations (zone maps, materialized views, result caching). Trino wins on federation and lakehouse access; it does not beat a tuned warehouse on data the warehouse already owns.

You need fine-grained access control out of the box. Trino has a built-in security model (file-based and LDAP authentication, system access control), but row-level security, column masking, and attribute-based access control require significant configuration — or Starburst’s commercial additions. In a regulated environment, the governance surface is non-trivial.

Your team cannot absorb the operational weight. Trino is a distributed JVM system. It requires JVM tuning, memory configuration, metastore management, connector configuration, and attention to query concurrency limits. Compared to spinning up a serverless warehouse, the operational cost is real.

Deploying Trino

Cluster Topology

A minimal production Trino cluster is one coordinator plus N workers. The coordinator is single-threaded for query planning and should not be overloaded; it is typically sized at 8–16 vCPUs and 32–64 GB RAM, with the JVM heap at roughly 70% of available RAM. Workers are where the compute happens: more RAM means more concurrent queries and fewer OOM failures. Worker sizes of 64–256 GB RAM are common for analytics workloads.

Workers must not be scheduled on the coordinator host in any but the smallest development setups. The coordinator’s heartbeat processing saturates under load, and collocated workers compete for heap.

Kubernetes with Helm

Trino ships an official Helm chart (helm repo add trino https://trinodb.github.io/charts):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

# values.yaml fragment
coordinator:
  jvm:
    maxHeapSize: "48G"
  config:
    query.max-memory: "40GB"
    query.max-memory-per-node: "4GB"
    query.max-total-memory-per-node: "8GB"

worker:
  replicas: 8
  jvm:
    maxHeapSize: "56G"
  config:
    query.max-memory-per-node: "8GB"
    query.max-total-memory-per-node: "12GB"

additionalCatalogs:
  iceberg: |
    connector.name=iceberg
    hive.metastore=glue
    hive.metastore.glue.region=us-east-1
    fs.native-s3.enabled=true

On Kubernetes, configure workers as a Deployment (or a StatefulSet if you want stable node identities for local disk exchange spooling). The coordinator should be a single-replica Deployment behind a ClusterIP Service.

Catalog Configuration

Each data source is configured as a catalog — a .properties file in the coordinator’s and workers’ etc/catalog/ directory (or injected as a ConfigMap in Kubernetes). The catalog name becomes the first component of a three-part name: catalog.schema.table.

Iceberg catalog on S3 with AWS Glue metastore:

1
2
3
4
5
6
7
8

# etc/catalog/iceberg.properties
connector.name=iceberg
hive.metastore=glue
hive.metastore.glue.region=us-east-1
hive.metastore.glue.catalogid=123456789012
fs.native-s3.enabled=true
iceberg.file-format=PARQUET
iceberg.compression-codec=ZSTD

PostgreSQL catalog:

1
2
3
4
5
6

# etc/catalog/pg.properties
connector.name=postgresql
connection-url=jdbc:postgresql://pg-host:5432/mydb
connection-user=trino_reader
connection-password=<password>
postgresql.include-system-tables=false

Coordinator JVM configuration:

# etc/jvm.config
-server
-Xmx48G
-XX:+UseG1GC
-XX:G1HeapRegionSize=32M
-XX:+UseGCOverheadLimit
-XX:+ExplicitGCInvokesConcurrent
-XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=/var/log/trino/heapdump.hprof
-Djdk.attach.allowAttachSelf=true
-Djdk.nio.maxCachedBufferSize=2000000

The Metastore Dependency

Trino does not discover table schemas on its own for the Hive, Iceberg, and Delta Lake connectors. It requires an external Hive Metastore Service (HMS) or AWS Glue Data Catalog as the source of table metadata: schema names, table locations, partition keys, and SerDe information. This is a hard operational dependency. If the metastore is down, queries fail immediately at the planning phase.

Running an open-source HMS (Apache Hive metastore standalone) means managing a Java service backed by a relational database (typically Postgres or MySQL). Glue is the managed alternative on AWS and is commonly used in cloud deployments. Project Nessie and the Iceberg REST Catalog are gaining traction as metastore-free alternatives for Iceberg-only workloads — Trino’s Iceberg connector supports the REST catalog — but HMS remains the most widely deployed path.

Creating an Iceberg Table Through Trino

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

CREATE SCHEMA IF NOT EXISTS iceberg.analytics
WITH (location = 's3://my-data-lake/analytics/');

CREATE TABLE iceberg.analytics.events (
    event_id     BIGINT,
    user_id      BIGINT,
    event_type   VARCHAR,
    event_ts     TIMESTAMP(6) WITH TIME ZONE,
    properties   MAP(VARCHAR, VARCHAR)
)
WITH (
    format           = 'PARQUET',
    partitioning     = ARRAY['day(event_ts)'],
    sorted_by        = ARRAY['user_id']
);

This creates the Iceberg table metadata in the configured catalog and an empty location in S3. Subsequent INSERT INTO or CREATE TABLE AS SELECT statements write Parquet files and Iceberg manifests to that location.

Trino vs Presto vs Starburst

No discussion of Trino is complete without the fork history and current landscape.

The Split

Presto was created at Facebook around 2012 and open-sourced in 2013. By 2018, disagreements over governance — specifically, Facebook’s assertion of unilateral commit privileges — led the three original creators (Martin Traverso, Dain Sundstrom, David Phillips) to fork the project in January 2019 as the Presto Software Foundation project, colloquially called PrestoSQL. Facebook subsequently donated its branch to the Linux Foundation in September 2019, forming the Presto Foundation around the prestodb repository. In December 2020, PrestoSQL was renamed Trino after the Linux Foundation enforced the Presto trademark.

Since the split, the two projects have diverged significantly. Trino has moved far faster, dropping legacy Raptor storage connectors, replacing the Hive metastore integration with native Iceberg support, and iterating rapidly on the cost-based optimizer, fault-tolerant execution, and new connectors. PrestoDB (maintained primarily by Meta and Ahana, now IBM) remains production-hardened for Meta-scale infrastructure but moves more conservatively.

Starburst

Starburst is the commercial entity founded by Trino’s original creators. Starburst Enterprise is Trino plus a proprietary layer that adds:

• Warp Speed — a caching and indexing layer that sits between Trino workers and object storage, backed by local NVMe SSDs on worker nodes (and as of the 479-e LTS release in February 2026, on the coordinator as well). Starburst claims up to 7x query acceleration for repeated access patterns. Under the hood, it splits Parquet/ORC blocks into indexed segments and tracks access patterns to pre-warm the cache.
• Built-in access control — row-level filtering and column masking configured through a policy UI, suitable for compliance use cases that require auditable data masking.
• Enhanced connectors — proprietary connectors for SAP, Salesforce, and other enterprise sources not covered by the open-source SPI.
• Starburst Galaxy — the hosted SaaS version, removing the operational burden of running the cluster entirely.

If you are on AWS, GCP, or Azure, already managing a Kubernetes cluster, and your use case fits the open-source connector set, the upstream Trino project is the correct choice. If you need the enterprise governance features or Warp Speed performance for a use case where raw Trino on cold S3 is too slow, Starburst Enterprise is worth evaluating — but it comes with per-node licensing costs that can exceed the infrastructure cost itself on large clusters.

Verdict

Trino is genuinely the best open-source tool for federated SQL across a heterogeneous data landscape. The connector SPI gives it coverage no other open-source engine matches. The Iceberg and Delta Lake connectors make it a first-class lakehouse compute engine. The cost-based optimizer, dynamic filtering, and pipelined execution model produce interactive performance on analytical workloads at data volumes that single-node tools cannot approach.

The honest caveats are real too. Classic execution mode’s all-or-nothing memory model means long queries on memory-pressured clusters fail regularly; fault-tolerant execution is the fix but not the default, and it adds operational complexity. The metastore dependency is a non-negotiable operational cost. The JVM tuning curve is steep. High-concurrency, low-latency serving is simply not what Trino is for.

For teams building a modern data platform — lakehouse storage in Iceberg on MinIO or S3, operational databases in Postgres, streaming pipelines through Kafka — Trino is the query federation layer that ties the room together. Load it, tune it carefully, collect table statistics, and configure fault-tolerant execution for your long-running jobs. Treat it as infrastructure, not SaaS magic, and it will repay the investment.

Sources

• Trino concepts — Trino 481 Documentation
• Fault-tolerant execution — Trino 481 Documentation
• Dynamic filtering — Trino 481 Documentation
• Iceberg connector — Trino 481 Documentation
• Delta Lake connector — Trino 481 Documentation
• Spill to disk — Trino 481 Documentation
• Trino on Kubernetes with Helm — Trino 481 Documentation
• Project Tardigrade delivers ETL at Trino speeds — Trino Blog
• We’re rebranding PrestoSQL as Trino — Trino Blog
• PrestoDB vs PrestoSQL vs Trino — Starburst Blog
• The journey from Presto to Trino and Starburst — Starburst Blog
• Starburst Warp Speed — Starburst Enterprise Documentation
• Release 479-e LTS (27 Feb 2026) — Starburst Enterprise
• Trino vs. Presto: History, Differences, and Which to Use in 2026 — Ksolves
• Trino (SQL query engine) — Wikipedia
• Fault-tolerant execution in Trino — Amazon EMR