LUNAROPS · OPERATIONAL UPLINK 100% UPTIME 1,247d POSTS 893 JEFF.MOON@LUNAROPS.DEV UTC --:--:--

Hash Functions Explained

16 min read Cryptography Deep Dives
cryptographyhashingshablake3securityperformance
Contents

Cryptographic hash functions are the workhorse primitive of modern computing. Every TLS handshake derives its keys with one. Every git commit identifier is one. Every password storage scheme worth using is built on one. Every blockchain block links to its parent through one. Every code-signing scheme depends on them, every digital signature signs them rather than the underlying message, every content-addressed storage system (IPFS, Cas, Nix, deduplicating backups) names files by them. The reason hash functions are everywhere is that they offer a unique combination of guarantees that nothing else provides: take an arbitrary-length input and produce a fixed-length output where the only way to find an input mapping to a specific output is to try inputs more or less at random, and the only way to find two distinct inputs producing the same output is to try about 2^(n/2) random inputs before getting lucky. Those two properties — preimage resistance and collision resistance — are what make a hash a hash. The interesting part is that nobody has proven any specific hash function actually has those properties; we have functions that have resisted attack for decades and inferring that resistance into the future, and we have a graveyard of functions (MD5, SHA-1, RIPEMD, snefru) whose resistance failed. This post walks what a cryptographic hash function actually guarantees, the Merkle-Damgard construction at the heart of MD5/SHA-1/SHA-2 and the sponge construction at the heart of SHA-3, the parallel Merkle tree underneath BLAKE3, why the length-extension attack matters in practice, and the honest 2026 decision tree for which hash to use where.

What a Hash Function Actually Promises

A cryptographic hash function H takes an input of essentially any length and produces a fixed-length output (32 bytes for SHA-256, 64 bytes for SHA-512, 32 or 64 bytes for BLAKE3 depending on configuration). It promises three things that anyone using it has to internalize.

Preimage resistance. Given a hash output h, it should be infeasible to find any input m such that H(m) = h. For a 256-bit hash, this should take about 2^256 work — far beyond anything possible classically. This is what makes password hashes survive being stolen: an attacker who steals H(password) cannot work backward to recover password.

Second-preimage resistance. Given an input m1 and its hash H(m1), it should be infeasible to find a second input m2 (different from m1) such that H(m2) = H(m1). This is what makes digital signatures work: a signature commits to a specific message via its hash, and the signer can trust that no attacker will be able to forge a different message that maps to the same hash.

Collision resistance. It should be infeasible to find any two distinct inputs m1, m2 (the attacker chooses both) such that H(m1) = H(m2). For a 256-bit hash, this should take about 2^128 work because of the birthday paradox — the square root of the output space — which is why 256-bit hashes are the modern floor and 128-bit hashes are deprecated. The collision-resistance guarantee is what makes git’s content-addressed identifiers reliable; if you can find a collision, you can swap one git object for another with the same hash and the difference becomes invisible.

A hash function that loses any of these three properties is broken for the use cases that depend on that property. The history of cryptographic hashing is largely a history of functions losing one of these properties earlier than expected:

  • MD5 (1992): collisions were demonstrated in 2004; today they can be generated in seconds on a phone. Still useful for non-security uses (file integrity against accidental corruption, content-addressed dedup) but completely unsafe for any adversarial application. The Flame malware in 2012 forged a Microsoft code-signing certificate by finding an MD5 collision.
  • SHA-1 (1995): theoretical collision attacks were published in 2005; the first real collision was demonstrated by Google’s SHAttered project in 2017. Git officially deprecated SHA-1 in 2018; signed certificates have been phased out. Still in some legacy uses but should not be used for new work.
  • SHA-2 family (SHA-256, SHA-384, SHA-512) (2001): no significant cryptanalytic break in 25 years. Still the default for most production systems.
  • SHA-3 (Keccak) (2015): no significant cryptanalytic break, and the sponge construction was chosen partly to provide structural diversity from SHA-2.
  • BLAKE2 and BLAKE3 (2012, 2020): no significant cryptanalytic break. Designed for speed.

The honest framing: cryptographic hashes are infrastructure that you trust because they have resisted twenty years of expert attack, not because there is a proof they are secure. Choose ones with the longest unbroken track record for the most consequential use cases.

The Merkle-Damgard Construction

The hash functions of the MD5/SHA-1/SHA-2 lineage all share the same basic structure, called the Merkle-Damgard construction after its inventors. The idea is:

  1. Take the input message, pad it (with a 1-bit, zeroes, and the message length encoded at the end) to a multiple of some block size.
  2. Initialize an internal state (h0) to some fixed initial value.
  3. For each block of the padded message, run a compression function that takes the current state and the message block and produces the next state.
  4. After all blocks have been processed, output the final state as the hash.
   MERKLE-DAMGARD HASH (SHA-256 style)

   message m, padded to multiples of 512 bits:    [m1] [m2] [m3] ... [mN]

   initial state h0 (fixed constant)
        │
        ▼
   ┌────────┐                                     compress function f:
   │ f(h0,  │───── h1
   │   m1)  │       │
   └────────┘       ▼
                  ┌────────┐                       takes 256-bit state +
                  │ f(h1,  │───── h2               512-bit message block,
                  │   m2)  │       │               produces 256-bit new state.
                  └────────┘       ▼
                                 ┌────────┐
                                 │ f(h2,  │───── h3
                                 │   m3)  │       │
                                 └────────┘       ▼
                                                ...
                                                  │
                                                  ▼
                                              ┌────────┐
                                              │ f(h_{N-1}│───── h_N = HASH
                                              │  , m_N)  │
                                              └────────┘

The compression function is where the cryptography happens. SHA-256’s compression function takes a 256-bit state and a 512-bit message block, runs 64 rounds of bit shuffling and mixing with addition modulo 2^32, and produces a new 256-bit state. The internal rounds are carefully designed to spread input changes across the whole state quickly — flipping any single input bit changes about half the output bits, which is the avalanche property.

Merkle-Damgard has a beautiful property: if the compression function is collision-resistant, then the whole hash is collision-resistant. The construction inherits security from its building block. It also has a famous weakness that we will come to: the length-extension attack.

The Length-Extension Attack

The Merkle-Damgard construction’s weakness is structural, not a flaw in any particular compression function. The hash output is the final internal state h_N. If you know h_N for some unknown input m, you can take h_N as your starting state, hash an additional message block m’, and compute the hash of m || padding || m' without knowing m. The attacker can extend a message they do not know and compute the new hash as if they hashed the whole thing themselves.

This is the length-extension attack and it has bitten real systems. The classic abuse is in poorly-designed authentication tokens. If a server computes an authentication token as H(secret || message) — concatenating a server-only secret with the message and hashing the result — an attacker can extend a valid (message, token) pair into (message || extension, new_token) without knowing the secret. This was a real vulnerability in early Flickr’s API authentication, in early Amazon S3’s, in multiple bespoke API designs.

The fix at the application level is to use a properly designed message authentication code: HMAC (HMAC(key, message) = H(key XOR opad || H(key XOR ipad || message))) sandwiches the message between two key-derived blocks, structurally preventing length extension. HMAC-SHA-256 is what every TLS, SSH, and TLS-based protocol uses for message authentication; never roll your own keyed hash by concatenating key and message.

The fix at the hash function level is to design the function so the output is not the entire internal state. SHA-2’s truncated variants (SHA-512/256, which runs SHA-512 internally and outputs only 256 bits) are immune to length extension because the truncation hides part of the state. SHA-3 is immune by construction because of its sponge architecture.

SHA-3 and the Sponge Construction

NIST ran a public competition from 2008 to 2012 to standardize a successor to SHA-2, motivated partly by the SHA-1 break (so the family had vulnerabilities) and partly to provide cryptographic diversity (so a future break in SHA-2 would not invalidate everything). Out of 64 submissions, the winner was Keccak, designed by Bertoni, Daemen, Peeters, and Van Assche, standardized as SHA-3 in FIPS 202 (August 2015).

SHA-3 is built on a fundamentally different construction called the sponge. The sponge has two phases:

  1. Absorb phase: take the message in blocks, and for each block, XOR it into part of a much larger internal state, then run a permutation (essentially a fixed bit-shuffling function) over the whole state.
  2. Squeeze phase: read out part of the state as output, run the permutation again, read out more state, and so on until you have as much output as you want.

The internal state in Keccak is 1600 bits (a 5x5x64 array). The bit-shuffling permutation (Keccak-f[1600]) is what does the cryptographic work; it runs 24 rounds of bit rotation, XOR, and parity-based mixing. For SHA3-256, the absorb phase mixes in 1088-bit blocks of input and outputs 256 bits in one squeeze.

   SPONGE CONSTRUCTION (SHA-3 / Keccak)

         message (padded)
            │
            ▼ XOR into "rate" portion of state
   ┌─────────────────────────┐
   │  STATE: rate || capacity│   1600 bits total
   └─────────────────────────┘
            │
            ▼ apply permutation f
   ┌─────────────────────────┐
   │  STATE'                 │
   └─────────────────────────┘
            │
            ▼ next block XOR'd in
   ...
            │
            ▼ after all input absorbed
   ┌─────────────────────────┐
   │  FINAL STATE            │
   └─────────────────────────┘
            │
            ▼ squeeze: read rate, run f, repeat
        OUTPUT BITS

   Capacity portion never appears in output.
   No length-extension attack possible: only rate is exposed.
   Output length is variable in principle; SHA-3 fixes it at 224, 256, 384, or 512.

The sponge avoids length-extension by hiding the “capacity” portion of the state from the output. It also natively supports variable-length output, which is what enables SHAKE (Secure Hash Algorithm Keccak — extendable-output functions) to produce any output length you want. SHAKE128 and SHAKE256 are the standardized variable-output variants and are useful for building higher-level cryptographic constructions.

The honest weakness of SHA-3: in pure software on a general-purpose CPU, it is slower than SHA-256. The Keccak permutation does not map as cleanly to common 32-bit and 64-bit CPU instructions as SHA-256’s compression function does. Modern CPUs with SHA-3 hardware acceleration (ARM v8.4-A and later, some Intel chips) close this gap, but on commodity hardware without acceleration, SHA-3 is typically 30-50% slower than SHA-2.

This is the reason SHA-3 has not displaced SHA-2 in most production work. SHA-256 remains the default of TLS, SSH host key fingerprinting, git, DNSSEC, Bitcoin, code signing, and most password-storage key-derivation functions like Argon2. SHA-3 is used where structural diversity from SHA-2 is valued (some government applications, some next-generation signature schemes like SPHINCS+) and where SHAKE’s variable output is useful.

BLAKE3 and the Parallel Merkle Tree

BLAKE3 was published in 2020 by Jack O’Connor, Jean-Philippe Aumasson, Samuel Neves, and Zooko Wilcox-O’Hearn. It descends from BLAKE2 (which descended from BLAKE, a SHA-3 competition finalist), and its design priority is speed without sacrificing security. The result is a hash function that is faster than SHA-256, faster than BLAKE2, much faster than SHA-3 in software, and structurally different from all of them.

The key BLAKE3 innovation is Merkle tree parallelism. Instead of processing the input serially block by block, BLAKE3 splits the input into 1024-byte chunks, hashes each chunk independently in parallel, then combines the chunk hashes pairwise up a Merkle tree to produce the final output. This is embarrassingly parallel: 4 CPU cores can hash a large file roughly 4x faster than 1 core, and SIMD instructions (AVX-512, NEON) can hash within a single core dramatically faster than scalar code. On a modern desktop CPU, BLAKE3 hashes at 6-10 GB/s for large inputs; SHA-256 hashes at 0.5-2 GB/s. The gap is large enough to matter for content-addressed storage, dedup engines, and any application doing heavy hashing.

BLAKE3 also supports:

  • Variable-length output (like SHAKE).
  • Keyed mode for built-in MAC functionality (no need for HMAC sandwiching).
  • Key derivation mode for deterministic key generation from a master key.

The honest concerns about BLAKE3 are essentially about adoption rather than the algorithm itself. BLAKE3 is newer than SHA-2 and SHA-3, has not been through a multi-decade adversarial review, and is not standardized by NIST or other major standards bodies. The security analysis to date is good but limited. Its track record is six years rather than two decades. For a brand-new internal application where you control the stack and care about hashing throughput, BLAKE3 is an excellent choice; for protocols that need cross-vendor interop or that must withstand audits demanding standardized crypto, SHA-256 or SHA-3 is the safer choice.

Property SHA-256 SHA-3 (SHA3-256) BLAKE3
Construction Merkle-Damgard Sponge Merkle tree
Standardized NIST FIPS 180-4 NIST FIPS 202 IETF informational, not NIST
Output size 256 bits 256 bits (variable via SHAKE) Variable (default 256)
Length-extension safe No (use HMAC) Yes Yes
Software throughput ~1-2 GB/s ~0.5-1 GB/s ~6-10 GB/s
Hardware acceleration Common (SHA-NI on x86, ARM v8) Some (ARM v8.4-A, recent x86) SIMD-based, broad CPU support
Track record 25 years unbroken 11 years unbroken 6 years unbroken
Native MAC support No (use HMAC) No (use HMAC or KMAC) Yes (built-in keyed mode)
Parallel hashing No No Yes (Merkle tree)
Used in TLS, SSH, git, code signing, password KDF SPHINCS+, some gov’t, niche Newer content-addressed systems, dedup

Choosing a Hash Function in 2026

A practical decision tree:

For protocol-level hashing where interop matters (TLS, certificate fingerprints, JWT signing, signed commits across services, code signing): use SHA-256. It is the boring, universally-supported default that everyone has and nobody will object to. Use HMAC-SHA-256 for keyed authentication.

For password hashing: use Argon2id (winner of the Password Hashing Competition) or scrypt or bcrypt with appropriate cost parameters. Do NOT use SHA-256 directly; the speed of a raw cryptographic hash is exactly the wrong property for password storage, where you want it to be slow on purpose to make brute-force attacks expensive. Argon2id is the modern recommendation.

For content-addressed storage where you control the implementation (internal deduplication, an internal CAS, a backup engine): use BLAKE3. Its throughput advantage is real and matters at scale. Some users still prefer SHA-256 for the longer track record; if you are storing data that must remain intact for decades, the conservative choice is SHA-256, but BLAKE3 is the modern default for new implementations.

For digital signatures over messages (the message-to-hash step before signing with Ed25519 or ECDSA): the signing protocol usually specifies the hash. Ed25519 internally uses SHA-512. ECDSA-P256 uses SHA-256. Don’t change this; use the protocol’s standard.

For applications needing variable-length output (key derivation, deterministic random bit generation, building KDFs): use SHAKE128 or SHAKE256 from the SHA-3 family. BLAKE3 also supports variable-length output natively.

For compliance-driven environments where NIST-standardized algorithms are mandated: use SHA-256, SHA-384, or SHA3-256. Avoid BLAKE3 in environments requiring FIPS 140 validation; BLAKE3 is not on the approved list.

For non-cryptographic uses where you just need a fast checksum or hash table index, and adversarial collision attacks are not a concern: use xxHash or MurmurHash3 or CityHash — these are not cryptographic hashes but are dramatically faster (50+ GB/s) and are the right tool when you do not need adversarial security. Don’t use SHA or BLAKE for hash table lookups; that is a category error.

For long-term cryptographic diversity — you want a fallback if SHA-2 is broken: signing with SHA-512/256 or SHA-3 alongside SHA-256 hedges the bet. Most working engineers do not need to do this; if you do, it is usually a regulatory requirement.

What This All Means Operationally

A few practical takeaways for working engineers in 2026:

  • The SHA-256 you call from your standard library, from openssl, from sha256sum, from git, from Python’s hashlib, from Rust’s Sha256::digest — all the same algorithm with hardware acceleration on virtually any modern CPU. The CPU cost of a hash operation is a few cycles per byte; for almost any application, it is free.

  • Never compose a hash and a secret naively (H(secret || message)); always use HMAC or a properly designed authenticated construction. The length-extension attack is decades old and still ships.

  • Never use a raw cryptographic hash for password storage. Use Argon2id with the recommended parameters (Iteration count and memory cost tuned for current hardware; OWASP publishes updated recommendations annually).

  • For file integrity checks where adversarial collision is plausible (verifying downloads, signing release binaries), use SHA-256 minimum, and prefer hash and signature rather than hash alone.

  • When designing a protocol, specify the hash algorithm explicitly and design for algorithm agility (the ability to swap hashes if one breaks). The HTTP Want-Digest and Digest headers, the Subresource Integrity standard’s sha256-/sha384-/sha512- prefixes, the way TLS 1.3 negotiates ciphersuites — all good examples of algorithm-flexible design.

  • BLAKE3 throughput differences become noticeable above a few megabytes of input. For small inputs (less than a few KB), the parallelism does not pay off and SHA-256 with hardware acceleration is similar. The big BLAKE3 wins are on file-scale hashing.

Verdict

Cryptographic hash functions are the invisible primitive holding modern computing together, and almost every interesting decision about which one to use traces back to a handful of fundamentals: what construction the function uses, what guarantees it provides, what its software performance is on the platform you actually run, and how long its track record is. The Merkle-Damgard family that includes SHA-256 remains the default of TLS, SSH, git, certificate signing, and most production work, both because it is fast on commodity hardware with widespread SHA-NI acceleration and because its 25-year unbroken track record makes it the conservative engineering choice; the length-extension attack is a real concern but is solved at the application layer with HMAC, never by rolling a custom keyed hash. SHA-3’s sponge construction was chosen by NIST for structural diversity from SHA-2 and is immune to length extension by design, but is slower than SHA-2 in software, which is why it has not displaced SHA-2 except in niches that value the cryptographic diversity or use the SHAKE variable-output capability. BLAKE3’s Merkle-tree parallelism delivers dramatically higher throughput than either SHA family for large inputs and is increasingly the right choice for internal content-addressed storage where you control the stack and value the speed, with the honest caveat that its track record is shorter and it is not NIST-standardized. The practical 2026 default for most working engineers is to use SHA-256 with hardware acceleration where it exists, HMAC-SHA-256 for keyed authentication, Argon2id for passwords, and BLAKE3 where hashing throughput matters and you can pick the algorithm yourself. Choose by both the cryptographic guarantees you need and the operational performance you can verify on real workloads, never roll your own keyed construction, and treat the hash family like any other infrastructure — boring, well-understood, and load-bearing for everything sitting on top of it.

Sources

Comments