LUNAROPS · OPERATIONAL UPLINK 100% UPTIME 1,247d POSTS 893 JEFF.MOON@LUNAROPS.DEV UTC --:--:--

Audio Codecs Compared

17 min read Audio Engineering
audiocodecsopusaacmp3flacmqapsychoacoustics
Contents

The honest summary of two decades of audio compression is uncomfortable for the people who spend the most money on it: for music, a competent lossy codec at a modest bitrate is indistinguishable from the original source in a properly controlled blind test, and almost everyone who claims otherwise fails that test the moment they cannot see the file’s label. This is not an opinion. It is the consistent result of ABX listening trials run by the people who actually build and tune these encoders, and it has held across MP3, AAC, and Opus as each one got better. The interesting questions in audio codecs are therefore not “which one sounds best” — at sane bitrates the answer for most listeners on most material is “you cannot tell” — but rather which one is free of patents, which one survives at low bitrate where the differences finally become audible, which one your streaming service actually ships, and why an entire wing of the industry spent the late 2010s selling a “lossless-ish” format called MQA that turned out to be neither lossless nor necessary. This post walks the real engineering — psychoacoustic masking, the encode pipeline, the lossless math — and then uses ABX evidence to separate the differences that exist from the ones that are sold to you.

Lossy vs Lossless: The Fundamental Split

Every audio codec falls into one of two categories, and the distinction is absolute rather than a spectrum. A lossless codec compresses the audio and then reconstructs the original samples bit-for-bit on decode. Decode a FLAC file, and you get back exactly the PCM that went in — every sample identical, verifiable with a checksum. A lossy codec throws information away permanently to achieve far smaller files, and the decoded output is mathematically different from the source. The art of a lossy codec is throwing away information you cannot hear.

The size difference is the whole reason lossy exists. Uncompressed CD-quality stereo audio (16-bit, 44.1 kHz) runs at 1,411 kbps — roughly 10 MB per minute. Lossless compression gets you down to about 700-850 kbps on typical music, a 40-60% reduction with zero quality cost. Lossy codecs reach transparency at 128-256 kbps and remain usable down to 64 kbps and below, a 10-to-20x reduction. That gap is why streaming, mobile, and broadcast run on lossy formats, and why lossless remains a niche for archival, mastering, and the audiophile market.

The critical and frequently misunderstood point: lossless does not sound better than transparent lossy. It sounds identical, because transparent lossy is, by definition, audibly identical to the lossless source. Lossless guarantees you can re-encode, archive, and process the file without generational loss. It does not guarantee, or even imply, an audible improvement on playback for a file that was already transparent. The value of lossless is provenance and editability, not a richer listening experience.

Psychoacoustics: How Lossy Codecs Decide What to Delete

A lossy encoder is not guessing at what to remove. It runs a psychoacoustic model — a mathematical description of the limits of human hearing — and uses it to identify which parts of the signal are inaudible because they are masked by louder parts nearby. Two masking phenomena do most of the work.

Frequency masking (simultaneous masking): a loud tone raises the threshold of audibility for quieter sounds at nearby frequencies. If a strong 1 kHz tone is present, a quiet tone at 1.1 kHz that would be audible in silence becomes inaudible — the ear’s frequency resolution is finite and the loud neighbor swamps it. The encoder can spend zero bits on that masked tone because no listener can hear it. The ear’s frequency axis is organized into roughly 24 critical bands (the Bark scale), and masking is computed band by band: within and adjacent to each band, the loudest content sets a masking threshold, and everything below that threshold is discarded.

Temporal masking: a loud transient masks quieter sounds immediately before and after it. Post-masking (after the loud event) lasts up to ~100-200 ms; pre-masking (before it) lasts only a few milliseconds but is real. A cymbal crash masks low-level detail in its wake. This is also where lossy codecs fail most audibly: a sharp transient against near-silence can produce pre-echo, a smear of quantization noise spread backward in time before the transient because the codec’s transform window is too long to localize it. Good encoders detect transients and switch to shorter windows to contain the damage.

   FREQUENCY MASKING (one critical band, schematic)

   loudness
      |        ####  <- masker (loud tone)
      |       ######
      |      ########
      |     ##########  masking threshold
      |    ../        \..
      |   ./   . . . .  \.        <- audible region (kept)
      | ../  x  x   x    \..      x = masked tones (discarded,
      |./   below thresh   \.          encoder spends 0 bits)
      +-----------------------------> frequency
            quiet tones under the curve are inaudible

The encoder allocates bits where the ear is sensitive and starves the regions the model says are masked. As the bitrate budget shrinks, the masking threshold is effectively the only thing keeping quality up: at low bitrate the encoder is forced to push quantization noise right up against (and eventually above) the masking curve, and that is the point where artifacts — warbling, “swooshing,” pre-echo, lost high-frequency air — become audible. The whole game of a modern encoder is a better psychoacoustic model and smarter bit allocation against it. If you want the signal-processing foundations under all of this, the transform and filtering machinery is covered in DSP for Audio and the intuitive Fourier transform post.

The Encode Pipeline

Nearly every modern lossy codec follows the same skeleton: a transform into the frequency domain, a psychoacoustic analysis to set the masking threshold, a quantizer that allocates bits against that threshold, and entropy coding to pack the result.

   GENERIC PERCEPTUAL AUDIO ENCODER

   PCM in
     |
     v
  +--------------------+        +-----------------------+
  | filter bank /      |        | psychoacoustic model  |
  | MDCT transform     |------->| (masking thresholds,  |
  | (time -> frequency)|        |  per critical band)   |
  +--------------------+        +-----------+-----------+
     |                                      |
     |  spectral coefficients               | bit allocation
     v                                      v
  +--------------------+        +-----------------------+
  | quantization       |<-------| allocate bits where   |
  | (lossy step here)  |        | the ear is sensitive  |
  +--------------------+        +-----------------------+
     |
     v
  +--------------------+
  | entropy coding     |   (Huffman / arithmetic / range)
  | + bitstream format |
  +--------------------+
     |
     v
  bitstream out

The lossy step is the quantizer: it rounds spectral coefficients coarsely where the model says noise will be masked, finely where it will not. Everything else — the transform (typically a Modified Discrete Cosine Transform, MDCT, with overlapping windows to avoid block-boundary artifacts) and the entropy coder — is lossless. The differences between MP3, AAC, and Opus are differences in transform design, window switching, joint-stereo handling, and the quality of the psychoacoustic model and bit allocation, not in the basic architecture.

The Lossy Codecs

MP3 (MPEG-1/2 Audio Layer III) is the codec that taught the world the word “codec.” Standardized in 1993, it is technically obsolete — beaten on quality-per-bit by AAC and decisively by Opus — and it persists anyway, for reasons that have nothing to do with audio quality. Its patents have fully expired (the last expired in 2017), it decodes on literally everything, and it is “good enough” at the bitrates people actually use (192-320 kbps), where a well-tuned LAME encode is transparent or close to it for most listeners. MP3’s real weaknesses are at low bitrate and on hard material: pre-echo on sharp transients, a hard 16 kHz-ish lowpass at lower bitrates, and a fixed filterbank structure that limits its efficiency. The crucial nuance is that MP3 quality is dominated by the encoder, not the format — LAME, the open-source encoder, is vastly better than the bad encoders that gave MP3 its “harsh” reputation in the 1990s. Most “MP3 sounds bad” memories are of a 128 kbps Xing encode from 1999, not of what the format can do.

AAC (Advanced Audio Coding) is MP3’s intended successor and the de facto modern lossy standard: it is what Apple Music, YouTube, and most broadcast and mobile pipelines ship. AAC improves on MP3 with a pure MDCT filterbank, better window switching, more flexible joint stereo, and Temporal Noise Shaping to control pre-echo. LC-AAC (Low Complexity) is the workhorse profile and reaches transparency around 128 kbps for most people. HE-AAC (High Efficiency, AAC+SBR) adds Spectral Band Replication — it codes the low frequencies normally and synthesizes the high band from the low band plus a small amount of side information — which makes it excellent at very low bitrates (32-64 kbps) for streaming and voice-plus-music, at the cost of some real high-frequency detail. HE-AACv2 adds Parametric Stereo for even lower bitrates. AAC is genuinely good. Its only real liability is licensing: it carries patent royalties, which is precisely the opening Opus exploited.

Opus is the modern winner on the merits. Standardized by the IETF in 2012 (RFC 6716), it is open, royalty-free, and the result of merging Xiph’s CELT (music-oriented, low-latency) with Skype’s SILK (speech-oriented). It dynamically blends or switches between the two, scales seamlessly from 6 kbps narrowband speech to 510 kbps stereo music, and supports very low algorithmic latency — which is why it is mandatory in WebRTC and ubiquitous in real-time voice (Discord, Zoom-class systems) and increasingly in streaming. In public ABX-based listening tests run by Hydrogenaudio and the Opus team, Opus consistently matches or beats AAC and HE-AAC across the bitrate range, and is clearly ahead at low-to-mid bitrates. It reaches transparency for most listeners around 96-128 kbps. There is essentially no technical reason to choose any new lossy codec other than Opus for a greenfield project; the only reasons to pick AAC are hardware decode support and ecosystem inertia.

The legacy ones, honestly. Ogg Vorbis was Opus’s predecessor — open, royalty-free, better than MP3 — and has simply been superseded by Opus; it survives mainly in game audio and older Spotify infrastructure. WMA sat near MP3/AAC on quality but lost on ecosystem and is dead. Musepack was a niche enthusiast codec tuned for transparency at ~160-180 kbps; respectable, irrelevant. None are worth choosing today.

The Lossless Codecs

Lossless compression cannot exploit psychoacoustics — it must reconstruct every sample exactly — so it works by prediction plus entropy coding. The encoder fits a short predictor (typically a linear predictor, like a low-order LPC filter) that estimates each sample from the previous few, then stores only the residual — the difference between the prediction and the actual sample. For real music, consecutive samples are highly correlated, so residuals are small and cluster near zero, which an entropy coder (Rice/Golomb coding in FLAC) packs efficiently. That is the entire trick: model the predictable part, store the small leftover compactly. Typical reduction is 40-60% with zero quality cost, varying with material — dense, loud, noisy music compresses less; sparse acoustic music compresses more.

FLAC (Free Lossless Audio Codec) is the open, royalty-free, dominant lossless format. It is fast to decode, well-supported, streamable, supports up to 32-bit/655 kHz, embeds metadata and replaygain, and has an explicitly patent-free design. It won the lossless format war for the same reasons Opus is winning the lossy one: it is free, open, good, and everywhere. ALAC (Apple Lossless) is Apple’s technically-equivalent answer; Apple open-sourced it in 2011. ALAC and FLAC are essentially the same idea with comparable compression and the same bit-exact guarantee — the difference is ecosystem, not quality. Other lossless formats exist (WavPack, Monkey’s Audio/APE, TAK) and some squeeze out marginally smaller files at much higher CPU cost, but FLAC’s speed, openness, and universal support make it the correct default. There is no audible difference between any two lossless codecs of the same source — they all decode to identical samples by definition.

The MQA Controversy

MQA (Master Quality Authenticated), launched in 2014, deserves an honest autopsy because it was marketed as the future of high-resolution audio and collapsed commercially by 2023, with the company entering administration in 2023 and its assets sold off. The pitch was seductive: a small file that delivered “studio master quality” through a proprietary “origami” folding scheme, with an authentication light confirming you were hearing the artist’s intended sound.

The reality was less flattering. MQA was not lossless — it was a lossy scheme wearing the language of high resolution. Its “folding” hid some high-frequency information inside the noise floor of a lossless-looking container, then “unfolded” it on decode, but the process was lossy and added MQA’s own signature (some filters introduced ringing, and the encode/decode was not bit-transparent to the original master). Authentication and unfolding required licensed hardware or software at every stage, which was the actual business model: MQA collected fees from labels, streaming services (notably Tidal), and DAC makers, with a “blue light” that functioned as much as a DRM-adjacent gatekeeping signal as a quality indicator. Independent analyses — most prominently Archimago’s measurement writeups and GoldenSound’s widely-circulated investigation — showed MQA delivered no demonstrable audible benefit over plain lossless, that its files were larger than the delivered resolution warranted, and that the “authentication” could be made to light up for content that had never touched a real master. The audiophile community, which MQA assumed would be its base, demanded blind tests and measurements and pushed back hard. When Tidal began offering plain FLAC HiRes and the licensing-toll model lost its anchor tenant, MQA’s rationale evaporated. The lesson is not that high-resolution audio is fake — it is that a proprietary, licensed, lossy format sold as “better than lossless” could not survive contact with measurement and blind testing.

Transparency and ABX: The Part Audiophiles Hate

Transparency is the bitrate at which a codec becomes statistically indistinguishable from the lossless source in a controlled blind test. The only legitimate way to establish it is ABX testing: you are given A (the reference), B (the encoded file), and X (randomly either A or B), and you must identify which X is, repeatedly. To claim you can hear a difference, you must score significantly above chance across enough trials — typically you need something like 12-of-16 or better to reach statistical significance. Crucially, ABX is blind and level-matched: you cannot see the label, and the two files are matched in loudness, because even a 0.2 dB level difference is reliably perceived as “better” and contaminates every sighted comparison.

The accumulated result of decades of these tests is blunt:

Codec Type Typical bitrate Transparency (most listeners) Licensing Primary use case
MP3 (LAME) Lossy 128-320 kbps ~192 kbps Patents expired Universal compatibility, legacy
LC-AAC Lossy 96-256 kbps ~128 kbps Royalty-bearing Apple Music, YouTube, broadcast
HE-AAC Lossy 32-80 kbps not transparent (low-bitrate) Royalty-bearing Low-bitrate streaming, voice+music
Opus Lossy 64-256 kbps ~96-128 kbps Royalty-free WebRTC, Discord, YouTube, streaming
Vorbis Lossy 96-256 kbps ~160 kbps Royalty-free Games, legacy Spotify
FLAC Lossless ~700-900 kbps always (bit-exact) Royalty-free Archival, audiophile, mastering
ALAC Lossless ~700-900 kbps always (bit-exact) Royalty-free Apple ecosystem lossless

The honest truth that follows: the overwhelming majority of “I can clearly hear the difference between this 256 kbps AAC and the FLAC” claims fail ABX testing. When the label is hidden and the levels are matched, the discrimination collapses to chance. This has been demonstrated so many times on Hydrogenaudio that “post your ABX log” is the standard, deserved response to a golden-ears claim. The differences people are confident about under sighted conditions are dominated by expectation bias, level mismatches, and the simple fact that they know which file is the “expensive” one.

To be fair, because fairness is the point: the differences are not always zero. At low bitrate (under ~96 kbps) codecs diverge audibly and Opus genuinely wins. On pathological “killer samples” — castanets, harpsichord, fatboy, certain glockenspiel and applause passages chosen specifically to stress pre-echo and transient handling — trained listeners can ABX even fairly high-bitrate MP3, and codec developers use exactly these samples to find weaknesses. Lossless is genuinely necessary for editing and archival to avoid generational loss. And there are real, audible problems in the chain that audiophiles correctly notice but misattribute to the codec: bad masters, loudness-war clipping, poor DAC implementations, and level mismatches. The skepticism is about transparency claims on normal music at normal bitrates, not a claim that human hearing is irrelevant.

Patents, Licensing, and Why Free Won

The codec landscape was shaped less by audio quality than by intellectual property. MP3 was encumbered by a thicket of Fraunhofer/Thomson patents that generated licensing revenue for two decades; those patents expired by 2017, which is part of why MP3 lingers as a no-cost universal format. AAC remains royalty-bearing — encoders, decoders, and sometimes content distribution carry fees administered through patent pools (Via LA, formerly Via Licensing) — which is a real cost for manufacturers and a real friction for open-source projects.

Opus was designed from the start to be royalty-free, with its contributors (Xiph, Mozilla, Skype/Microsoft, Broadcom, Google) committing their relevant patents under irrevocable free licenses, and the IETF standardizing it so the web could have an unencumbered audio codec. That single property — free to implement, ship, and decode in a browser — is why Opus is mandatory in WebRTC and spread through the real-time-communication industry without anyone negotiating a license. FLAC won the lossless side the same way. The pattern is consistent: when an open, royalty-free codec reaches quality parity with the proprietary incumbent, the open one wins the platforms, because platforms hate per-unit royalties more than they love a marginal quality edge nobody can ABX anyway.

The Streaming Format Wars

What you actually hear depends on which service you use, and the choices are revealing.

Spotify historically used Ogg Vorbis (around 96/160/320 kbps for free/premium/high tiers) and has moved toward AAC on some platforms; its long-promised lossless tier finally arrived as a higher tier. Spotify’s bet for years was that 320 kbps Vorbis is transparent for its audience, which the ABX evidence supports. Apple Music ships AAC at 256 kbps as its standard quality and offers ALAC lossless (up to 24-bit/192 kHz) at no extra charge — a notable move that undercut the entire “pay more for lossless” market. Tidal built its brand on lossless FLAC HiRes and was MQA’s flagship, then quietly pivoted to plain FLAC as MQA collapsed. YouTube uses Opus (and AAC as a fallback) at variable bitrates negotiated per stream, which is why it scales gracefully across connection qualities. Amazon Music offers FLAC-based HD/Ultra HD tiers.

The through-line: every major service ships a lossy codec at a bitrate at or above its transparency threshold for normal listening (256 kbps AAC, 320 kbps Vorbis, adaptive Opus), and offers lossless mostly as a marketing tier and an archival/audiophile reassurance rather than because most subscribers can hear the difference on their hardware. The services know the ABX literature. They ship lossless because it is cheap insurance against the loudest complaints and a clean differentiator, not because their data shows users discriminating it blind.

Verdict

For new engineering work, the answers are clear and the patents are why. Use Opus for any lossy application — streaming, voice, real-time, web — because it is open, royalty-free, lowest-latency, and wins or ties every relevant ABX test; the only reason to choose AAC instead is hardware decode support or Apple-ecosystem inertia. Use FLAC for anything lossless — archival, mastering, distribution — because it is open, fast, universal, and bit-exact; choose ALAC only if you are locked into Apple tooling. Treat MP3 as a compatibility fallback, not a quality choice, and remember its quality is dominated by the encoder (use LAME). Ignore MQA entirely; it was a licensed, lossy format sold as better-than-lossless and the measurements and the market both rejected it.

On the perennial argument: at the bitrates every major service actually ships, lossy is transparent for the overwhelming majority of listeners on the overwhelming majority of material, and the way to know whether you are an exception is to run a real, level-matched ABX test and post the log — not to trust a sighted impression. Lossless is worth using for archival and editing where generational loss matters, and worth nothing audible over transparent lossy on playback. The differences that survive blind testing are at low bitrate, on killer samples, and in the master and the DAC — not in the 256 kbps stream you were about to spend money to upgrade.

Sources

Comments