Autofocus Systems Explained

June 20, 2026 16 min read Photography & Imaging Engineering

photographyautofocuscamerasimagingmachine-learningengineering

Contents

Autofocus is the camera feature most photographers stopped thinking about somewhere in the late 1990s. That apparent stasis hides two complete reinventions: the migration of phase detection from a dedicated sub-mirror module onto the imaging sensor itself in the early 2010s, and the bolting of convolutional neural networks onto the AF pipeline since 2018. The headline numbers on a 2026 spec sheet are extraordinary - 759 phase-detect points, 92% frame coverage, -7.5 EV sensitivity, 120 AF/AE calculations per second, simultaneous detection of humans, birds, insects, trains, planes, motorcycles - and also misleading about what actually focuses in real-world shoots. This post unpacks the engineering: how cameras infer focus state without a distance sensor, why contrast and phase detection are fundamentally different solutions, what the on-sensor PDAF revolution changed, how neural networks decide what your subject is, and where the marketing diverges from the hit rate you will see on your card.

The autofocus problem, stated honestly

A camera needs to position the focusing element of its lens such that light from the subject converges at the image sensor plane. To do that it needs to know how far out of focus the current lens position is and in which direction. There is no direct sensor for distance in a normal camera body. The autofocus problem reduces to: infer focus state from the image itself, or from an optical signal correlated with focus state, and command the lens to move.

If you have followed how a camera sensor works you already know that the imaging sensor is a Bayer-filtered photodiode array reading out at fixed pixel positions. AF asks the camera to use that same array, or a small dedicated array, as a focus estimator while still capturing usable images. The two dominant solutions are contrast detection and phase detection.

Contrast-detect AF: hill-climb on a single measurement

Contrast-detect autofocus (CDAF) is the brute-force solution. An in-focus image has more high-frequency detail than an out-of-focus one. Measure the contrast of the focus area, move the lens, remeasure, hill-climb to the maximum.

The algorithm is exactly what it sounds like. Capture frame at lens position x. Compute a contrast metric (sum of absolute differences between adjacent pixels, or a Laplacian-like high-pass filter) on the focus region. Move the lens. Capture again. If contrast went up, keep going. If it went down, reverse and reduce step size. Iterate until contrast stops increasing.

CDAF has two virtues. It uses the imaging sensor directly with no special pixels. And the optimum it converges on is, by construction, the position of maximum sharpness for that specific lens and sensor - no calibration offset, no front-focus or back-focus problem of the kind that haunted DSLR PDAF.

The cost is large. CDAF needs at least three measurements - direction, confirm, overshoot - and in practice five to ten lens movements to converge. Each one takes a sensor readout (tens of ms) plus a lens motor move. You cannot tell direction from a single measurement: the camera guesses and checks. The contrast curve is roughly bell-shaped, so CDAF tends to overshoot the peak and hunt back - the characteristic wobble that compact cameras spent the 2000s being mocked for. Nearly useless on continuously moving subjects.

CDAF dominated compact cameras and the first generation of mirrorless (Panasonic G1, Olympus E-P1, Sony NEX). It survives today as the refinement stage of hybrid systems.

Phase-detect AF: one measurement, both direction and magnitude

Phase detection autofocus (PDAF) is the elegant solution. The premise: split the light coming through the lens into two beams that pass through different parts of the aperture. If the lens is focused, both beams converge to the same point on the sensor and the two images are aligned. If the lens is out of focus, the two images are shifted relative to each other. The direction of the shift tells you which way to move the lens. The magnitude tells you how far.

                    PHASE-DETECT OPTICAL PATH (DSLR style)

       lens (out-of-focus, subject too close)
       =========
         \\   //          main mirror (partially transparent)
          \\ //          /
   subject \X/ ---------/----------> viewfinder
           / \           \
          //   \\          \  submirror (folds light down)
         //     \\          \
        //       \\          \
   ---/-----------\\--------> AF sensor module
     / separator   \\           |
     \  lenses     /           [line array A]  upper beam
       \         /             [line array B]  lower beam
                                    |
                                    v
              phase shift dx -> defocus + direction
              dx = 0 -> in focus
              dx > 0 -> back focus (move lens forward)
              dx < 0 -> front focus (move lens back)

A pair of separator lenses splits the cone of light into two beams sampling different halves of the exit pupil. Each beam projects onto a one-dimensional line array of photosites. If the main lens is focused, the two line images are aligned. If not, they shift in opposite directions, and the camera measures the cross-correlation to extract the shift. One readout, both direction and magnitude, one lens move. PDAF benefits from clean optics - lens engineering makes the PDAF signal possible in the first place.

PDAF is fast - one to three lens movements - and it works on moving subjects because each readout gives a fresh estimate. It is what made continuous AF viable for sports and wildlife.

The cost: PDAF is an open-loop estimator. The cross-correlation gives a shift in pixels, mapped through a calibration table to a lens move in millimeters. If the calibration is wrong - and on DSLRs it routinely was, drifting with temperature, lens, and individual copies of the same lens - you get front focus or back focus. The PDAF sensor is also a separate module. On a DSLR, light hits the main mirror, most goes up to the viewfinder, a partially silvered section passes through to a sub-mirror reflecting to the PDAF module in the bottom of the mirror box. None of that works when the mirror is up - which is exactly when you are shooting live view, video, or stills on a mirrorless camera.

For thirty years that was the deal. The Canon 5D Mark III had 61 AF points, the Nikon D850 had 153, the Sony A99 had 79 dedicated PDAF plus on-sensor assist. All clustered in the center because sub-mirror geometry prevented edge coverage.

The on-sensor PDAF revolution

The clever idea: put the phase-detection step on the imaging sensor itself. Mask a subset of photosites so each receives light from only one half of the exit pupil. Pair them up - one masked left, one masked right - and you have a tiny phase-detection system at every pair location.

              ON-SENSOR PDAF PIXEL PAIR (simplified)

   normal Bayer cell:        masked AF pixel pair:

   +--+--+                   +-----------+-----------+
   |R |G |                   | left-mask | right-mask|
   +--+--+                   |  AF pix L |  AF pix R |
   |G |B |                   +-----------+-----------+
   +--+--+                        ^             ^
                                  |             |
                       light from \\           // light from
                       right half  \\         //  left half
                       of aperture  \\       //  of aperture
                                     \\     //
                                      \\   //
                                       \\ //
                                  ======lens======
                                       /   \
                                      /     \
                                  (out-of-focus subject)

      compare L vs R intensities along a row -> phase shift -> defocus
      coverage limited only by where you place AF pixel pairs
      (modern sensors: ~90-100% of frame)

Fujifilm shipped early versions in the S5 Pro and X-series. Nikon’s 2011 Nikon 1 had a hybrid CDAF/PDAF on-sensor approach. The technology became really good with the Sony A7 III in 2018, which paired 693 on-sensor PDAF points with motion prediction. Canon went a parallel route with Dual Pixel CMOS AF, where every photosite is split into two half-pixels that can be read independently for phase detection and summed for imaging - making 100% of pixels AF-capable without sacrificing imaging signal.

Coverage went from sparse center-clustered grids to near-total: Sony A1 II has 759 points covering ~92% of sensor area; Nikon Z9 and Z8 use a 493-point grid with similar coverage; Canon’s Dual Pixel covers essentially the entire frame. Live view, video, and stills share the same AF pipeline. The DSLR-era front-focus/back-focus problem largely vanished because PDAF and imaging measurements are made on the same physical sensor at the same time.

The cost is contained but real. Masked AF pixels receive light from half the aperture, so their values are interpolated from neighbors in the final image. With a few thousand AF pixels in a 50-megapixel sensor that interpolation is invisible. The more interesting failure mode is PDAF striping: in high-contrast scenes with bright point sources, light reflects off the metal masks and produces horizontal banding in shadows. Jim Kasson’s testing on the Sony A7R IV found the artifact much reduced compared to earlier bodies, but the underlying mechanism is intrinsic to masked-pixel PDAF and can only be minimized.

Hybrid AF: PDAF gets close, CDAF finishes the job

Modern flagship cameras essentially always use hybrid AF. PDAF runs first: read phase shift across the AF area, compute a single lens move, execute. That gets the lens within a few microns of the focused position - usually within depth of focus, but not always at the peak of the MTF curve. CDAF then refines: a small dither around the PDAF-predicted position, measuring contrast, locking on the maximum.

The hybrid stage is invisible because it executes in tens of milliseconds. It matters because PDAF alone, with its calibration dependence and sensitivity to lens decentering, is not quite reliable enough for 60-megapixel sensors. CDAF alone is too slow. Hybrid gets the speed of PDAF and the absolute accuracy of CDAF.

Neural networks arrive: subject detection takes over

Around 2018 the AF problem changed shape. Until then, AF was “given that the photographer told me where the subject is, focus there.” The neural-network era reframed it: “find the subject, decide which part to focus on, and track it as it moves.”

Sony’s Real-Time Tracking shipped on the A9, Animal Eye AF on the A7R IV. Canon’s EOS iTR AF X arrived with the R3 and matured into Dual Pixel Intelligent AF on the R1 and R5 Mark II. Nikon’s 3D Tracking with subject detection appeared on the Z9 and propagated to the Z8. All run convolutional neural networks - or hybrid CNN/transformer architectures in newer generations - on dedicated NPU silicon. The Sony A1 II adds a dedicated AI Processing Unit on top of BIONZ XR; Canon’s DIGIC Accelerator on the R5 II and R1 does the same; Nikon’s Expeed 7 has comparable acceleration. These accelerators descend from the same silicon lineage discussed in how a transistor actually works, optimized for low-precision matrix multiply at watt-class power budgets.

The networks are trained to detect a fixed taxonomy of subject classes. The 2026 standard list: people (head, face, eye, sometimes torso and joint angles); animals (cats, dogs, mammals broadly); birds (often the trickiest because of partial occlusion in foliage); vehicles (cars, sometimes broken out into formula, rally, and road; motorcycles); trains; planes (sometimes hierarchically: whole plane, front section, cockpit); insects (Sony added this on the A1 II and A9 III).

Once the detector finds a subject, it scores candidate eye/face/head/body regions and places a focus point on the highest-priority region. Tracking is handled either by re-running the detector each frame or by lighter-weight optical-flow trackers anchored on detector outputs. Canon’s R1 and R5 II extended this with Action Priority AF, using pose estimation to score subjects by their probability of being “the important one” in a sports scene - a basketball player about to take a shot prioritized over the defender behind them.

The numbers are dazzling. The Sony A1 II runs 120 AF/AE calculations per second with a 30% improvement in human eye detection and 50% on animal eyes over the original A1. The Canon R5 II focuses down to -7.5 EV at f/1.2. The Nikon Z9 and Z8 detect nine subject types with 3D Tracking across the full Wide-area AF region.

AF method comparison

Method	Where measured	Speed (good light)	Accuracy	Coverage	Direction	Live view / video	Era
CDAF	Imaging sensor	200-800 ms	Excellent at peak	Anywhere	No	Yes	Compacts, early mirrorless, hybrid refine stage
DSLR PDAF	Sub-mirror module	50-100 ms	Calibration-dependent	Sparse, center (45-153 pts)	Yes	No	DSLR 1985-2015
On-sensor PDAF	Imaging sensor (masked/split pixels)	30-50 ms	Good, drift-free	~100% (493-759+ pts)	Yes	Yes	Modern mirrorless
Hybrid PDAF + CDAF	Both	30-50 ms + CDAF refine	Excellent	~100%	Yes	Yes	All current flagships
Hybrid + NN detection	Both + NPU	+5-15 ms detector	Excellent on trained classes	~100%	Yes	Yes	2018-present

Flagship comparison: the 2026 numbers

Camera	AF points	Coverage	Low-light	Calc rate	Subject classes	NPU
Sony A1 II	759 (627 video)	~92%	-4 EV @ f/2	120 Hz	People, animals, birds, insects, cars, trains, planes, auto	AI Processing Unit
Sony A9 III	759 (global shutter)	~96%	-5 EV	120 Hz	Same as A1 II	AI unit
Sony A7R V	693	~79%	-4 EV @ f/2	120 Hz	People, animals, birds, insects, vehicles	AI unit
Canon EOS R1	1053 zones	~100%	-7.5 EV @ f/1.2	High (Action Priority)	People, animals, birds, vehicles + ball/action	DIGIC Accelerator
Canon EOS R5 II	1053 zones	~100%	-7.5 EV @ f/1.2	High	Same as R1	DIGIC Accelerator
Nikon Z9	493	~90%	-8.5 EV	120 Hz	9 types (people, dog, cat, bird, car, motorcycle, bike, train, plane)	Expeed 7
Nikon Z8	493	~90%	-7 to -8.5 EV	120 Hz	Z9 list + plane sub-types	Expeed 7
Fujifilm X-H2S	425	~100%	-7 EV	High	People, animals, birds, vehicles	X-Processor 5
Olympus OM-1 II	1053 cross-type	~100%	-8 EV	120 Hz	People, animals, birds, vehicles	TruePic X
Panasonic GH7	779	~100%	-7.5 EV	High	People, animals, vehicles	Venus engine
Hasselblad CFV 100C	294	~92%	-5.5 EV	Lower	Faces only	None
iPhone 17 Pro	Dense full-sensor + LiDAR	~100%	Extreme	Per-frame	People, pets, food	Apple Neural Engine
Pixel 10 Pro	Full-sensor PDAF	~100%	Extreme	Per-frame	People, pets, scenes	Tensor G5 NPU

A note on Panasonic: through the GH5 and GH6 they stayed on a CDAF-only system called Depth from Defocus, using lens characterization data to predict defocus direction from blur shape. It was clever but visibly hunted in video. The GH7 and S5 II ended that era by adding on-sensor PDAF. That they held out so long is a reminder that hybrid PDAF is not the only viable solution - just the one that wins in 2026.

What the marketing actually means

The claims on the spec sheets are not wrong, but they are carefully phrased.

“World’s fastest AF.” True under specific conditions: high-contrast subject, good light, fast lens. Real-world acquisition is more like 50-100 ms for a fresh subject, longer if moving or low-contrast.

“Eye detection in 95%+ of frames.” True for portrait shooting of a single subject facing the camera. Hit rate drops with multiple subjects, partial occlusion (hair, glasses, masks), profile views (networks are trained predominantly on frontal faces), backlit faces, and rapid head turns. Sports photographers will tell you eye AF reliability on a fast-moving subject is closer to 70-80%.

“Animal/bird/insect/vehicle detection.” Highly trained-class dependent. Common species - domestic dogs and cats, common songbirds, sedans and SUVs - work brilliantly. A horse may be detected as a deer; a kestrel may not register; a vintage car may detect as “vehicle” without eye/headlight prioritization. The classic failure: bird-mode AF locks onto a far seagull instead of the foreground eagle because the seagull’s pose better matches the training set.

"-7.5 EV low-light AF." True with a specific fast lens (Canon quotes f/1.2). Mount an f/4 zoom and effective sensitivity drops several stops. Also a measurement of the system acquiring focus at all, not quickly.

“120 AF/AE calculations per second.” True, and it matters for high-frame-rate continuous shooting. It does not mean AF latency is 1/120 second - the lens motor and algorithm pipeline impose their own delays.

PDAF banding. Largely solved on current bodies. Current sensors have shrunk the masked-pixel footprint and improved microlens geometry. It can only be managed, not zeroed.

Phone autofocus is the same problem, easier

Smartphone AF is impressive, and easier than it looks. iPhone 17 Pro and Pixel 10 Pro both use dense on-sensor PDAF (Apple calls them Focus Pixels) combined with neural-network subject detection on the Apple Neural Engine or Tensor G5 NPU. The iPhone adds LiDAR assist for low-light, giving a direct distance measurement when phase signal is weak.

The reason it works so well is geometric. A phone main camera has a 5-7 mm focal length on a 1/1.3" sensor. A phone at f/1.8 has depth of field comparable to full-frame at f/8 or smaller, so “close enough” focus is a much wider zone. This is also why portrait mode exists: phones synthesize shallow depth of field via depth maps and compositing - the territory of computational photography.

Where AF still fails

Even the best 2026 systems hit hard limits. Dim scenes below the rated EV limit force hunt-and-refine cycles that take a full second. Flat low-contrast subjects (blank wall, uniform sky) have no phase signal. Shooting through dirty windows produces phase signals that lock onto the surface. Reflective subjects (chrome, polished water) produce unstable phase because the effective light source moves with viewpoint. Dense foliage with partially occluded subjects confuses the detection network frame-by-frame.

For focus stacking, tilt-shift focus, and astrophotography below the AF threshold, manual focus with peaking or magnified live view is still the correct tool. The AF system has not failed; the application is outside what AF was designed for.

Verdict

Autofocus in 2026 is a remarkable engineering achievement that the marketing overstates only mildly. The on-sensor PDAF revolution erased the decades-long compromise between viewfinder speed and live-view accuracy. The neural-network subject detection era erased most of the burden of telling the camera what to focus on. The Sony A1 II, Canon R1 and R5 II, and Nikon Z9 and Z8 deliver real-world hit rates that would have been science fiction in 2010.

The honest gap is in the edge cases. Eye detection at 95% in a portrait studio is not 95% on a midfielder running diagonally through stadium light. Subject detection on a kestrel is not subject detection on a domestic cat. PDAF banding is managed but not gone. Low-light EV limits assume a specific fast lens.

If you are buying in 2026, the AF system on any flagship will outperform you on hit rate. The right question is which subject categories the manufacturer trained for: Canon and Sony lead on people and sports, Nikon on birds, Olympus on small-and-fast wildlife, Fujifilm on people, phones on auto-everything social photography. Pick the system whose networks were trained on your subjects. AF in 2026 has stopped being a feature you choose cameras on; it has become a baseline that lets photographers think about composition instead.