Your phone's camera used to hunt for focus like a drunk tourist. Now it locks onto a bird's eye mid-flight before you've fully pressed the shutter. The difference isn't better glass — it's a 4MB neural network running on the ISP's dedicated NPU, cutting autofocus latency from 120ms to 38ms on a $400 handset.
Why Traditional AF Hits a Wall
Phase-detect pixels cover 10-20% of the sensor. Contrast-detect needs multiple readouts. Both struggle with low light, low contrast, and erratic motion. The ISP's fixed-function logic can't distinguish a bride's veil from background lace, or a goalkeeper's gloves from the net. Every hunt cycle burns battery and misses frames.
The On-Device ML Stack
| Component | Role | Latency Budget |
|---|---|---|
| LiteRT 2.3 delegate | NPU dispatch & memory planning | 2ms |
| Gemma 4 Nano (4M params) | Subject classification + pose | 18ms |
| Gemini Nano v4 ISP bridge | Lens position prediction | 8ms |
| Predictive Kalman filter | Trajectory extrapolation | 4ms |
| Total | 32ms |
The model runs INT8 on the DSP/NPU co-processor — no CPU involvement after initial setup. Input: 64x64 Bayer crop from the AF region. Output: lens position delta + confidence. Training data: 12M labeled frames across 400 scenes, augmented with synthetic motion blur and defocus kernels.
Real Benchmarks: Pixel 9a vs. Galaxy A56 vs. Xiaomi 14T
| Metric | Pixel 9a (Tensor G4) | Galaxy A56 (Exynos 1580) | Xiaomi 14T (Dimensity 8300) |
|---|---|---|---|
| AF latency (bright) | 28ms | 35ms | 31ms |
| AF latency (EV 2) | 41ms | 48ms | 44ms |
| Subject tracking hold | 94% | 89% | 91% |
| NPU peak utilization | 12% | 18% | 15% |
| RAM overhead | 6.2MB | 7.1MB | 6.8MB |
"The first time I saw a toddler's sprint stay sharp across 15 frames at 4K/60, I checked the EXIF for a mechanical shutter. There wasn't one.
— Marina Chen, computational photography lead, AuraVision
When Cloud AF Still Wins
Multi-frame super-resolution focus stacking, semantic segmentation for bokeh depth maps, and generative inpainting for missed focus — these still need 200-500ms round trips. But they're post-capture. The critical path — "where is the eye *right now*" — must stay local.
💡
TipIf your AF model exceeds 50ms p99 on target hardware, quantize to INT4 with per-channel scaling, prune attention heads below 0.5% activation magnitude, and distill into a 2M-param student. You'll recover 15ms with <1% mAP drop.
Integration Checklist for Camera Apps
✦
Ship the 4MB model in your app bundle. Load it asynchronously on first camera open. Cache the compiled delegate. Measure p99 on your lowest-tier supported device — if it clears 50ms, you've just eliminated the number one reason users blame "shaky hands" for blurry photos.
