LUNAROPS · OPERATIONAL UPLINK 100% UPTIME 1,247d POSTS 893 JEFF.MOON@LUNAROPS.DEV UTC --:--:--

Frigate: AI-Powered NVR

19 min read Homelab & Self-Hosting
frigatenvrhome-assistantobject-detectionself-hosting
Contents

Cloud camera subscriptions have normalized a model where you pay monthly for the privilege of having a corporation analyze footage of your home, store it on their servers, and potentially hand it to law enforcement with or without your knowledge. Ring and Google Nest have made this arrangement feel ordinary. It is not. You are paying recurring fees for a service with worse detection latency than local inference, a privacy posture you have no control over, and a dependency on vendor uptime that has already produced outages during which cameras silently stopped recording. Frigate is the serious alternative: a self-hosted, open-source network video recorder built from the ground up around real-time object detection, with no cloud dependency and latency measured in milliseconds from event to notification.

Frigate’s architecture is not just “run YOLO on your server.” It is a carefully layered pipeline — go2rtc handles stream acquisition and restreaming, motion detection gates which frames reach the AI detector, dedicated accelerator hardware handles inference at 7–30ms per frame, ffmpeg with hardware decoding handles the decode path that is frequently the actual CPU bottleneck, and MQTT bridges the detection events into Home Assistant and any other automation platform. Getting this right requires understanding each layer and its trade-offs. Getting it wrong means a system that either floods you with false positives or burns CPU on software decode and never actually accelerates inference.

What Frigate Is

Frigate is an open-source NVR written primarily in Python and Go, containerized via Docker, and distributed as a multi-arch image. It consumes RTSP streams from IP cameras, performs continuous motion analysis using OpenCV, and routes motion-triggered frame crops to a configured AI detector for object classification. Detected events are stored as video clips and snapshots with configurable retention policies, exposed through a clean web UI, and published as structured MQTT messages for downstream automation.

The project is maintained by Blake Blackshear and has grown substantially since its early Coral-only days. As of 2025-2026, Frigate supports Google Coral TPU (USB and M.2), Hailo-8 and Hailo-8L M.2 accelerators, OpenVINO on Intel integrated and Arc GPUs, ONNX-based inference on Nvidia GPUs (with automatic TensorRT compilation), and CPU-only detection as a fallback. Each detector type represents a genuinely different trade-off in cost, performance, power draw, and operational complexity.

The motion-to-detection pipeline matters because it explains why Frigate can run comfortably with modest hardware. OpenCV motion detection is cheap — it is pixel diff math, not neural inference. Only regions that show meaningful pixel change are cropped and sent to the detector. On a typical suburban home setup with 4–6 cameras, you might see a continuous 3–5 FPS detection load per camera rather than the 15–30 FPS of the raw stream. The detector’s job is to process those crops quickly enough that no frames back up in the detection queue.

Detector Hardware: The Real Trade-Offs

Choosing a detector is the single most consequential hardware decision in a Frigate build. The difference between a 10ms inference time and a 150ms inference time scales directly with how many cameras you can support before detections start queuing.

Detector Hardware Comparison
─────────────────────────────────────────────────────────────────────────
Hardware             Inference    TOPS   Power   Cost      Model Format
─────────────────────────────────────────────────────────────────────────
Google Coral USB     8–9 ms       4      ~2W     $60–90    TFLite INT8
Google Coral M.2     6–8 ms       4      ~2W     $25–45    TFLite INT8
Hailo-8L M.2         10–13 ms     13     ~2.5W   $60–100   HEF (compiled)
Hailo-8 M.2          7–10 ms      26     ~3W     $90–150   HEF (compiled)
Intel iGPU OpenVINO  20–35 ms     varies ~5–15W  $0 (reuse) ONNX / IR
Nvidia GPU ONNX      5–12 ms      varies 50–200W $150–600+ ONNX / TRT
CPU-only (software)  80–250 ms    N/A    10–30W  $0        TFLite / ONNX
─────────────────────────────────────────────────────────────────────────

Google Coral remains the canonical choice for low-power, low-cost builds. At 8–9ms per inference for a 320x320 frame, a single USB Coral can comfortably handle 4–6 cameras. The Coral M.2 variant (used in the Coral M.2 Accelerator and the Pineberry Pi AIHat for Raspberry Pi) is preferred over USB where bandwidth matters. The significant caveat: Coral hardware has been difficult to source consistently since 2023, and Google has not shipped a revised Edge TPU. The 4 TOPS figure also means the Coral is running MobileDet or similar compact architectures — not a YOLOv9 variant. For Frigate’s standard detection workload the accuracy is adequate, but for a Frigate+ fine-tuned model on a large or complex camera layout, you may want more headroom. The official Frigate documentation now explicitly notes the Coral is no longer recommended for new installations except in low-power deployments. See edge AI accelerators for a deeper hardware comparison.

Hailo-8 and Hailo-8L are the current recommended path for M.2-based accelerator builds. The Hailo-8L (13 TOPS) is sufficient for most residential deployments; the Hailo-8 (26 TOPS) is warranted when running 8+ cameras or using larger YOLO variants. Real-world community benchmarks show 12–13ms inference with YOLOv6n on the Hailo-8L, dropping to 9–10ms with the optimized yolov6n 0.2.1 model, and around 7ms on the Hailo-8 at 640x640. The tradeoff is model format: Hailo requires models compiled to .hef format using the Hailo Model Zoo toolchain. You cannot pull an arbitrary ONNX model and run it — it must be compiled for the specific Hailo chip. Frigate ships pre-compiled HEF models for the supported architecture list, but custom models (including Frigate+ fine-tuned models) require you to run the Hailo compiler, which requires a Linux host with the Hailo SDK installed. This is manageable but not trivial.

OpenVINO on Intel iGPU is the best zero-cost option for anyone running Frigate on an Intel N100, i3, i5, or any modern Intel box with a built-in GPU. Inference times on an Intel Arc iGPU (like the Arc 3 in the N100) run 20–35ms for YOLOv9 small at 320x320. That is slower than Coral or Hailo, but it costs nothing beyond the hardware you already have. An Intel N100 NUC doing double duty as a Frigate host and handling decode and detection via VAAPI and OpenVINO simultaneously is a very practical setup. The OpenVINO path uses .xml/.bin IR format or ONNX directly — model flexibility is good.

ONNX on Nvidia GPU is the highest-performance option and the most overkill for most home setups. As of Frigate 0.16+, the TensorRT detector type was removed in favor of a unified ONNX detector that automatically compiles to TensorRT when the stable-tensorrt image is used and an Nvidia GPU is present. On a mid-range card (RTX 3060 or similar), inference runs at 5–12ms for YOLOv9 models, and the GPU can handle 12+ camera detection loads without breaking a sweat. The cost is power (50–200W continuous for the GPU alone), price, and the fact that you need the Nvidia container toolkit configured in Docker. This path makes most sense if you already have a gaming PC or server with a discrete Nvidia GPU that is otherwise idle.

CPU-only detection is not a viable production configuration. At 80–250ms per inference frame, a 6-camera deployment will saturate the detection queue within minutes of activity, frames will queue faster than they clear, and RAM usage climbs. CPU detection is useful for testing configuration and verifying streams before detector hardware arrives.

The go2rtc Restreaming Layer

go2rtc is an open-source multi-protocol media server that Frigate embeds directly as a subprocess. Its role in the Frigate pipeline is specific and important: it connects to each camera’s RTSP stream exactly once and fans that stream out to multiple consumers, preventing the camera from being overwhelmed by multiple simultaneous connections.

Camera (RTSP)
      |
      v
  [ go2rtc ]
   /       \
  /         \
detect      record
(substream   (main stream
 640x480)    1080p/4K)
  |               |
  v               v
[OpenCV          [ffmpeg
 motion]          muxer]
  |
  v
[detector crops]
  |
  v
[MQTT events / clips / snapshots]

A well-configured go2rtc setup uses the camera’s low-resolution substream for detection and the high-resolution main stream for recording. Most IP cameras expose both as separate RTSP paths. The substream — typically 640x480 or 1024x576 at 5–10 FPS — is all the detector needs. Running detection against a 4K stream is wasteful and counterproductive: the model still receives a 320x320 or 640x640 crop regardless of source resolution, and you are burning decode cycles on a stream that provides no detection accuracy benefit.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

# go2rtc inside frigate config.yml
go2rtc:
  streams:
    front_door:
      - rtsp://admin:password@192.168.1.101:554/stream1    # main
      - rtsp://admin:password@192.168.1.101:554/stream2    # sub

cameras:
  front_door:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/front_door          # restream via go2rtc
          roles:
            - detect
            - record

In practice, it is better to route the substream explicitly to detect and main stream to record to avoid Frigate sending the 4K feed through the detection pipeline. The roles field on each input controls this assignment.

ffmpeg Hardware Acceleration: Decode Is the Real Bottleneck

Every Frigate deployment guide focuses on detector hardware. Far fewer explain that on multi-camera builds, software video decode by ffmpeg — not AI inference — is typically the process consuming the most CPU. A single 1080p H.265 stream at 15 FPS requires meaningful CPU time in software decode. Six cameras doing this simultaneously will saturate a midrange processor before the detector is even stressed.

Frigate exposes ffmpeg hardware acceleration through the hwaccel_args configuration:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21

# Intel VAAPI (iGPU)
cameras:
  front_door:
    ffmpeg:
      hwaccel_args: preset-intel-vaapi
      inputs:
        - path: rtsp://127.0.0.1:8554/front_door
          roles:
            - detect
            - record

# Nvidia NVDEC
cameras:
  driveway:
    ffmpeg:
      hwaccel_args: preset-nvidia-h264
      inputs:
        - path: rtsp://127.0.0.1:8554/driveway
          roles:
            - detect
            - record

Frigate ships preset strings (preset-intel-vaapi, preset-intel-qsv, preset-nvidia-h264, preset-nvidia-h265) that wrap the correct ffmpeg flags. For VAAPI on Intel hardware, you need /dev/dri/renderD128 passed through to the container. For Nvidia, the full Nvidia container toolkit must be installed on the host, and the container needs runtime: nvidia or the deploy resources block.

Docker Compose example with Intel VAAPI and a Coral USB:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21

services:
  frigate:
    container_name: frigate
    image: ghcr.io/blakeblackshear/frigate:stable
    privileged: true
    restart: unless-stopped
    shm_size: "128mb"
    devices:
      - /dev/bus/usb:/dev/bus/usb        # Coral USB TPU
      - /dev/dri/renderD128              # Intel VAAPI
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /opt/frigate/config:/config
      - /mnt/nvr/recordings:/media/frigate
    environment:
      FRIGATE_RTSP_PASSWORD: "changeme"
    ports:
      - "5000:5000"
      - "8554:8554"
      - "8555:8555/tcp"
      - "8555:8555/udp"

If ffmpeg processes are showing 15–30% CPU each in htop despite hardware acceleration being configured, the most common cause is that the codec negotiation failed silently. Check docker logs frigate 2>&1 | grep -i "hwaccel\|vaapi\|nvdec" — a line saying the hardware decoder was not initialized means ffmpeg fell back to software. The /dev/dri device must be owned by the render group, and the container must run with a user that has access to that group.

Zones, Motion Masks, and the False Positive Problem

A freshly installed Frigate with default configuration and a driveway camera will generate a detection event every time a car passes on the street behind your property boundary. This is correct behavior — a person is a person, a car is a car — but it is not useful behavior. The false positive problem is really a context problem, and Frigate provides several tools to inject spatial context.

Motion masks prevent OpenCV from registering motion in a defined polygon. They are appropriate for camera timestamps, tree branches that sway in wind, or a public road where you want to avoid wasting detection cycles entirely. The critical limitation: a motion mask only prevents that area from triggering the detection pipeline. If motion in an unmasked area starts a detection pass and the model finds an object in the masked region, it will still be reported.

Object filter masks suppress detections of a specific object type within a defined region. These are surgical — use them when you know a particular zone will always generate false positives for a specific class (e.g., a neighbor’s parked car that should never trigger a car alert).

Zones are named polygons that define semantically meaningful areas: front_stoop, driveway, sidewalk. Zones do not suppress detection — they add spatial metadata to events. The powerful tool is required_zones, used in the review configuration:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

cameras:
  front_door:
    zones:
      front_stoop:
        coordinates: 250,450,650,450,650,720,250,720
      sidewalk:
        coordinates: 0,300,1280,300,1280,450,0,450
    review:
      alerts:
        required_zones:
          - front_stoop
    objects:
      filters:
        person:
          min_score: 0.65
          threshold: 0.75

With this configuration, a person on the sidewalk will be detected and tracked — and visible in the timeline — but will not generate an alert notification until they enter front_stoop. This is the correct tool for separating background activity from actionable events. The min_score and threshold values are distinct: min_score is the per-frame floor, and threshold is the averaged confidence across frames that must be met to confirm a track as a true positive. Raising threshold without raising min_score is usually the right way to reduce false positives without missing real events.

Recording, Retention, and Storage Growth

Frigate separates recording behavior from detection: you can record continuously, on motion, or only on object detection events, with independent retention periods.

1
2
3
4
5
6
7
8
9

record:
  enabled: true
  retain:
    days: 3
    mode: motion        # all | motion | active_objects
  events:
    retain:
      default: 14       # days to keep event clips
      mode: active_objects

mode: all retains every segment regardless of activity and grows storage fastest. mode: motion keeps only segments where OpenCV detected motion — practical for outdoor cameras with variable activity. mode: active_objects keeps only segments where the AI detector confirmed a true-positive object — the most aggressive retention mode and the one that keeps your clips directory reviewable.

Storage sizing requires actual measurement from your specific camera compression and scene complexity, but rough guidelines for H.264 at 1080p/15fps: continuous recording consumes 1–2 GB per camera per day. Motion-only retention on a typical residential camera drops this to 200–600 MB per day. On a 4-camera outdoor setup with 7-day motion retention and 14-day event retention, expect 30–80 GB of active storage. Use NVR-rated drives (WD Purple, Seagate SkyHawk) for anything beyond a test setup — consumer drives are not rated for the continuous write workload of video surveillance.

Snapshots are JPEG files written per event and stored separately from recordings. They are negligible in storage terms individually but accumulate over time with aggressive event rates. Snapshots can be independently retained with snapshots.retain.default and are the image source for MQTT notification thumbnails.

The detect, record, and snapshot pipelines use separate resolution configurations:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

detect:
  width: 1280
  height: 720
  fps: 5

cameras:
  front_door:
    detect:
      width: 640
      height: 480
      fps: 5

Detection resolution should be the lowest that preserves object recognition accuracy — typically 640x480 or 1280x720 for most cameras at typical mounting distances. Higher detection resolution increases inference time and decode load with marginal accuracy gain. Record resolution is whatever your camera’s main stream provides. Snapshot resolution is typically the detect-resolution crop, which is why snapshots look like tight crops rather than full-frame images.

Home Assistant Integration

Frigate’s MQTT integration and the official Frigate Home Assistant custom component together deliver the tightest possible coupling between local camera AI and home automation. The integration is installed via HACS and auto-discovers cameras, binary sensors (person detected, car detected, etc.), and switches (detection toggle per camera) from the MQTT topics Frigate publishes.

1
2
3
4
5
6
7
8
9

# Frigate config.yml - enable MQTT
mqtt:
  enabled: true
  host: 192.168.1.10     # your MQTT broker (usually Mosquitto)
  port: 1883
  topic_prefix: frigate
  client_id: frigate
  user: frigate_user
  password: !secret mqtt_password

The Frigate integration creates entities automatically once MQTT is configured and the integration is added in Home Assistant. Each camera gets a binary sensor per tracked object class, a last-event image entity, and sensor entities for detection state. For deeper automation, the frigate/reviews MQTT topic is the recommended trigger — it carries the review ID, which can be used to construct the snapshot URL directly.

Example Home Assistant automation for a person alert with snapshot:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

alias: "Frigate: Person at Front Door"
trigger:
  - platform: mqtt
    topic: frigate/reviews
    payload: "new"
    value_template: "{{ value_json.type }}"
condition:
  - condition: template
    value_template: >
      {{ trigger.payload_json.after.camera == 'front_door'
         and 'person' in trigger.payload_json.after.data.detections
         and trigger.payload_json.after.severity == 'alert' }}
action:
  - service: notify.mobile_app_your_phone
    data:
      title: "Person detected"
      message: "Front door camera"
      data:
        image: >
          http://192.168.1.20:5000/api/review/{{
            trigger.payload_json.after.id }}/snapshot.jpg
        url: "http://192.168.1.20:5000"

The required_zones in the camera configuration feeds directly into whether an event achieves severity: alert (actionable) versus severity: detection (tracked but not notified). This is how you get zero notifications for sidewalk traffic while still recording every event for review.

See the Home Assistant home automation overview for broader context on building reliable HA automations, and Home Assistant OS deep dive for HA infrastructure prerequisites.

Hardware Sizing for Real Deployments

The right sizing question is not just “what detector can I use” — it is the combined budget of decode work, detection throughput, and storage I/O. A rule of thumb:

  • Each 1080p H.264 camera at 15 FPS in software decode costs 8–15% of a modern CPU core.
  • Hardware decode (VAAPI/NVDEC) drops this to 1–3% per stream.
  • A single Coral USB handles approximately 4–6 cameras at 5 FPS detection with headroom.
  • A Hailo-8L handles 8–12 cameras at 5 FPS detection comfortably.
  • An Nvidia GPU (ONNX) can handle 16+ cameras limited by decode bandwidth, not inference.

For a 4-camera build on a budget:

An Intel N100 mini PC with 16 GB RAM running Frigate in Docker is a solid foundation. VAAPI handles decode, the N100’s iGPU provides OpenVINO inference at 25–35ms, and the whole system draws under 20W. A used Intel NUC 11 or 12 with an M.2 slot for a Hailo-8L upgrades the inference path substantially. The homelab hardware guide covers small form factor options that work well as dedicated NVR hosts.

For 8–16 cameras at higher resolution, a dedicated host matters. Consider dedicated SAS/SATA HDD storage for recordings on a separate mount path from the OS drive, with the Frigate container’s /media/frigate volume pointing there.

Frigate+: Fine-Tuning for Your Environment

Frigate+ is a paid service ($50/year, includes 12 training credits) that allows you to submit labeled images from your own cameras and receive a fine-tuned model trained on the community corpus plus your specific camera angles, lighting conditions, and edge cases.

The base models available without a subscription use the COCO object set and perform reasonably well for common detections. The Frigate+ model adds training examples submitted by other Frigate+ users from real security camera deployments, which produces better results at typical camera resolutions, oblique angles, and night-vision conditions than models trained on academic datasets.

The fine-tuning workflow requires you to review events in the Frigate UI, label true positives and false positives in the Frigate+ web interface, submit them as training examples, and request a model build. New base models drop quarterly. Trained models download to /config/model_cache/ and are referenced in your detector configuration. Models remain available for download indefinitely even after a subscription lapses.

Frigate+ is worth it for setups that have persistent difficult detection scenarios — backlit cameras, unusual animal activity, uncommon vehicles — where the default model consistently misclassifies or misses objects after basic tuning.

On-Device vs. Cloud: The Honest Comparison

The case for Frigate over Ring or Google Nest is not primarily about money, though the math is favorable. Ring Protect Plus runs $100/year. Nest Aware Plus runs $180/year. A four-camera Ring install accumulates $500–900 in subscription fees over five years, before any hardware refresh. Frigate’s ongoing cost after initial hardware is zero.

The latency argument is concrete: cloud AI systems involve stream upload, backend inference, notification routing, and device delivery — commonly 3–8 seconds from event to phone notification. Frigate on a local LAN delivers MQTT messages within 1–2 seconds of the event frame, and with good tuning often under 500ms. For a package theft or door approach, 7 seconds is the difference between a recording and an intervention.

Privacy is the argument that is difficult to quantify but should not be understated. Ring’s law enforcement data-sharing programs, Google’s broad data retention practices, and documented cases of unauthorized access to cloud camera footage represent real risks for footage that includes your home interior, your children’s play areas, and your daily patterns. Frigate processes nothing outside your network. There is no API to subpoena, no corporation to pressure, and no breach to expose.

The honest downsides:

Setup complexity is real. A new Frigate deployment involves Docker networking, RTSP stream debugging, VAAPI device passthrough, detector hardware sourcing, zone polygon coordinate calculation, and MQTT broker configuration — before you touch Home Assistant. Plan 8–20 hours for a first install that is properly tuned. The Frigate UI’s debug view (showing motion boxes and detection zones in real-time) is invaluable for this work, but the feedback loop is slow.

You own the uptime. A misconfigured ffmpeg input, a failing hard drive, or a Docker container crash means no recording. There is no SLA, no support line, and no “works out of the box.” Monitoring Frigate itself (the /api/stats endpoint, watchdog scripts, MQTT heartbeat checks) is your responsibility.

Detector hardware is not always available. Coral USB units have periodic supply gaps. Hailo modules require locating a carrier board or M.2 enclosure and navigating SDK installation. Nvidia GPU ONNX inference requires an Nvidia host and correct driver-to-container-toolkit version pairing, and older GPU driver versions can break with ffmpeg symbol changes.

Storage management requires attention. The retention system works, but a misconfigured retention policy (or an outdoor camera in a high-motion environment with mode: all) can fill a drive in days. Monitor disk usage actively and set Frigate’s storage limits under record.retain.days conservatively until you have measured your actual growth rate.

Verdict

Frigate is the best self-hosted NVR for homelab environments where Home Assistant is already in use. Its architecture — go2rtc for stream management, hardware-accelerated decode, dedicated AI inference, MQTT event bus — is technically sound and the configuration model is explicit and auditable. The result is a surveillance system that produces fewer false positives than Ring after tuning, generates notifications faster than any cloud platform, and keeps all footage on hardware you own.

The investment is front-loaded. The first Frigate install is non-trivial. Detector hardware choices require research. Zone configuration requires iteration. But the system converges: once tuned, it runs reliably and the ongoing effort is minimal. The docker-compose homelab guide covers the container orchestration foundations you will need for a production Frigate deployment alongside other services.

If you have been tolerating Ring or Nest subscription fees and accepting their privacy terms by default, Frigate is the correct answer. The only question is which detector hardware fits your host.

Sources

Comments