VISION ENGINE

See, track, and act on-device

A model registry with custom model upload and on-vehicle inference. Detect generic classes like person and car, track them frame to frame, and feed the flight stack. Inference never leaves the aircraft.
View on GitHubBrowse extensions
3
Inference backends
0
Cloud inference calls
3
Model formats
Open
Model registry

Detections on the aircraft's own compute.

Detections on the aircraft's own compute.

How it works

Camera to detection bus, all on the companion

Frames flow from the camera through inference and tracking, then out on a local detection stream that plugins subscribe to. Inference never leaves the vehicle.

Vision pipeline
CameraCSI or USB
Frame capture
InferenceNPU / CPU
Object detectionperson, car, generic classes
TrackingTrack and re-identify
track id and lock
Detection busvision.detection
plugins subscribe
Detection runs on the board's NPU or CPU. Tracks are published on the vision.detection stream; a plugin such as follow-me turns a track into a guided setpoint for the flight controller.

Capabilities

A registry, not a single model

Model registry

Fetch, download, and cache detection models on the companion. Models stay on the device and run locally.

Custom model upload

Sideload your own model from Mission Control. The agent records the file's SHA-256 in its model catalog.

On-vehicle inference

The agent matches the model variant to the board family and the NPU TOPS its board profile declares.

Vision navigation

GPS-denied position hold is a separate install: the ADOS Vision Navigation extension reads the frame bus and feeds the flight controller optical flow over MAVLink.

Hardware

Three inference backends

The registry carries a model variant per backend. A Rockchip board routes to its NPU automatically, a Pi 5 with an AI HAT+ selects Hailo in its config, and a board with no accelerator runs the detector on the CPU through ONNX Runtime.

  • Rockchip RKNN

    Hardware (0.8 to 6 TOPS NPU), Model file (.rknn), Selection (Automatic on Rockchip boards)

  • Hailo-8 / 8L

    Hardware (Pi AI HAT+), Model file (.hef), Selection (Chosen in config)

  • CPU ONNX

    Hardware (Any Linux board), Model file (.onnx), Selection (Automatic without an NPU)

Vision navigation

Position hold without GPS, from optical flow

Vision navigation is an extension, not part of the agent. Install it from the registry and a downward camera feeds the flight controller a 30 Hz angular-flow measurement over MAVLink.

Install it from the extension catalog

The drone agent's web UI lists the extensions the aircraft can run. Vision navigation installs from there like any other plugin.

my-drone.local:8080
The extension catalog in the drone agent's web interfaceThe extension catalog in the drone agent's web interface

Three modes

Every tick the extension checks itself. If flow quality drops or the scale source goes unhealthy it steps down the scale ladder and warns the ground station at each step; a sustained degraded streak flips the node to CRITICAL.

  • Optical flow + rangefinderLow altitude, textured ground
  • Optical flow onlyBaro or GPS scale, drifts above 5 m
  • Off

Works with a stock flight controller

ArduPilot 4.5+, PX4 1.14+, and iNav 7.0+ on a USB UVC or CSI camera, with a Kalibr camera calibration file and a per-mode pre-arm gate. There is no visual-inertial odometry in the pipeline.

Related

Explore the stack

Sensor FusionMulti-sensor fusion hub design for the companion computerLearn moreAtlasCapture-to-reconstruction world model: splats, point clouds, meshLearn more

Put vision on your drone

The model registry is open-source and documented on GitHub. Bring your own model or start from the catalog.

View on GitHub
Get Early Access