benchmarks/ultralytics/README.md

Ultralytics benchmark

This optional integration runs the YOLO26n runtime through the Ultralytics validation harness for accuracy and performance comparison with other runtimes.

Usage

Run these commands from the yolo26-raspberry-pi-5/ runtime directory. Install the project prerequisites first.

Run the benchmark:

make ultralytics-benchmark

The target prepares any missing default runtime, converted model, pinned benchmark environment, and pinned revision of the public Arm/coco-128-by-sa dataset.

Select another Ultralytics dataset configuration with DATA:

make ultralytics-benchmark DATA=/path/to/dataset.yaml

Override RUNTIME or MODEL to measure existing artifacts:

make ultralytics-benchmark \
  RUNTIME=/path/to/libarm-yolo26n.so.0 \
  MODEL=/path/to/converted-model

Each invocation:

1. obtains the default runtime and converts the pinned YOLO26n checkpoint when those artifacts are missing;
2. creates or reuses the pinned Ultralytics virtual environment;
3. applies the integration patch and installs this runtime's backend;
4. runs validation and records the result.

See Recording Ultralytics benchmark evidence for the output layout.

Layout

• arm_yolo26n_pi5_backend.py adapts the runtime to the Ultralytics backend interface.
• patches/ contains the patch that registers the backend with the pinned Ultralytics release.
• patch.sh prepares the virtual environment and applies the integration patch.
• benchmark.py creates the temporary model descriptor and runs the recorded benchmark.