Skip to main content
Version: v1.7

Model Compilation

important

Alpha Core operators (detection, classification, pose, segmentation, tracking) are stable. Cascade (op.for_each, op.crop_roi) and streaming APIs are still in development.

The Voyager SDK compiler quantizes your model to mixed-precision and compiles it for the Metis AIPU, producing an .axm file. This page covers two paths: through a supported third-party integration (e.g. Ultralytics, where compilation is handled for you) or directly via the compiler API for any ONNX or PyTorch model.

Ultralytics Integration​

If your model is trained with Ultralytics, a single call handles quantization and compilation:

from ultralytics import YOLO

model = YOLO("yolo11n.pt")
model.export(format="axelera")
# Output: yolo11n_axelera_model/yolo11n.axm

The output directory yolo11n_axelera_model/ contains the .axm file ready for op.load().

To validate accuracy on the AIPU vs the original model, use yolo val:

yolo val model=yolo11n_axelera_model format=axelera

For full details, see the Ultralytics Axelera integration guide.

What Happens Under the Hood​

This shows what the Ultralytics integration does internally — useful if you're curious how the exporter works, or if you want to build a similar integration for another framework.

The Ultralytics exporter calls the same compiler API you can use directly:

from axelera import compiler
from axelera.compiler import CompilerConfig
from axelera.compiler.config.model_specific import extract_ultralytics_metadata

# 1. Extract Ultralytics-specific metadata (task type, class names, keypoint shape, etc.)
#    Generic models: skip this step and omit model_metadata from CompilerConfig.
metadata = extract_ultralytics_metadata(model)

# 2. Configure the compiler
config = CompilerConfig(
    model_metadata=metadata,
    aipu_cores_used=1,
    output_axm_format=True,
    model_name=model_name,
)

# 3. Quantize and compile
qmodel = compiler.quantize(
    model="yolo11n.onnx",
    calibration_dataset=calibration_images(),
    config=config,
    transform_fn=transform_fn,
)
compiler.compile(model=qmodel, config=config, output_dir=export_path)

The exporter does two things your own code needs to handle manually:

  1. model_metadata -- Ultralytics-specific. The exporter extracts task type, class names, and keypoint shape and embeds them in the .axm; at runtime, op.load() reads this to auto-select optimized C++ postprocessing. For generic models, omit model_metadata and wire up your own postprocessing pipeline instead.

  2. Auto-tuned CompilerConfig -- the exporter picks the right resources_used, quantization_scheme, tiling_depth, etc. for each model architecture. When you compile your own model, you set these yourself (sensible defaults work for most models).

Generic Path: From ONNX or PyTorch​

For models not trained with Ultralytics, use the compiler API directly.

Required settings for the pipeline builder: output_axm_format=True must be set so the compiler produces an .axm file that op.load() can consume. The pipeline builder supports aipu_cores_used=1 only.

From an ONNX Model​

from pathlib import Path
from axelera import compiler
from axelera.compiler import CompilerConfig

config = CompilerConfig(
    model_name="my_detector",
    aipu_cores_used=1,
    resources_used=0.25,        # fraction of device memory to use
    output_axm_format=True,     # required for op.load()
)

# Provide a calibration dataset: an iterator yielding numpy arrays
# matching the model's input shape and dtype (typically float32 NCHW)
def calibration_data():
    for path in Path("calibration_images/").glob("*.jpg"):
        img = preprocess(path)  # your preprocessing: resize, normalize, etc.
        yield img

qmodel = compiler.quantize(
    model="model.onnx",
    calibration_dataset=calibration_data(),
    config=config,
)

compiler.compile(
    model=qmodel,
    config=config,
    output_dir=Path("compiled_output/"),
)

For model-specific tuning (e.g., quantization_scheme, tiling_depth), see the Full Compiler Config Reference.

From a PyTorch Model with DataLoader​

from pathlib import Path
from torch.utils.data import DataLoader
from axelera import compiler
from axelera.compiler import CompilerConfig

config = CompilerConfig(
    model_name="my_classifier",
    aipu_cores_used=1,
    resources_used=0.25,
    output_axm_format=True,     # required for op.load()
)
loader = DataLoader(my_dataset, batch_size=1)

def extract_images(batch):
    images, labels = batch
    return images

qmodel = compiler.quantize(
    model=torch_model,
    calibration_dataset=loader,
    config=config,
    transform_fn=extract_images,
)

compiler.compile(
    model=qmodel,
    config=config,
    output_dir=Path("compiled_output/"),
)

Finding Your .axm​

After compilation, the .axm file is named after model_name in your CompilerConfig. The output_dir you pass to compiler.compile() receives intermediate build artifacts (manifests, quantized graphs, etc.), while the .axm is placed relative to it -- typically in the current working directory.

For example, with model_name="my_detector":

$ ls *.axm
my_detector.axm

The Ultralytics exporter automatically organizes output into <model_name>_axelera_model/. When using the API directly, you handle file placement yourself.

Once you have the .axm, see the Pipeline Overview for how to build inference pipelines around it.

Validate Before Deploying​

After quantization, you can run the quantized model on CPU to check accuracy before compiling for hardware:

import numpy as np

# Run a sample through the quantized model on CPU
sample = np.random.randn(1, 3, 640, 640).astype(np.float32)
output = qmodel(sample)
# Compare output against the original model to verify quantization quality

Next Steps​