Skip to main content
Version: v1.9

Model Compilation

note

Beta. The Pipeline Builder is in beta. See Pipeline Builder for what that means for your project.

The Voyager SDK compiler quantizes your model to mixed-precision and compiles it for the Metis AIPU, producing an .axm file. This page covers two paths: through a supported third-party integration (e.g. Ultralytics, where compilation is handled for you) or directly via the compiler API for any ONNX or PyTorch model.

Ultralytics Integration​

If your model is trained with Ultralytics, a single call handles quantization and compilation:

from ultralytics import YOLO

model = YOLO("yolo11n.pt")
model.export(format="axelera")
# Output: yolo11n_axelera_model/yolo11n.axm

The output directory yolo11n_axelera_model/ contains the .axm file ready for op.load().

To validate accuracy on the AIPU vs the original model, use yolo val:

yolo val model=yolo11n_axelera_model format=axelera

For full details, see the Ultralytics Axelera integration guide.

What Happens Under the Hood​

This shows what the Ultralytics integration does internally. It is useful if you're curious how the exporter works, or if you want to build a similar integration for another framework.

The Ultralytics exporter calls the same compiler API you can use directly:

from axelera import compiler
from axelera.compiler import CompilerConfig
from axelera.compiler.config.model_specific import extract_ultralytics_metadata

# 1. Extract Ultralytics-specific metadata (task type, class names, keypoint shape, etc.)
#    Generic models: skip this step and omit model_metadata from CompilerConfig.
metadata = extract_ultralytics_metadata(model)

# 2. Configure the compiler
config = CompilerConfig(
    model_metadata=metadata,
    aipu_cores_used=1,
    output_axm_format=True,
    model_name=model_name,
)

# 3. Quantize and compile
qmodel = compiler.quantize(
    model="yolo11n.onnx",
    calibration_dataset=calibration_images(),
    config=config,
    transform_fn=transform_fn,
)
compiler.compile(model=qmodel, config=config, output_dir=export_path)

The exporter does two things your own code needs to handle manually:

  1. model_metadata: Ultralytics-specific. The exporter extracts task type, class names, and keypoint shape and embeds them in the .axm; at runtime, pipeline.optimized() reads this to pick a native postamble kernel (see Postambles). For generic models, omit model_metadata and wire up your own postprocessing pipeline instead.

  2. Auto-tuned CompilerConfig: the exporter picks the right resources_used, quantization_scheme, tiling_depth, etc. for each model architecture. When you compile your own model, you set these yourself (sensible defaults work for most models).

Generic Path: From ONNX or PyTorch​

For models not trained with Ultralytics, use the compiler API directly.

Required settings for the pipeline builder: output_axm_format=True must be set so the compiler produces an .axm file that op.load() can consume. The pipeline builder supports aipu_cores_used=1 only: compile for one core, and the runtime spreads work across the available cores at run time (see core_allocation in the Pipeline Overview).

From an ONNX Model​

from pathlib import Path
from axelera import compiler
from axelera.compiler import CompilerConfig

config = CompilerConfig(
    model_name="my_detector",
    aipu_cores_used=1,
    resources_used=0.25,        # fraction of device memory to use
    output_axm_format=True,     # required for op.load()
)

# Provide a calibration dataset: an iterator yielding numpy arrays
# matching the model's input shape and dtype (typically float32 NCHW)
def calibration_data():
    for path in Path("calibration_images/").glob("*.jpg"):
        img = preprocess(path)  # your preprocessing: resize, normalize, etc.
        yield img

qmodel = compiler.quantize(
    model="model.onnx",
    calibration_dataset=calibration_data(),
    config=config,
)

compiler.compile(
    model=qmodel,
    config=config,
    output_dir=Path("compiled_output/"),
)

For model-specific tuning (e.g., quantization_scheme, tiling_depth), see the Full Compiler Config Reference.

From a PyTorch Model with DataLoader​

from pathlib import Path
from torch.utils.data import DataLoader
from axelera import compiler
from axelera.compiler import CompilerConfig

config = CompilerConfig(
    model_name="my_classifier",
    aipu_cores_used=1,
    resources_used=0.25,
    output_axm_format=True,     # required for op.load()
)
loader = DataLoader(my_dataset, batch_size=1)

def extract_images(batch):
    images, labels = batch
    return images

qmodel = compiler.quantize(
    model=torch_model,
    calibration_dataset=loader,
    config=config,
    transform_fn=extract_images,
)

compiler.compile(
    model=qmodel,
    config=config,
    output_dir=Path("compiled_output/"),
)

Finding Your .axm​

After compilation, the .axm file is named after model_name in your CompilerConfig. The output_dir you pass to compiler.compile() receives intermediate build artifacts (manifests, quantized graphs, etc.), and the .axm is written next to it, in its parent directory.

For example, with model_name="my_detector" and output_dir=Path("compiled_output/"):

$ ls
compiled_output/  my_detector.axm

The Ultralytics exporter automatically organizes output into \<model_name\>_axelera_model/. When using the API directly, you handle file placement yourself.

Once you have the .axm, see the Pipeline Overview for how to build inference pipelines around it.

Validate Before Deploying​

After quantization, you can run the quantized model on CPU to check accuracy before compiling for hardware:

import torch

# Run a sample through the quantized model on CPU
sample = torch.randn(1, 3, 640, 640)
output = qmodel(sample)
# Compare output against the original model to verify quantization quality

Next Steps​