Skip to main content

axmonitor

axmonitor provides real-time visibility into Metis device metrics — temperatures, utilization, memory, power, and FPS — similar to how htop or nvidia-smi work for CPUs and GPUs.


What it monitors

MetricDescription
Core utilizationCPU usage per core and thread
TemperaturesPer-core and board temperatures with thermal threshold indicators
Kernels Per Second (KPS)Number of fused AI kernels executed per second (≈ FPS for vision pipelines)
PowerReal-time power consumption (Metis PCIe 4-AIPU card only)
DDR memoryMemory allocation per active application context
PCIe bandwidthDMA bandwidth per channel in MB/s
ProcessesRunning processes with PIDs and container IDs (Linux only)
Device versionFirmware version, hardware ID, protocol version
ConfigurationThermal management settings, clock frequency per core

Architecture

axmonitor is a frontend that connects to axsystemserver, a backend service that collects and streams device metrics over TCP. You must start the backend before running the tool.

axsystemserver  ←── collects device metrics
↕ TCP (default: 127.0.0.1:5555)
axmonitor ──── displays metrics

Setup

Start the backend service

Linux:

sudo systemctl start axsystemserver.service

Verify it's running:

sudo systemctl status axsystemserver.service

Linux (Docker):

sudo service axsystemserver start
sudo service axsystemserver status

Windows (install as Windows Service first, from an admin console):

axsystemserver --install-service
sc.exe start axsystemserver

Verify:

sc.exe query axsystemserver   # look for STATE: RUNNING

Installing after pip installation

note

If you followed the pip installation guide, install the base package before using axmonitor. If you used the SDK installer, skip this step.

# Add the Axelera apt repository (if not already configured)
sudo sh -c "curl -fsSL https://software.axelera.ai/artifactory/api/security/keypair/axelera/public | gpg --dearmor -o /etc/apt/keyrings/axelera.gpg"
# Ubuntu 22.04
sudo sh -c "echo 'deb [signed-by=/etc/apt/keyrings/axelera.gpg] https://software.axelera.ai/artifactory/axelera-apt-source ubuntu22 main' > /etc/apt/sources.list.d/axelera.list"
# Ubuntu 24.04
sudo sh -c "echo 'deb [signed-by=/etc/apt/keyrings/axelera.gpg] https://software.axelera.ai/artifactory/axelera-apt-source ubuntu24 main' > /etc/apt/sources.list.d/axelera.list"

sudo apt-get update
sudo apt-get install -y axelera-voyager-sdk-base-1.6.0
source /opt/axelera/sdk/1.6.0/axelera_activate.sh

Run axmonitor

tip

Make sure the SDK virtual environment is activated (source venv/bin/activate) before running axmonitor.

axmonitor --server-address "127.0.0.1:5555"

This connects to the backend and opens the monitoring interface. Metrics update every second.


Interface modes

GUI (default)

The GUI organizes metrics into three sections:

  • OVERVIEW — dashboard with key metrics and visualizations
  • SYSTEM — process monitoring, I/O metrics
  • DEVICE — per-device detail: CPU, memory, temperature, power, firmware, configuration

Console mode

For SSH sessions or scriptable environments:

axmonitor --server-address "127.0.0.1:5555" --ui console

Interactive commands at the axmonitor> prompt:

CommandDescription
printPrint current measurements
helpList available commands
exitQuit

One-shot query

axmonitor --server-address "127.0.0.1:5555" --ui console -c print

Prints current metrics and exits — useful in scripts.


Multi-device monitoring

To monitor specific devices:

axmonitor --server-address "127.0.0.1:5555" --topics DEV0 DEV1

Topics are DEV0, DEV1, ... DEVN where N is the number of connected devices. Default: DEV0.


Changing the service port

If port 5555 is in use, reconfigure axsystemserver.

Linux — edit /lib/systemd/system/axsystemserver.service:

ExecStart=/bin/bash -c 'exec axsystemserver --bind "127.0.0.1:5556"'

Then restart: sudo systemctl restart axsystemserver.service

Windows — reinstall with a different port:

sc.exe delete axsystemserver
axsystemserver --install-service --bind 127.0.0.1:5556

Recording and export

axmonitor can record metrics to disk and export them in human-readable or structured formats for offline analysis.

Record a session

Use --record to save all received measurements to a .raw binary file:

axmonitor --server-address "127.0.0.1:5555" --record

By default the file is written to axmonitor.data.raw. Use --data-file to change the base path:

axmonitor --server-address "127.0.0.1:5555" --record --data-file my_session

Export to text formats

Use --export together with --record to write metrics alongside the .raw file:

FormatExtensionDescription
json.jsonStructured JSON for scripting and post-processing
pretty.prettyHuman-readable text output

Multiple formats can be specified at once:

axmonitor --server-address "127.0.0.1:5555" --record --export json pretty

This produces axmonitor.data.raw, axmonitor.data.json, and axmonitor.data.pretty.

Replay a recorded session

Use --replay to replay a previously recorded .raw file through the normal UI without a live device:

axmonitor --replay --data-file my_session

Warnings and threshold events

axmonitor displays current temperatures and highlights active thermal thresholds visually, but does not emit event-based messages when thresholds are crossed. Threshold crossing events — such as software throttling activating or frequency downscaling engaging — are only visible in the device firmware log stream.

To capture these events in real time, stream the firmware log using axlogdevice:

axlogdevice --slog-level inf:service_monitor
axlogdevice --slog
note

These events are not surfaced in axmonitor output or in axmonitor --export data files. If you need to correlate thermal events with performance data, run axlogdevice alongside axmonitor in a separate terminal.


Troubleshooting

No data displayed — the most common cause is axsystemserver not running or a port conflict.

Check if the service is running (see Setup above). Check if the port is in use:

# Linux
sudo lsof -i :5555

# Windows
netstat -ano | findstr :5555

If the port is occupied, change the service configuration to a free port, then launch axmonitor with --server-address pointing to the new port.

Check if messages are arriving:

axmonitor --server-address "127.0.0.1:5555" -l info

If the log shows no messages being received, the backend isn't running.


See also