Claude Skill

routeros-qemu-chr

MikroTik RouterOS CHR (Cloud Hosted Router) with QEMU. Use when: running RouterOS in QEMU, booting CHR images, debugging CHR boot failures, setting up VirtIO devices for RouterOS, choosing between SeaBIOS and UEFI boot, configuring QEMU port forwarding for RouterOS REST API, sett

LLM Mart · 0 points · 0 views 0 listing impressions 0 install-command copies
Download tikoci-routeros-skills-routeros-qemu-chr-09eda23.zip · 16 KB

Install

skills CLI npx skills add https://github.com/tikoci/routeros-skills/tree/main/routeros-qemu-chr
Claude Code claude plugin marketplace add https://llmmart.ai/marketplace.json && claude plugin install tikoci-routeros-skills@llmmart
Git git clone https://github.com/tikoci/routeros-skills.git

The skills CLI installs just this skill, for any of its supported agents. Claude Code installs the whole tikoci/routeros-skills collection as a plugin from our marketplace. Git is the plain clone.

Skill manifest

RouterOS CHR with QEMU

What Is CHR

Cloud Hosted Router (CHR) is MikroTik's x86_64 and aarch64 RouterOS image designed for virtual machines. Free license allows unlimited use with 1 Mbps speed limit — sufficient for development, testing, API work, and packet sniffer debugging. A free 60-day trial removes the speed limit entirely (requires a free mikrotik.com account). See CHR licensing reference for full details on license tiers, trial activation, and expiry behavior.

Image Variants

Image Architecture Boot method Source
chr-<ver>.img x86_64 SeaBIOS (MBR chain-load) download.mikrotik.com
chr-<ver>-arm64.img aarch64 UEFI (EDK2 pflash) download.mikrotik.com
chr-efi.img (fat-chr) x86_64 UEFI (OVMF) tikoci/fat-chr GitHub

Standard x86 image has a proprietary boot partition — it looks like an EFI System Partition in GPT but is NOT FAT. UEFI firmware (OVMF) cannot read it. Only SeaBIOS can boot it via MBR chain-load.

The fat-chr repackaged image converts this to standard FAT16 with EFI/BOOT/BOOTX64.EFI, enabling UEFI boot. Required for Apple Virtualization.framework on X86 macOS, optional everywhere else.

Disk layout (128 MiB, both architectures): Hybrid GPT+MBR, partition 1 = boot (~33 MiB), partition 2 = ext4 root (~94 MiB).

Downloading CHR Images

// Resolve current version
const channel = "stable"; // or: long-term, testing, development
const version = await fetch(
  `https://upgrade.mikrotik.com/routeros/NEWESTa7.${channel}`
).then(r => r.text()).then(s => s.trim());

// Download x86_64 image
const url = `https://download.mikrotik.com/routeros/${version}/chr-${version}.img.zip`;
// Download aarch64 image
const armUrl = `https://download.mikrotik.com/routeros/${version}/chr-${version}-arm64.img.zip`;

Images are distributed as .img.zip — unzip to get the raw .img disk file.

Pattern Choices: QEMU Invocation

There are several valid approaches to launching CHR under QEMU. Each has tradeoffs:

Pattern A: Inline arguments (simplest, good for scripts)

Everything on the command line. Easy for an LLM to construct and debug — all state is visible in one place.

qemu-system-x86_64 -M q35 -m 256 -smp 1 \
  -drive file=chr.img,format=raw,if=virtio \
  -netdev user,id=net0,hostfwd=tcp::9180-:80 \
  -device virtio-net-pci,netdev=net0 \
  -display none -serial stdio

Pros: Single command, easy to read, easy to modify. Cons: Long command lines, hard to version-control, no persistence.

Pattern B: Wrapper script (good for reuse)

A shell script that detects acceleration, handles firmware paths, manages PID files.

#!/bin/sh
# detect acceleration
if [ "$(uname -s)" = "Linux" ] && [ -w /dev/kvm ]; then
  ACCEL="-accel kvm"
elif [ "$(uname -s)" = "Darwin" ] && [ "$(sysctl -n kern.hv_support 2>/dev/null)" = "1" ]; then
  ACCEL="-accel hvf"
else
  ACCEL="-accel tcg"
fi

qemu-system-x86_64 -M q35 -m 256 -smp 1 \
  $ACCEL \
  -drive file=chr.img,format=raw,if=virtio \
  -netdev user,id=net0,hostfwd=tcp::${PORT:-9180}-:80 \
  -device virtio-net-pci,netdev=net0 \
  -display none -serial stdio

Pros: Portable, handles platform differences, parameterizable. Cons: Shell scripting limitations, harder to compose from TypeScript.

Pattern C: Programmatic launch from Bun/TypeScript (good for integration tests)

Launch QEMU as a child process with full control:

import { $ } from "bun";

const port = 9180;
const accel = await detectAccel();
const proc = Bun.spawn([
  "qemu-system-x86_64", "-M", "q35", "-m", "256",
  "-accel", accel,
  "-drive", `file=chr.img,format=raw,if=virtio`,
  "-netdev", `user,id=net0,hostfwd=tcp::${port}-:80`,
  "-device", "virtio-net-pci,netdev=net0",
  "-display", "none",
  "-chardev", `socket,id=serial0,path=/tmp/chr-serial.sock,server=on,wait=off`,
  "-serial", "chardev:serial0",
  "-monitor", `unix:/tmp/chr-monitor.sock,server,nowait`,
], { stdio: ["ignore", "pipe", "pipe"] });

// Wait for boot
await waitForBoot(`http://127.0.0.1:${port}/`);

Pros: Full lifecycle control, parallel instance management, TypeScript-native. Cons: More code, QEMU args still need to be correct.

Pattern D: Config file (--readconfig) (declarative, used by mikropkl)

QEMU's --readconfig loads an INI-format file for device/machine config. The mikropkl project uses this for its declarative VM packaging.

Tradeoffs: Separates concerns (config vs launch), but the INI format is obscure and not all QEMU options can be expressed in it (pflash, -accel, -netdev user,hostfwd all require command-line args). Best suited for projects that generate configs programmatically.

Boot Tracks

x86_64 with SeaBIOS (default, fastest)

No firmware setup needed — QEMU's built-in SeaBIOS handles MikroTik's proprietary boot sector:

qemu-system-x86_64 -M q35 -m 256 \
  -drive file=chr-7.22.img,format=raw,if=virtio \
  -netdev user,id=net0,hostfwd=tcp::9180-:80 \
  -device virtio-net-pci,netdev=net0 \
  -display none -serial stdio

-M pc (i440fx, legacy PCI) also works for CHR. CHR doesn't exercise most of what q35 adds (modern PCIe topology, ACPI-based hotplug, etc.) because the RouterOS kernel bypasses the BIOS/ACPI stack after boot — the VirtIO PCI devices are what it actually uses, and both machine types expose those identically. For the kernel-config evidence behind this, see tikoci/mikrotik-gpl (v7.2 kernel config archive).

Boot time: ~5s (KVM), ~30s (TCG). For TCG, -accel tcg,tb-size=256 enlarges the translation block cache and reduces boot time noticeably on repeated runs.

aarch64 with UEFI (EDK2)

Requires UEFI pflash firmware files. Both pflash units must be identical size (typically 64 MiB):

# Copy vars file (writable) — never modify the original
cp /path/to/edk2-arm-vars.fd /tmp/my-vars.fd

qemu-system-aarch64 -M virt -cpu cortex-a710 -m 256 \
  -drive if=pflash,format=raw,readonly=on,unit=0,file=/path/to/edk2-aarch64-code.fd \
  -drive if=pflash,format=raw,unit=1,file=/tmp/my-vars.fd \
  -drive file=chr-arm64.img,format=raw,if=none,id=drive0 \
  -device virtio-blk-pci,drive=drive0 \
  -netdev user,id=net0,hostfwd=tcp::9180-:80 \
  -device virtio-net-pci,netdev=net0 \
  -display none -serial stdio

Boot time: ~10s (KVM), ~20s (TCG native), ~20s (TCG cross-arch on x86 host).

UEFI Firmware Locations

Platform Code ROM Vars File
macOS Homebrew (Apple Silicon) /opt/homebrew/share/qemu/edk2-aarch64-code.fd edk2-arm-vars.fd
macOS Homebrew (Intel) /usr/local/share/qemu/edk2-aarch64-code.fd edk2-arm-vars.fd
Ubuntu/Debian /usr/share/AAVMF/AAVMF_CODE.fd AAVMF_VARS.fd
x86 OVMF (Homebrew) edk2-x86_64-code.fd edk2-i386-vars.fd
x86 OVMF (Linux) /usr/share/OVMF/OVMF_CODE.fd OVMF_VARS.fd

VirtIO — Critical Details

See the VirtIO driver matrix for the full table.

The one rule: RouterOS has virtio_pci but NOT virtio_mmio. This matters on aarch64.

The if=virtio Trap (aarch64)

                         x86_64 (q35)              aarch64 (virt)
if=virtio shorthand →    virtio-blk-pci (PCI) ✅    virtio-blk-device (MMIO) ❌
-device virtio-blk-pci → virtio-blk-pci (PCI) ✅    virtio-blk-pci (PCI) ✅

On x86_64 q35, if=virtio resolves to PCI — works fine. On aarch64 virt, it resolves to MMIO — RouterOS kernel stalls silently. Always use explicit -device virtio-blk-pci on aarch64:

# WRONG on aarch64 — silent boot failure
-drive file=chr.img,format=raw,if=virtio

# CORRECT on aarch64 — explicit PCI device
-drive file=chr.img,format=raw,if=none,id=drive0
-device virtio-blk-pci,drive=drive0

On x86_64, both work. The explicit form is always safe on both architectures.

Network — Universal

All architectures: virtio-net-pci. No exceptions:

-netdev user,id=net0,hostfwd=tcp::9180-:80
-device virtio-net-pci,netdev=net0

user (SLIRP) is a userspace NAT — ideal for management (hostfwd of REST/SSH/WinBox), but it terminates Layer 2: broadcasts and non-forwarded UDP never reach the host or other VMs. For anything L2 (inter-VM links, neighbor discovery, MAC-Telnet) add a second NIC on a socket netdev.

L2 Networking: socket netdevs (inter-VM links + host capture)

QEMU's socket netdev carries raw Ethernet frames over a host socket — rootless, no extra setup:

# TCP, point-to-point — one side listens, the other connects
-netdev socket,id=net1,listen=:4001            # VM A (start first)
-netdev socket,id=net1,connect=127.0.0.1:4001  # VM B
# UDP multicast — shared L2 segment for N VMs
-netdev socket,id=net1,mcast=230.0.0.1:4001

⚠ mcast is broken on macOS. QEMU's mcast socket sets only SO_REUSEADDR, but macOS/BSD require SO_REUSEPORT on every socket sharing a multicast port — so on macOS nothing is delivered between local sockets: two VMs on the same group don't discover each other, and a host listener gets zero frames. It works on Linux (and CI). For point-to-point on macOS use TCP listen/connect; there is no rootless multi-VM L2 segment on macOS today (use socket_vmnet, privileged-once, for that).

Host capture of guest L2 frames (e.g. MNDP)

To let a host process receive a guest's broadcasts — MNDP neighbor discovery (the protocol WinBox's "Neighbors" tab uses, UDP/5678) being the first case — make the host one end of a TCP socket netdev:

  1. Host runs a TCP server on a loopback port.
  2. CHR gets a second NIC: -netdev socket,id=net1,connect=127.0.0.1:<port> — the host must be listening before QEMU starts, since QEMU is the connecting side.
  3. QEMU streams every guest frame to the host, length-prefixed: a 4-byte big-endian length, then the raw Ethernet frame.
  4. Host strips the prefix and parses Ethernet → IPv4 → UDP/5678 → MNDP TLVs.

Writing a length-prefixed frame back over the same connection injects L2 into the guest (e.g. an MNDP request, which draws a broadcast announcement after a short random delay — see routeros-mndp for the 8-byte payload; the same primitive MAC-Telnet needs). Loopback-only, so nothing leaks onto the LAN; works on macOS, Linux, and Windows alike.

Reference implementation + verified evidence: tikoci/quickchr (examples/mndp/, docs/mndp.md, test/lab/mndp/REPORT.md). For the MNDP wire format and TLV table, see the routeros-mndp skill.

Acceleration Detection

import { $ } from "bun";

async function detectAccel(guestArch: string): Promise<string> {
  const hostOs = process.platform;  // "darwin" | "linux"
  const hostArch = process.arch;    // "x64" | "arm64"

  if (hostOs === "linux") {
    // KVM requires host/guest architecture match
    const kvm = await Bun.file("/dev/kvm").exists();
    const archMatch = (guestArch === "x86_64" && hostArch === "x64")
      || (guestArch === "aarch64" && hostArch === "arm64");
    if (kvm && archMatch) return "kvm";
  }

  if (hostOs === "darwin") {
    // HVF may not be available (e.g., GitHub Actions VMs)
    const hvOk = await $`sysctl -n kern.hv_support`.text().then(s => s.trim() === "1").catch(() => false);
    const archMatch = (guestArch === "aarch64" && hostArch === "arm64")
      || (guestArch === "x86_64" && hostArch === "x64");
    if (hvOk && archMatch) return "hvf";
  }

  return "tcg";  // Software emulation — always available
}

Key rule: KVM and HVF both require host/guest architecture match. Cross-arch always falls back to TCG. Don't check just for /dev/kvm — verify the architecture matches too.

HVF + CPU Model Gotcha (macOS)

With -accel hvf, QEMU exposes the host CPU directly. Specifying a CPU model like cortex-a710 (ARMv9, requires SVE2) on Apple Silicon (ARMv8.5) crashes QEMU before the VM starts. Use -cpu host with HVF:

# TCG/KVM — specify exact model
CPU_FLAGS="-cpu cortex-a710"

# HVF — passthrough host CPU
if [ "$ACCEL" = "hvf" ]; then
  CPU_FLAGS="-cpu host"
fi

Health Check and Boot Wait

RouterOS WebFig responds with HTTP 200 on port 80 without authentication — this works as a minimal health check:

async function waitForBoot(url: string, timeoutMs = 60_000): Promise<boolean> {
  const deadline = Date.now() + timeoutMs;
  while (Date.now() < deadline) {
    try {
      const r = await fetch(url, { signal: AbortSignal.timeout(2000) });
      if (r.ok) return true;
    } catch { /* not ready yet */ }
    await Bun.sleep(2000);
  }
  return false;
}

Startup race — pitfall if you then call the REST API immediately

RouterOS boot is staged from the client's perspective:

  1. Connection refused — QEMU still booting
  2. ECONNRESET — HTTP server up but REST subsystem not accepting
  3. 401 from /rest/* — auth middleware up; REST handler may still be initializing
  4. 200 but wrong body — REST initialized before routing tables settled; /system/resource can briefly return an array (e.g., a /user list) before it returns the expected singleton object
  5. 200 with correct body — fully operational

A health check on the root / reaches stage 2 but gives no signal about 3–5. For code that will immediately call the REST API, probe /rest/system/resource with auth and require two consecutive successful probes with the expected body shape (singleton object containing board-name). On auth-only checks (unprovisioned admin), accept 401/403 as "REST layer responded." The / + HTTP 200 check is fine for "is it up?" monitoring but not for "can I start calling REST now?"

Port Forwarding

QEMU user-mode networking (-netdev user,hostfwd=...) for typical RouterOS services:

Service Guest Port Example Host Port hostfwd
WebFig/REST API 80 9180 tcp::9180-:80
SSH (RouterOS CLI) 22 9122 tcp::9122-:22
API protocol 8728 9728 tcp::9728-:8728
API-SSL 8729 9729 tcp::9729-:8729
WinBox 8291 9291 tcp::9291-:8291

Multiple forwards in one netdev:

-netdev user,id=net0,hostfwd=tcp::9180-:80,hostfwd=tcp::9122-:22,hostfwd=tcp::9728-:8728

Use unique host ports per instance when running multiple CHRs (9180, 9181, 9182...).

Known Limitations

  • QGA (Guest Agent) requires KVM — RouterOS CHR's QGA daemon only starts when it detects a KVM hypervisor via CPUID. Under HVF (macOS) or TCG (software emulation), CPUID 0x40000000 returns no KVM vendor string and 0x40000001 returns no KVM features, so the daemon never starts. QEMU correctly provides the virtio-serial port and sends PORT_OPEN (event 6) — the guest simply never opens it (query-chardev shows frontend-open=false). This is NOT a QEMU bug. MikroTik documents QGA exclusively under the "KVM" section. QGA testing requires Linux + KVM (e.g., mikropkl lab).
  • check-installation fails on aarch64 in all QEMU environments — this is an unresolvable firmware/DTB issue (see known issues)
  • Direct -kernel boot does not work for either architecture — RouterOS needs its full firmware boot path
  • Cross-arch TCG: x86_64 on aarch64 host is not viable — x86 I/O port emulation is too slow (~300s+ timeouts). The reverse (aarch64 on x86_64) works fine (~20s)
  • No virtio_mmio driver — always use explicit -device virtio-blk-pci, never rely on if=virtio on aarch64
  • socket,mcast= inter-VM networking is broken on macOS — QEMU sets only SO_REUSEADDR; macOS/BSD need SO_REUSEPORT to share a multicast port, so no frames are delivered between local sockets (VMs don't discover each other; host capture gets nothing). Use TCP socket (listen/connect) for point-to-point, or socket_vmnet for a shared segment. Works on Linux/CI. Verified in tikoci/quickchr (test/lab/mndp/REPORT.md)

Additional Resources

  • VirtIO driver matrix — full driver support table
  • Known issues — boot failures, cross-arch limitations
  • GitHub Actions CI patterns — running CHR on GitHub-hosted runners
  • CHR licensing — free tier (1 Mbps), 60-day trial, paid tiers, expiry behavior
  • For the quickchr reference implementation — driving CHR from tests & automation (QuickCHR.start, exec/rest, networking recipes, the connection surface for harnesses): see the routeros-tikoci-quickchr skill
  • For RouterOS CLI/REST once booted: see the routeros-fundamentals skill
  • For packet capture and TZSP streaming from CHR: see the routeros-sniffer skill
  • For MNDP neighbor-discovery wire format / TLVs (received via the host-capture recipe above): see the routeros-mndp skill
  • For /app YAML container format (requires CHR with container package): see the routeros-app-yaml skill
Files (routeros-skills)
  • references
    • chr-licensing.md 3 KB
      Loading…
    • github-actions-ci.md 9.9 KB
      Loading…
    • known-issues.md 3.9 KB
      Loading…
    • virtio-drivers.md 2.8 KB
      Loading…
  • SKILL.md 17.2 KB
    ---
    name: routeros-qemu-chr
    description: 'MikroTik RouterOS CHR (Cloud Hosted Router) with QEMU. Use when: running RouterOS in QEMU, booting CHR images, debugging CHR boot failures, setting up VirtIO devices for RouterOS, choosing between SeaBIOS and UEFI boot, configuring QEMU port forwarding for RouterOS REST API, setting up inter-VM socket networking or host-side L2 capture of guest broadcasts (e.g. MNDP), or selecting QEMU acceleration (KVM/HVF/TCG).'
    ---
    
    # RouterOS CHR with QEMU
    
    ## What Is CHR
    
    Cloud Hosted Router (CHR) is MikroTik's x86_64 and aarch64 RouterOS image designed for virtual machines. Free license allows unlimited use with 1 Mbps speed limit — sufficient for development, testing, API work, and packet sniffer debugging. A free 60-day trial removes the speed limit entirely (requires a free mikrotik.com account). See [CHR licensing reference](./references/chr-licensing.md) for full details on license tiers, trial activation, and expiry behavior.
    
    ## Image Variants
    
    | Image | Architecture | Boot method | Source |
    |---|---|---|---|
    | `chr-<ver>.img` | x86_64 | SeaBIOS (MBR chain-load) | download.mikrotik.com |
    | `chr-<ver>-arm64.img` | aarch64 | UEFI (EDK2 pflash) | download.mikrotik.com |
    | `chr-efi.img` (fat-chr) | x86_64 | UEFI (OVMF) | tikoci/fat-chr GitHub |
    
    **Standard x86 image has a proprietary boot partition** — it looks like an EFI System Partition in GPT but is NOT FAT. UEFI firmware (OVMF) cannot read it. Only SeaBIOS can boot it via MBR chain-load.
    
    The `fat-chr` repackaged image converts this to standard FAT16 with `EFI/BOOT/BOOTX64.EFI`, enabling UEFI boot. Required for Apple Virtualization.framework on X86 macOS, optional everywhere else.
    
    **Disk layout** (128 MiB, both architectures): Hybrid GPT+MBR, partition 1 = boot (~33 MiB), partition 2 = ext4 root (~94 MiB).
    
    ## Downloading CHR Images
    
    ```typescript
    // Resolve current version
    const channel = "stable"; // or: long-term, testing, development
    const version = await fetch(
      `https://upgrade.mikrotik.com/routeros/NEWESTa7.${channel}`
    ).then(r => r.text()).then(s => s.trim());
    
    // Download x86_64 image
    const url = `https://download.mikrotik.com/routeros/${version}/chr-${version}.img.zip`;
    // Download aarch64 image
    const armUrl = `https://download.mikrotik.com/routeros/${version}/chr-${version}-arm64.img.zip`;
    ```
    
    Images are distributed as `.img.zip` — unzip to get the raw `.img` disk file.
    
    ## Pattern Choices: QEMU Invocation
    
    There are several valid approaches to launching CHR under QEMU. Each has tradeoffs:
    
    ### Pattern A: Inline arguments (simplest, good for scripts)
    
    Everything on the command line. Easy for an LLM to construct and debug — all state is visible in one place.
    
    ```sh
    qemu-system-x86_64 -M q35 -m 256 -smp 1 \
      -drive file=chr.img,format=raw,if=virtio \
      -netdev user,id=net0,hostfwd=tcp::9180-:80 \
      -device virtio-net-pci,netdev=net0 \
      -display none -serial stdio
    ```
    
    **Pros:** Single command, easy to read, easy to modify.
    **Cons:** Long command lines, hard to version-control, no persistence.
    
    ### Pattern B: Wrapper script (good for reuse)
    
    A shell script that detects acceleration, handles firmware paths, manages PID files.
    
    ```sh
    #!/bin/sh
    # detect acceleration
    if [ "$(uname -s)" = "Linux" ] && [ -w /dev/kvm ]; then
      ACCEL="-accel kvm"
    elif [ "$(uname -s)" = "Darwin" ] && [ "$(sysctl -n kern.hv_support 2>/dev/null)" = "1" ]; then
      ACCEL="-accel hvf"
    else
      ACCEL="-accel tcg"
    fi
    
    qemu-system-x86_64 -M q35 -m 256 -smp 1 \
      $ACCEL \
      -drive file=chr.img,format=raw,if=virtio \
      -netdev user,id=net0,hostfwd=tcp::${PORT:-9180}-:80 \
      -device virtio-net-pci,netdev=net0 \
      -display none -serial stdio
    ```
    
    **Pros:** Portable, handles platform differences, parameterizable.
    **Cons:** Shell scripting limitations, harder to compose from TypeScript.
    
    ### Pattern C: Programmatic launch from Bun/TypeScript (good for integration tests)
    
    Launch QEMU as a child process with full control:
    
    ```typescript
    import { $ } from "bun";
    
    const port = 9180;
    const accel = await detectAccel();
    const proc = Bun.spawn([
      "qemu-system-x86_64", "-M", "q35", "-m", "256",
      "-accel", accel,
      "-drive", `file=chr.img,format=raw,if=virtio`,
      "-netdev", `user,id=net0,hostfwd=tcp::${port}-:80`,
      "-device", "virtio-net-pci,netdev=net0",
      "-display", "none",
      "-chardev", `socket,id=serial0,path=/tmp/chr-serial.sock,server=on,wait=off`,
      "-serial", "chardev:serial0",
      "-monitor", `unix:/tmp/chr-monitor.sock,server,nowait`,
    ], { stdio: ["ignore", "pipe", "pipe"] });
    
    // Wait for boot
    await waitForBoot(`http://127.0.0.1:${port}/`);
    ```
    
    **Pros:** Full lifecycle control, parallel instance management, TypeScript-native.
    **Cons:** More code, QEMU args still need to be correct.
    
    ### Pattern D: Config file (`--readconfig`) (declarative, used by mikropkl)
    
    QEMU's `--readconfig` loads an INI-format file for device/machine config. The mikropkl project uses this for its declarative VM packaging.
    
    **Tradeoffs:** Separates concerns (config vs launch), but the INI format is obscure and not all QEMU options can be expressed in it (pflash, `-accel`, `-netdev user,hostfwd` all require command-line args). Best suited for projects that generate configs programmatically.
    
    ## Boot Tracks
    
    ### x86_64 with SeaBIOS (default, fastest)
    
    No firmware setup needed — QEMU's built-in SeaBIOS handles MikroTik's proprietary boot sector:
    
    ```sh
    qemu-system-x86_64 -M q35 -m 256 \
      -drive file=chr-7.22.img,format=raw,if=virtio \
      -netdev user,id=net0,hostfwd=tcp::9180-:80 \
      -device virtio-net-pci,netdev=net0 \
      -display none -serial stdio
    ```
    
    `-M pc` (i440fx, legacy PCI) also works for CHR. CHR doesn't exercise most of what q35 adds (modern PCIe topology, ACPI-based hotplug, etc.) because the RouterOS kernel bypasses the BIOS/ACPI stack after boot — the VirtIO PCI devices are what it actually uses, and both machine types expose those identically. For the kernel-config evidence behind this, see [tikoci/mikrotik-gpl](https://github.com/tikoci/mikrotik-gpl) (v7.2 kernel config archive).
    
    Boot time: ~5s (KVM), ~30s (TCG). For TCG, `-accel tcg,tb-size=256` enlarges the translation block cache and reduces boot time noticeably on repeated runs.
    
    ### aarch64 with UEFI (EDK2)
    
    Requires UEFI pflash firmware files. **Both pflash units must be identical size** (typically 64 MiB):
    
    ```sh
    # Copy vars file (writable) — never modify the original
    cp /path/to/edk2-arm-vars.fd /tmp/my-vars.fd
    
    qemu-system-aarch64 -M virt -cpu cortex-a710 -m 256 \
      -drive if=pflash,format=raw,readonly=on,unit=0,file=/path/to/edk2-aarch64-code.fd \
      -drive if=pflash,format=raw,unit=1,file=/tmp/my-vars.fd \
      -drive file=chr-arm64.img,format=raw,if=none,id=drive0 \
      -device virtio-blk-pci,drive=drive0 \
      -netdev user,id=net0,hostfwd=tcp::9180-:80 \
      -device virtio-net-pci,netdev=net0 \
      -display none -serial stdio
    ```
    
    Boot time: ~10s (KVM), ~20s (TCG native), ~20s (TCG cross-arch on x86 host).
    
    ### UEFI Firmware Locations
    
    | Platform | Code ROM | Vars File |
    |---|---|---|
    | macOS Homebrew (Apple Silicon) | `/opt/homebrew/share/qemu/edk2-aarch64-code.fd` | `edk2-arm-vars.fd` |
    | macOS Homebrew (Intel) | `/usr/local/share/qemu/edk2-aarch64-code.fd` | `edk2-arm-vars.fd` |
    | Ubuntu/Debian | `/usr/share/AAVMF/AAVMF_CODE.fd` | `AAVMF_VARS.fd` |
    | x86 OVMF (Homebrew) | `edk2-x86_64-code.fd` | `edk2-i386-vars.fd` |
    | x86 OVMF (Linux) | `/usr/share/OVMF/OVMF_CODE.fd` | `OVMF_VARS.fd` |
    
    ## VirtIO — Critical Details
    
    See the [VirtIO driver matrix](./references/virtio-drivers.md) for the full table.
    
    **The one rule:** RouterOS has `virtio_pci` but NOT `virtio_mmio`. This matters on aarch64.
    
    ### The `if=virtio` Trap (aarch64)
    
    ```text
                             x86_64 (q35)              aarch64 (virt)
    if=virtio shorthand →    virtio-blk-pci (PCI) ✅    virtio-blk-device (MMIO) ❌
    -device virtio-blk-pci → virtio-blk-pci (PCI) ✅    virtio-blk-pci (PCI) ✅
    ```
    
    On x86_64 `q35`, `if=virtio` resolves to PCI — works fine. On aarch64 `virt`, it resolves to MMIO — **RouterOS kernel stalls silently**. Always use explicit `-device virtio-blk-pci` on aarch64:
    
    ```sh
    # WRONG on aarch64 — silent boot failure
    -drive file=chr.img,format=raw,if=virtio
    
    # CORRECT on aarch64 — explicit PCI device
    -drive file=chr.img,format=raw,if=none,id=drive0
    -device virtio-blk-pci,drive=drive0
    ```
    
    On x86_64, both work. The explicit form is always safe on both architectures.
    
    ### Network — Universal
    
    All architectures: `virtio-net-pci`. No exceptions:
    
    ```sh
    -netdev user,id=net0,hostfwd=tcp::9180-:80
    -device virtio-net-pci,netdev=net0
    ```
    
    `user` (SLIRP) is a userspace NAT — ideal for management (hostfwd of REST/SSH/WinBox),
    but it **terminates Layer 2**: broadcasts and non-forwarded UDP never reach the host or
    other VMs. For anything L2 (inter-VM links, neighbor discovery, MAC-Telnet) add a second
    NIC on a `socket` netdev.
    
    ### L2 Networking: `socket` netdevs (inter-VM links + host capture)
    
    QEMU's `socket` netdev carries raw Ethernet frames over a host socket — rootless, no extra
    setup:
    
    ```sh
    # TCP, point-to-point — one side listens, the other connects
    -netdev socket,id=net1,listen=:4001            # VM A (start first)
    -netdev socket,id=net1,connect=127.0.0.1:4001  # VM B
    # UDP multicast — shared L2 segment for N VMs
    -netdev socket,id=net1,mcast=230.0.0.1:4001
    ```
    
    > **⚠ `mcast` is broken on macOS.** QEMU's mcast socket sets only `SO_REUSEADDR`, but
    > macOS/BSD require `SO_REUSEPORT` on every socket sharing a multicast port — so on macOS
    > nothing is delivered between local sockets: two VMs on the same group don't discover each
    > other, and a host listener gets zero frames. It works on **Linux** (and CI). For
    > point-to-point on macOS use TCP `listen`/`connect`; there is **no rootless multi-VM L2
    > segment on macOS** today (use `socket_vmnet`, privileged-once, for that).
    
    #### Host capture of guest L2 frames (e.g. MNDP)
    
    To let a **host process** receive a guest's broadcasts — MNDP neighbor discovery (the
    protocol WinBox's "Neighbors" tab uses, UDP/5678) being the first case — make the host one
    end of a TCP `socket` netdev:
    
    1. Host runs a TCP **server** on a loopback port.
    2. CHR gets a second NIC: `-netdev socket,id=net1,connect=127.0.0.1:<port>` — the host must
       be listening **before** QEMU starts, since QEMU is the connecting side.
    3. QEMU streams every guest frame to the host, **length-prefixed**: a 4-byte big-endian
       length, then the raw Ethernet frame.
    4. Host strips the prefix and parses Ethernet → IPv4 → UDP/5678 → MNDP TLVs.
    
    Writing a length-prefixed frame back over the same connection **injects** L2 into the guest
    (e.g. an MNDP request, which draws a broadcast announcement after a short random delay —
    see `routeros-mndp` for the 8-byte payload; the same primitive MAC-Telnet needs).
    Loopback-only, so nothing leaks onto the LAN; works on macOS, Linux, and Windows alike.
    
    Reference implementation + verified evidence: `tikoci/quickchr` (`examples/mndp/`,
    `docs/mndp.md`, `test/lab/mndp/REPORT.md`). For the MNDP wire format and TLV table, see the
    `routeros-mndp` skill.
    
    ## Acceleration Detection
    
    ```typescript
    import { $ } from "bun";
    
    async function detectAccel(guestArch: string): Promise<string> {
      const hostOs = process.platform;  // "darwin" | "linux"
      const hostArch = process.arch;    // "x64" | "arm64"
    
      if (hostOs === "linux") {
        // KVM requires host/guest architecture match
        const kvm = await Bun.file("/dev/kvm").exists();
        const archMatch = (guestArch === "x86_64" && hostArch === "x64")
          || (guestArch === "aarch64" && hostArch === "arm64");
        if (kvm && archMatch) return "kvm";
      }
    
      if (hostOs === "darwin") {
        // HVF may not be available (e.g., GitHub Actions VMs)
        const hvOk = await $`sysctl -n kern.hv_support`.text().then(s => s.trim() === "1").catch(() => false);
        const archMatch = (guestArch === "aarch64" && hostArch === "arm64")
          || (guestArch === "x86_64" && hostArch === "x64");
        if (hvOk && archMatch) return "hvf";
      }
    
      return "tcg";  // Software emulation — always available
    }
    ```
    
    **Key rule:** KVM and HVF both require host/guest architecture match. Cross-arch always falls back to TCG. Don't check just for `/dev/kvm` — verify the architecture matches too.
    
    ### HVF + CPU Model Gotcha (macOS)
    
    With `-accel hvf`, QEMU exposes the host CPU directly. Specifying a CPU model like `cortex-a710` (ARMv9, requires SVE2) on Apple Silicon (ARMv8.5) crashes QEMU before the VM starts. Use `-cpu host` with HVF:
    
    ```sh
    # TCG/KVM — specify exact model
    CPU_FLAGS="-cpu cortex-a710"
    
    # HVF — passthrough host CPU
    if [ "$ACCEL" = "hvf" ]; then
      CPU_FLAGS="-cpu host"
    fi
    ```
    
    ## Health Check and Boot Wait
    
    RouterOS WebFig responds with HTTP 200 on port 80 without authentication — this works as a minimal health check:
    
    ```typescript
    async function waitForBoot(url: string, timeoutMs = 60_000): Promise<boolean> {
      const deadline = Date.now() + timeoutMs;
      while (Date.now() < deadline) {
        try {
          const r = await fetch(url, { signal: AbortSignal.timeout(2000) });
          if (r.ok) return true;
        } catch { /* not ready yet */ }
        await Bun.sleep(2000);
      }
      return false;
    }
    ```
    
    ### Startup race — pitfall if you then call the REST API immediately
    
    RouterOS boot is staged from the client's perspective:
    
    1. Connection refused — QEMU still booting
    2. `ECONNRESET` — HTTP server up but REST subsystem not accepting
    3. 401 from `/rest/*` — auth middleware up; REST handler may still be initializing
    4. 200 but **wrong body** — REST initialized before routing tables settled; `/system/resource` can briefly return an array (e.g., a `/user` list) before it returns the expected singleton object
    5. 200 with correct body — fully operational
    
    A health check on the root `/` reaches stage 2 but gives no signal about 3–5. For code that will immediately call the REST API, probe `/rest/system/resource` with auth and require **two consecutive successful probes** with the expected body shape (singleton object containing `board-name`). On auth-only checks (unprovisioned admin), accept 401/403 as "REST layer responded." The `/` + HTTP 200 check is fine for "is it up?" monitoring but not for "can I start calling REST now?"
    
    ## Port Forwarding
    
    QEMU user-mode networking (`-netdev user,hostfwd=...`) for typical RouterOS services:
    
    | Service | Guest Port | Example Host Port | hostfwd |
    |---|---|---|---|
    | WebFig/REST API | 80 | 9180 | `tcp::9180-:80` |
    | SSH (RouterOS CLI) | 22 | 9122 | `tcp::9122-:22` |
    | API protocol | 8728 | 9728 | `tcp::9728-:8728` |
    | API-SSL | 8729 | 9729 | `tcp::9729-:8729` |
    | WinBox | 8291 | 9291 | `tcp::9291-:8291` |
    
    Multiple forwards in one netdev:
    
    ```sh
    -netdev user,id=net0,hostfwd=tcp::9180-:80,hostfwd=tcp::9122-:22,hostfwd=tcp::9728-:8728
    ```
    
    Use unique host ports per instance when running multiple CHRs (9180, 9181, 9182...).
    
    ## Known Limitations
    
    - **QGA (Guest Agent) requires KVM** — RouterOS CHR's QGA daemon only starts when it
      detects a KVM hypervisor via CPUID. Under HVF (macOS) or TCG (software emulation),
      CPUID 0x40000000 returns no KVM vendor string and 0x40000001 returns no KVM features,
      so the daemon never starts. QEMU correctly provides the virtio-serial port and sends
      PORT_OPEN (event 6) — the guest simply never opens it (`query-chardev` shows
      `frontend-open=false`). This is NOT a QEMU bug. MikroTik documents QGA exclusively
      under the "KVM" section. QGA testing requires Linux + KVM (e.g., mikropkl lab).
    - **`check-installation` fails on aarch64** in all QEMU environments — this is an unresolvable firmware/DTB issue (see [known issues](./references/known-issues.md))
    - **Direct `-kernel` boot does not work** for either architecture — RouterOS needs its full firmware boot path
    - **Cross-arch TCG: x86_64 on aarch64 host is not viable** — x86 I/O port emulation is too slow (~300s+ timeouts). The reverse (aarch64 on x86_64) works fine (~20s)
    - **No `virtio_mmio` driver** — always use explicit `-device virtio-blk-pci`, never rely on `if=virtio` on aarch64
    - **`socket,mcast=` inter-VM networking is broken on macOS** — QEMU sets only
      `SO_REUSEADDR`; macOS/BSD need `SO_REUSEPORT` to share a multicast port, so no frames are
      delivered between local sockets (VMs don't discover each other; host capture gets nothing).
      Use TCP `socket` (listen/connect) for point-to-point, or `socket_vmnet` for a shared
      segment. Works on Linux/CI. Verified in `tikoci/quickchr` (`test/lab/mndp/REPORT.md`)
    
    ## Additional Resources
    
    - [VirtIO driver matrix](./references/virtio-drivers.md) — full driver support table
    - [Known issues](./references/known-issues.md) — boot failures, cross-arch limitations
    - [GitHub Actions CI patterns](./references/github-actions-ci.md) — running CHR on GitHub-hosted runners
    - [CHR licensing](./references/chr-licensing.md) — free tier (1 Mbps), 60-day trial, paid tiers, expiry behavior
    - For the **quickchr** reference implementation — driving CHR from tests &
      automation (`QuickCHR.start`, `exec`/`rest`, networking recipes, the connection
      surface for harnesses): see the `routeros-tikoci-quickchr` skill
    - For RouterOS CLI/REST once booted: see the `routeros-fundamentals` skill
    - For packet capture and TZSP streaming from CHR: see the `routeros-sniffer` skill
    - For MNDP neighbor-discovery wire format / TLVs (received via the host-capture recipe above): see the `routeros-mndp` skill
    - For /app YAML container format (requires CHR with container package): see the `routeros-app-yaml` skill
    

Comments (0)

Sign in to join the conversation.

No comments yet.

Reviews (0)

No reviews yet.