Simulator Overview (`netdoc-sim`)

netdoc-sim builds a throwaway virtual network from a scenario, breaks it on purpose, runs the real netdoc binary inside it, and reports whether the diagnosis matched the injected fault.

It is development and regression-testing infrastructure that happens to be fun. Challenge Mode is the part meant to be played with; everything else here is meant to be scripted.

The authoritative guide is docs/simulation.md in the repository. It lives beside the code because most of it (scenario schema, fault semantics, evidence fields, the challenge contract) has to change in the same commit as the implementation. This page explains the shape of the thing so that document is easier to read.

Why it exists

netdoc has to distinguish DNS, routing, transport, proxy, TLS, and service failures. Testing that against the real internet is hopeless: you cannot break the internet on demand, and you cannot know for certain what was broken.

A simulation makes the fault known by construction, so netdoc’s machine-readable diagnosis can be graded instead of eyeballed. No model, no heuristic, and no language model decides whether it was right; the comparison is between a structured expectation and a structured report.

Getting it

On Linux, every package installs netdoc-sim beside netdoc, from the same release and at the same version. Nothing else to fetch. See the README’s Linux install section.

On macOS and Windows, the binary is not in the download, and this is the one thing about the simulator people most often misunderstand:

The backend is Linux namespaces, and there is no other one. macOS and Windows do not get a port and do not get an emulation. They run the same Linux binary inside a Linux container, on the Linux kernel their container runtime already provides.

macOS / Windows / Linux host
  └── Docker / Podman                 Linux kernel, container runtime's own
        └── netdoc-sim                the released binary, unmodified
              └── user + net + mount namespaces   the same backend as native
                    └── netdoc        /usr/bin/netdoc from this same image

The portability boundary is the runtime, not the simulator. Anything that runs a Linux container will do. No clone, no Go toolchain, no knowledge of namespaces.

The exact image name, tags, run flags, capability requirements, and the two hosts that need extra policy flags are documented in docs/simulation.md, and they are tested against the real image in container_test.go, which is why they are not restated here.

Contributors build both binaries from the clone, so a run grades the netdoc they just changed rather than the installed one:

CGO_ENABLED=0 go build -o netdoc .
CGO_ENABLED=0 go build -o netdoc-sim ./cmd/netdoc-sim
./netdoc-sim run broken-dns

Which `netdoc` gets run

Worth knowing before you trust a result, because a run that quietly measured a different build is worse than a run that did not happen. The binary is resolved once, in the launcher, where $PATH and the working directory still mean what you meant by them:

-netdoc when given: a path names that file, a bare name is looked up on $PATH. An explicit binary that does not exist or cannot execute is an error; nothing falls back.
A netdoc sitting next to the netdoc-sim binary. This is what makes ./netdoc-sim run healthy use the ./netdoc you just built beside it.
A netdoc on $PATH.

Two traps: go run ./cmd/netdoc-sim puts the binary in a build cache, so step 2 finds nothing, so build it, or pass -netdoc. And build with CGO_ENABLED=0, or a cgo build may resolve through the host’s system resolver instead of the node’s private /etc/resolv.conf, testing the host rather than the simulation.

Every result records the absolute path and the version string that binary printed, so a result names the build that produced it.

Requirements and safety

Linux, unprivileged user namespaces, and the ip, nsenter, nft, and tc tools. Seeded netem loss or jitter additionally needs iproute2 6.6 or newer.

Do not guess, ask the host:

netdoc-sim capabilities

That reports this machine’s support and the privileged operations a run would perform.

A run needs no root, sudo, or setuid helper. The launcher re-executes a director inside a new user, network, and mount namespace, with your uid mapped to root only there. Inside its own namespaces the director can create bridges, veth pairs, routes, nftables rules, qdiscs, and low-port listeners; on the host side the kernel gives it nothing.

Nothing on your real network is touched. No host routes, no host resolver, no host firewall, no host trust store. Nothing is registered under /run/netns or /etc/netns. Node processes carry PDEATHSIG=SIGKILL, so when the owning process exits the kernel reclaims their namespaces and network objects.

Getting oriented

Four commands, in the order they are usually wanted:

netdoc-sim capabilities     # can this host simulate, and what would a run do
netdoc-sim scenarios        # the complete built-in scenario list
netdoc-sim validate <name>  # parse and check without building anything
netdoc-sim run <name>       # build it, run netdoc in it, print the report

netdoc-sim scenarios is the source of truth for the built-in inventory: neither this wiki nor the README keeps a copy, because a hand-maintained list is a list that goes stale. A scenario argument may also be a path to a YAML file.

For debugging the backend itself:

netdoc-sim run <name> -dry-run  # print every privileged command, execute none
netdoc-sim run <name> -v        # log commands as they run
netdoc-sim run <name> -keep     # hold the network open for inspection
netdoc-sim list                 # kept simulations
netdoc-sim inspect <id>         # a kept simulation's nodes, and how to enter them
netdoc-sim cleanup <id>         # release it (or -all)

The generated ip, nft, tc, and nsenter commands -dry-run prints are safe because the director executes them after isolation. Do not copy them into a host shell as a substitute for running the simulator.

netdoc-sim help prints every command and flag with its default. That output is tested for exact agreement with the real command, the man page, and all six shell completion files, so it is never out of date, and this page does not repeat it.

How a run is built

launcher                         host namespaces, no privileges
  └── director                   new user + network + mount namespace
        ├── bridge × segment
        ├── node holder × node   private network + mount namespace
        │     └── test services
        └── nsenter … netdoc     unmodified binary under test

Each logical segment is one bridge. A node interface is a veth peer attached to that bridge; a router is just an ordinary node attached to multiple segments. Each node gets a private generated /etc/resolv.conf.

netdoc runs unmodified in the selected client node. The simulator does not reimplement probes or verdict logic, which would defeat the purpose.

What a scenario is

A YAML file with four operational parts:

Part	What it declares
`topology`	Logical segments, nodes, interfaces, routes, resolvers, aliases, and bounded test services
`faults`	Impairments applied before or during the diagnostic
`tests`	One or more real `netdoc` invocations in a client node
`expect`	The verdict and probe results that should follow

The schema accepts logical intent, and trusted code derives the operating system arguments. Authors write {type: drop, node: resolver, direction: inbound, protocol: udp, port: 53}, not an nft expression. They cannot supply kernel interface names, commands, executable paths, arbitrary proxy URLs or environment variables, certificate keys or paths, qdisc handles, or raw firewall expressions. Addresses are parsed and rendered canonically with net/netip, and routes accept a prefix or default, never free-form ip syntax.

Expectations match netdoc’s stable machine-readable contract: probe ID, PASS/WARN/FAIL/SKIP/N/A, verdict, and optional structured cause or address-family state. They never match English diagnosis text. That is what lets the prose be improved without breaking the test suite.

Writing one

The scenario files themselves are the authoritative examples, and the guide names which one to start from for each feature: single-segment, routed, dual-stack, multipath, proxies, TLS, timed faults, campaigns. Start there rather than from a blank file:

internal/simulation/scenarios/ lists every built-in, embedded into the binary at build time.
Scenario authoring in docs/simulation.md covers the fault and service semantics that are easy to get wrong even after reading a scenario, and the step-by-step workflow for adding one.

Two semantics genuinely worth reading before your first scenario, because getting them backwards produces a scenario that tests nothing:

A drop with direction: outbound behaves like a local firewall and can return an immediate refusal. direction: inbound silently discards on arrival, so the sender waits for its timeout. Black-holed remote resolvers and services need inbound.
A pmtu_blackhole needs both halves: a narrowed router interface and dropped ICMP fragmentation-needed replies. Narrowing a hop that still reports the smaller MTU gets discovered and worked around; narrowing an endpoint makes the local kernel refuse the send instead of losing the packet silently. It is rejected on a node that is not a router, and both endpoints must keep the default MTU.

The `healthy` canary

The simulation has no internet. Scenarios claim netdoc’s compiled-in public addresses as node aliases and serve its fixed probe names from simulator DNS.

So if you change a fixed probe endpoint in internal/diagnostic, the healthy scenario will fail, with a false_positive suggestion naming the stale probe. That failure is intentional and is the mechanism that stops endpoint drift from going unnoticed. Update the affected scenario aliases, DNS records, and expectations, and rerun netdoc-sim run healthy.

Do not make the control tolerant. A second manually maintained endpoint table would drift for exactly the same reason.

Beyond a single run

Command	What it does	Page
`campaign`	Resolves bounded ranges in a scenario and runs seeded iterations sequentially, for reproducible fault campaigns	`docs/simulation.md`
`hunt`	Mutates a known-good control scenario to generate faults and rank likely bugs	Hunts and Triage
`triage`	Hunts fixed baselines, reproduces findings, optionally files issues	Hunts and Triage
`challenge`	The hunt with a human as the second contestant	Challenge Mode

All of them run through the same simulation lifecycle. There is no second backend, no second scoring path, and no second fault model anywhere in that list.

Maintenance scope

Worth stating, because it shapes what pull requests get accepted:

The simulator, including campaigns, hunts, and triage, is maintained for bug, safety, correctness, determinism, compatibility, and regression work. A fault model or scenario is added only for a real bug, a diagnostic blind spot, a reproducible field condition, a regression, or an identified missing network behaviour relevant to Network Doctor.

General simulator expansion is not a project goal. It is infrastructure for testing netdoc, not a network emulator competing with the real ones.

Known limitations

Linux is the only maintained backend; the container image is packaging around it, not a second one.
Topology is static unicast IPv4/IPv6 over simulator-owned bridges, with no NAT, address autoconfiguration, dynamic routing, tunnels, ECMP, or VLAN model.
Services are deliberately narrow probe fixtures, not general DNS, HTTP, proxy, TLS, QUIC, encrypted-DNS, or TCP implementations.
Timed faults reproduce requested content and ordering, not hard real-time application.
Campaigns are sequential fault-injection runs, not performance or statistical-significance tooling.

Where next

Challenge Mode: play against netdoc.
Hunts and Triage: automated fault generation.
Architecture: the simulator pipeline in context.

This page has one editable copy: edit it at the source.

Simulator Overview (netdoc-sim)