Dev machines and the workspace syncer

Any lab machine, VM or container, Windows or Linux, can be the lab's development environment. Mark it @dev and give it a workspace, and vmlab keeps a host directory and a guest working copy of it in step: you edit on the host with your own tools, and the build, the tests and the debugger run guest-side, against real guest paths, the real toolchain and, where the lab has one, the real domain identity. **A dev machine is a machine with a synced workspace.** This chapter explains the declaration, what the guest must have, how a dev verb picks its machine, and the workspace syncer. The verbs are in vmlab dev.

The @dev declaration

@dev is a decorator on a vm or container block, not a child block, because it states something *about* the machine rather than configuring something inside it; nothing it carries is a setting the guest sees. Any number of machines may carry it and zero is normal. Its arguments are all optional, and a bare @dev is a complete dev machine.

@dev(default = true, workspace = "./src", workspace_guest = "C:\\src")
vm "dev01" {
  template   = "x86_64/windows-server-2025"
  depends_on = ["dc01"]
  nic { segment = "corp" }
  login "dev" { user = "PROBE\\dev" password = "vmlab123!" }
}

@dev(workspace = "./src")
container "buildbox" {
  image = "mcr.microsoft.com/dotnet/sdk:9.0"
  cpus = 4  memory = 4GiB
  nic { segment = "corp" }
}

Unset arguments resolve @dev argument, then profile, then floor. The default is profile-sourced because it is guest-OS-shaped: the shipped Windows profiles say C:\src and the Linux ones /src. A profile with no dev keys still hosts a dev machine; a missing key means the floor applies, never that the profile cannot be a dev target. The guest path does not have to exist: when the default login cannot create it, as with /src under a root-owned / on Linux, vmlab creates it as the agent identity and gives it to that login before the first sync. This resolver is deliberately separate from the hardware resolver. "First in file order wins" was rejected for the default because declaration order already means something in vmlab and a block reorder would silently move it.

Two things were kept off the decorator on purpose. Ports: a dev machine's ports are ordinary port {} and forward {} declarations. Toolchain and package lists: that is provision and playbook. A distributable template is vmlab's answer to devcontainer features, installed once at build time and pulled by every developer.

What the guest needs

The guest needs two things: the agent and the toolchain. The workspace path is created by the syncer. The agent advertises its feature strings in its handshake, and the syncer needs two of them: fileops, the file session every transfer runs over, and watch, the recursive tree watch that tells the host what changed in the guest. The syncer checks watch && fileops for itself when it starts, and a machine whose agent lacks either runs with no syncer and says so. A template built with agent = false reports the features missing through the same path as one whose agent is merely old. `vmlab machine capabilities` reports what the agent serves.

An old agent is refreshed by vmlab up itself: when a VM's agent stamp differs from the agent this vmlab ships, up pushes the shipped agent over the agent's own channel before any provision runs, and marks the machine diverged, because the template's sealed agent_version no longer describes it. vmlab machine repair-agent does the same push on demand, for a machine already running. Rebuilding the template is the route that leaves the machine matching its template, and agent_update = false keeps up from touching the agent. The push needs an agent that serves fileops; an older one can only be rebuilt. It is meaningless on a container, whose agent lives in the initramfs vmlab ships and tracks the installed vmlab; the verb says so. Two further preconditions of a dev-capable image: the guest must be symlink-capable, on Windows through SeCreateSymbolicLinkPrivilege or Developer Mode, and a full Linux VM's kernel must be recent enough that inotify survives an overlayfs copy-up. See vmlab machine.

Which dev machine a verb acts on

A lab may declare several dev machines, and a dev verb given no machine climbs a fixed ladder and never guesses:

1. an explicit argument,
2. the VMLAB_DEV_MACHINE environment variable,
3. the lab's default dev machine (@dev(default = true), or the lone @dev),
4. otherwise an error listing the candidates.

Every rung that names a machine is checked rather than trusted: a rung naming something that is not a dev machine in this lab is an error at that rung, never a silent fall-through, so an environment variable left over from another lab cannot land you somewhere nothing said out loud. There is no dev list or dev status; vmlab status shows every machine.

The workspace

**The workspace is a guest-local working copy on the machine's own disk; the host directory is canonical; a vmlab-integrated syncer keeps them in step.** Neither obvious alternative works. A shared folder cannot carry a watched source tree: host-side edits do not reach ReadDirectoryChangesW over virtiofs or SMB, nor inotify over virtiofs on Linux, and both fail silently, with the watcher armed and quiet and the language server simply no longer re-analysing. Source kept guest-side alone fails the other way: destroy is a first-class verb on disposable clones and a snapshot restore would roll uncommitted work back with the machine. With the host canonical, destroy loses nothing and a restore re-converges the guest from the host. share {} stays exactly as useful as it was for datasets, installers and build output; the line is a watched source tree.

The syncer is a task in the lab daemon, started by up **after provisioning** rather than at machine-ready, and it runs as the machine's default login: the one exception to vmlab's own machinery keeping the agent identity, because it produces the developer's files, and the account it writes as does not exist until provisioning creates it. Ownership always matches whoever works in the guest. One pass walks the host, learns what the guest holds, reconciles, applies and saves the ledger. The seed is simply the first pass.

Edit on either side

Take a Linux dev machine, dev01, declared with `@dev(workspace = "./workspace")` and up. Edit on the host and the change lands in the guest; write in the guest and the output lands on the host. vmlab exec runs one command in the guest, and vmlab shell dev01 opens an interactive shell for longer work.

printf 'print("edited on the host")\n' > workspace/hello.lua
vmlab exec dev01 -- cat /src/hello.lua
vmlab exec dev01 -- sh -c 'echo built > /src/out.txt'
cat workspace/out.txt

The host edit reaches /src/hello.lua, and the file written in the guest appears in ./workspace/. vmlab dev sync status reports what the syncer last decided: paths it holds a halt on, warnings such as a subtree large enough to be worth naming, and the special files it skipped by name. `vmlab dev sync flush` runs a pass now and waits for it rather than for the next edit. The rest of this chapter explains what those passes do.

The ledger and what a conflict is

The agreement point is a host-side sync ledger under the lab's .vmlab/, one per (machine, workspace), so destroy wipes it. It holds one record per relative path: a content digest plus each side's own size and mtime as a change detector. A host mtime is never compared to a guest mtime, only to the host's own recorded value, because a restored guest resumes with a clock behind the host and every file it holds would look older; that alone rules out newest-wins. Digest is the truth and the stat pair is only a pre-filter, since a same-size in-place write is exactly what the share transports were caught missing. A missing ledger is not a decision: on first run, or after a wiped .vmlab/ with a live guest, paths whose digests match are adopted as agreed and paths that differ take the ordinary conflict path, rather than a blind host-to-guest seed eating a developer's work.

Per path, each side is unchanged, modified, deleted or replaced by the other kind relative to the ledger. One side changed: propagate. Both changed: conflict, with four riders. Both modified to identical content is not a conflict and transfers nothing, which is common after a host-side `git checkout` lands bytes the guest already had. Modified on one side and deleted on the other is a conflict, not delete-wins, because deletion is unrecoverable. Mode-only changes are not conflicts and are not synced across kinds. A file replaced by a directory, or the reverse, is a conflict.

Ignore rules and the prune list

Ignore rules live in the tree, not in the lab file: a built-in floor, then the repo's .gitignore, then .vmlabignore for the delta including ! negations. What you do not want to sync is almost exactly what you do not commit, and .vmlabignore covers where "almost" fails: a gitignored .env, a local cert or appsettings.Development.json that the app needs guest-side takes a negation. Precedence is git's own, deepest file first, .vmlabignore beating .gitignore in one directory, and a path under an ignored directory staying ignored. The floor covers the syncer's own scratch names and .git/**/*.lock, and no repo rule may override it.

An ignored path is not skipped, it is guest-owned: node_modules is the proving case, where the guest runs its own install and holds guest-native binaries, diverging permanently and on purpose. Neither direction ever touches one. The host computes a coarse prune list, ignored directory prefixes with no negation below them, and hands it to the agent when the watch opens, so no watcher is registered under them. That matters because inotify costs one watch descriptor per directory against a default limit of 8192, so an unpruned registration would be silently incomplete on Linux. The guest is never asked to decide what is in the synced set; it is handed a list.

When the rules change under the syncer, leaving scope is free: a newly guest-owned path leaves the ledger and both copies stay. Entering scope is a conflict, because no agreement exists and both sides may hold content, so un-ignoring a populated directory halts naming every file in it. The rules' own digest is part of the ledger, so the halt can say these conflict because you just changed the rules.

Both directions

Host changes arrive from vmlab's own watcher. Guest changes arrive from the agent's dirty set: the agent holds a coalescing set of paths that changed, sends a single nudge when the set goes from empty to non-empty, and the host drains it. A drained record is the path plus its current stat, or a tombstone if the path is gone, so no platform event kind ever crosses the seam and inotify and ReadDirectoryChangesW disagreeing about renames never becomes a vocabulary problem. Both directions get the same per-path debounce, a quiet period before a path is read, because editors write a temp and rename and compilers write in chunks, and a torn read guest-to-host would land on the canonical copy. A path that keeps moving keeps waiting, so a burst under one subtree de-prioritises rather than starving a single save elsewhere.

Every apply is temp-name-then-rename in the target's own directory, and the ledger records agreement only after the rename, never after the last write. Renames are delete plus create at the ledger level; a directory delete expands via the ledger, not the event stream. Symlinks sync verbatim and are never followed; their target string is content, untranslated across the seam. Special files, FIFOs, sockets, device nodes and non-symlink reparse points, are skipped loudly and never enter the ledger, and so is a drained path the login cannot read, because a build leaving a root-owned artefact in the tree must not stop the dev machine.

The stat-walk

The steady state never walks the tree; it probes named paths. A full guest stat-walk, where the guest reports every path's kind, size and mtime and the host applies the ignore set on receipt and asks for digests only for suspects, runs on a watch discontinuity and nowhere else: first sync, ledger loss, an overflow, a dropped channel. That list is exactly the list of watch discontinuities, which is why there is no resync token. An overflow, on either platform or in the agent's own capped set, collapses to a single rescan value that warns, forces the walk and never halts. **The rescan is a barrier in both directions**: between the overflow and the completed walk the host does not know the guest moved, so propagating host-to-guest meanwhile would overwrite guest work silently through the ledger with no conflict raised. It is a deferral, needing no developer action, and it clears itself.

Windows preconditions

A Windows guest costs the syncer three actions, each a precondition of the mechanism, so vmlab does them rather than documenting them:

1. The NTFS case-sensitive flag on every directory the syncer creates, at creation, including the workspace root. The host can hold Foo.cs and foo.cs; a default Windows guest cannot, and the second write would silently land on the first. The flag only takes on an empty directory, which the syncer's always are, and inheritance is not relied on. Where it cannot be set, a case collision at that path is a loud refusal by name. This also makes a shared .git/config with one core.ignorecase right on both sides.
2. Symlinks attempted, with a warning by name on failure. A symlink-capable image is a documented precondition.
3. core.autocrlf = false in the guest's global git config, set as the default login. Git for Windows ships it true, which would rewrite the working tree to CRLF on the first guest-side checkout, sync every file back as modified, and halt the workspace if the host had touched anything. The syncer translates nothing; bytes cross verbatim and git normalises on both sides from settings that agree.

A Windows dev login declared elevated = false degrades the workspace in exactly two ways, no case-sensitive directories and no symlinks, and the syncer says so up front rather than at a random path hours in. Line-ending policy belongs in each side's global git config, never the repo's, because the home directory is guest-local and .git/config is shared; .gitattributes is the escape for genuine CRLF needs.

The conflict halt

The developer authors guest-side; the host is doing durability work, not authorship. The host-side writer set is small: git operations, occasional tooling, vmlab's own restore. A conflict is therefore an anomaly, which licenses an expensive, loud, safe policy over a winner rule that must be right thousands of times a day. That policy is halt and surface: the whole workspace stops, both directions, on one machine, naming every conflicting path in the batch. A pass scans and reconciles before it applies anything, so the halt is computed from a whole reconciliation, and a host-side git pull that collides in batches is one halt rather than thirty resolve-and-resume round trips. Ten conflicts do not become a bigger hammer.

While halted, the watch keeps running and the host keeps draining the guest's dirty set into its own pending set, so a long halt costs no rescan and edits made during it drain normally on resume. No conflict copy is written: the two copies already exist, one per side, and a halt writes neither and deletes neither. The scope is one machine's workspace; two dev machines may share one host workspace because the host is a hub rather than a peer, each with its own ledger, and one halting must not stop the other. The halt message names the machine.

From inside the guest a halt is otherwise nothing happening, and there is no guest-to-host control path to say otherwise, so the halt writes a marker file, .vmlab-sync-halt, at the guest's workspace root, listing up to 200 of the halted paths and saying how many it left out. It is in the ignore floor so it never syncs, and its git status noise is the point: it is the developer noticing. Resolution is host-side, necessarily, and the routes are:

A free third route needs no verb: make the two sides identical by hand and the next pass adopts them as agreed. These verbs live under vmlab dev because a workspace exists only for a dev machine, and a --machine flag or the selection ladder picks which. See vmlab dev.

Guards that are not halts

The size guard refuses loudly, per file, before transfer. A file over the workspace_max_file cap in the host configuration, 256 MiB by default, is never hashed or sent; the refusal names the file and the cap and states the two ways out, an ignore rule or a raised cap. It exists to catch the 4 GB .vhdx nobody wrote a rule for. Volume warns and never halts: a pass moving over a thousand paths or 256 MiB, dominated by one subtree, names that subtree and suggests a .vmlabignore rule, because a build burst into an un-ignored target/ is wanted work that happens to be large.

The guest-to-host bulk-delete guard is asymmetric on purpose: host-to-guest deletes are unguarded, and a git checkout removing 400 guest files just removes them, but guest-to-host deletions past a threshold are withheld and halt. The threshold is a proportion with a floor: more than 20 paths and more than half of what the ledger had agreed on. A fixed count would punish large repos and a bare proportion would let a ten-file project lose everything. A single deletion still propagates immediately. --guest on the halt is what releases the withheld removals.

.git syncs bidirectionally, because the guest can stay offline while the host has the network, so a host-side git fetch is a first-class operation, and because a coding agent inside the dev machine commits and branches with no host shell. Most of .git is content-addressed or write-once and syncs freely. The mutable set, index, HEAD, ORIG_HEAD, FETCH_HEAD, packed-refs, config, and everything under refs/ and logs/, is deferred while a *.lock is held on either side. Lock files themselves never sync. That deferral is timing, not a conflict rule: nothing is reported, nothing needs resolving, and the loop looks again after a second so a lock released in the guest does not stall until the next unrelated edit. Running git on both sides at once can still reach an ordinary halt, where both copies survive.

Snapshots bracket the syncer

A restore rewinds the guest by hundreds of files at once, which a naive syncer cannot tell from the developer having edited them and would push onto the canonical copy. vmlab performs the restore, so it brackets it. Capture first flushes, and refuses with no escape flag if the guest holds work the canonical copy has never seen, whether a halt stands or the pass could not finish. Restore refuses while a halt stands, and --discard-guest-changes is the one escape: it throws the guest copy of every halted path away, by name. Restore then takes the syncer off the workspace, rewinds, and puts it back owing a re-seed: a host-only, digest-based reconcile that overwrites anything differing from host truth, deletes anything the ledger does not hold, transfers nothing else, and emits no guest-to-host action at all. It compares by digest because a restored guest's clock runs behind the host. It completes before the watch reopens, and it replaces the stat-walk rather than following one. Both the owed re-seed and the halt a restore refuses on ride the ledger, because a restore does not need a running machine. Every surface that takes or restores one says the same sentence, and the Snapshots chapter repeats it.

Rebuilds and what survives them

There is no rebuild verb because two existing verbs are one. Destroying the machine takes its clone and the guest working copy; the next up provisions a fresh one and the syncer seeds it again from the host.

vmlab container destroy dev01
vmlab up dev01

Everything the provision declared is back, and the guest workspace holds the host directory's current contents. Anything installed by hand in the guest is gone, and so is anything else in the login's home, which lives outside the workspace: it survives reboot, down/up and restore to a later snapshot, and dies on destroy plus up. That is the durability rule: bake what every developer needs into the image or a provision, hand-install what you want today, and expect to redo the latter after a rebuild. The worked example examples/dev-container places files into the dev login's home from a provision {} using as_login, before that user has ever logged on; see Example labs.