11. Testing

// tcm: namespace lib
import { expect, vi } from "vitest";

export const testSortIsStable = () => {
  expect(sortWith(byKey, [/* … */])).toEqual([/* … */]);
};

export const testRetriesOnce = async () => {
  const fetch = vi.fn<() => Promise<number>>().mockRejectedValueOnce(new Error("down")).mockResolvedValue(1);
  expect(await withRetry(fetch)).toBe(1);
  expect(fetch).toHaveBeenCalledTimes(2);
};

A test is a definition named test…, or any definition under a tests namespace. It passes by returning (or resolving) without throwing.
tcm test [namespace] runs them (and suites). The watcher, tcm check and VS Code run a scratch file's tests inline.
Tests run with Vitest, with nothing to set up. Each test definition is one Vitest test: expect, vi (mocks, spies, fake timers) and the rest come from vitest, which comes with tcm (nothing to install), and failures read like Vitest's, diffs included. Tests that use node:assert (or just throw) work too.
Results are cached by hash, and by the Node and Vitest versions. A test's hash covers everything it calls, so only tests whose code or dependencies changed re-run. A test whose code reaches files, the environment (process.env, node:os) or other processes is cached for the platform too (darwin-arm64), which those differ by. Tests that read the clock or the network can still go stale in the cache; --fresh re-runs everything. Only passes are reused: a failure can be the environment's or the runner's (a variable the test inherited, a bug in how tcm runs tests), which no key holds, so a test that failed runs again next time, and once that's fixed it passes without --fresh. One that fails again with nothing it runs changed says so (it failed last time too: a fix that didn't save shows as this). Watch expressions are cached the same way (a value, not an error), except one whose code reaches outside the codebase (Date.now(), Math.random(), a file), which is evaluated every time (scratch files).
Tests run apart from a session. A test, or a watch expression, doesn't see the session's branch (TCM_BRANCH, sessions): that's how tcm reads this codebase, and code that opens a codebase of its own (a temp store, a tcm it starts) would look for that branch there. Nor does it see TCM_BRIEF, the shorter reports an agent's session asks for: what the code under test prints is what it prints anywhere.
What ran is named. tcm test prints a line per test, (cached) or how long it took; the MCP server's tcm_test, which an agent reads, lists them by name instead, under the namespace they share: the ones that ran, and the cached ones too when tests were named or something failed (otherwise it counts them).
Results travel with the code. tcm push sends the results of the branch's tests the remote doesn't have, and tcm pull fetches the ones run on this Node version (and, where it matters, platform), so a fresh clone's tests don't run again (sync). A codebase that hasn't run tests takes the remote's Vitest version with them. A pulled result is trusted as the remote's code is.

11.1 Suites: testing a command as a program

A package's command (its bin) can be tested the way people use it: by running it. Its suites are test definitions that do, in namespaces the package's manifest names under suites (command → namespaces, relative to the package's):

// tcm: namespace greeter
export const manifest = { name: "greeter", bin: { greet: "main" }, suites: { greet: ["e2e"] } };

// tcm: namespace greeter.e2e
import assert from "node:assert/strict";
import { spawnSync } from "node:child_process";

export const testGreets = () => {
  const r = spawnSync(process.execPath, [process.env.GREET_BIN!, "tcm"], { encoding: "utf8" });
  assert.equal(r.stdout, "hello, tcm\n");
};

tcm test runs them after the other test definitions: each as a program of its own (passing if it returns), against the codebase's own build of the command, which it finds in <COMMAND>_BIN (GREET_BIN here; MY_TOOL_BIN for my-tool). They run side by side, as many as the machine's budget allows, each reported as it finishes; --definitions leaves them out, and --suites runs only them. They aren't run with Vitest. --bin greet=pkg/bin/greet.js runs them against another build of the command instead, such as the package tcm pack makes. A result is the code the test ran's, whichever build ran it: one kept from the codebase's own build holds against a package that has that code (a packed module is named by its component's hash), so a test runs once for its code (a merge's run serves pnpm verify after it), and a build without it (an older tcm.js) runs the test. It's also how to see a new test fail before a fix: tcm test tcmts.cli.test.e2e.merge --bin tcm=<an older tcm.js> (tcm_test's bin, {"tcm": "…"}), or, for tcm, a kept build's name (below).
tcm run <test> runs one as a plain program, once, uncached: it finds the command where suites do, the codebase's own build, unless the variable is set already (TCM_BIN=<tcm.js> tcm run <test>), which is kept, and said.
Results are cached by the code a test ran. V8's coverage of each run (every process it starts, a tcm it stops with a signal included) says which definitions executed, and the result holds until the test changes, or the code of one of those definitions does. A definition it didn't run can change without running it again, though everything using it is re-hashed: that changes no code the test ran. A class counts by the methods that ran, each on its own, and the rest of it (fields, constructor): a change to a method the test didn't call keeps its result. A constant counts only when code that ran refers to it (and then so do the constants its value refers to) if loading it does nothing anyone could see but make its value: one written out (a string, an object of literals), or one computed by code that reaches nothing outside (worked out as for pure constants), uses no npm package that isn't pure and no global but JavaScript's own, and doesn't wait or import. What a value only holds isn't run by loading it, so an array of tool definitions holding functions that read files is computed quietly. A computed one also counts if a function in it ran, or if state its computation could change (an object or array it uses, a let, a class) counts. Any other value computed when a module loads counts as run. A tcm the test started (a server it stops at the end) writes what it ran as it exits, so the run's result waits for every tcm it started to end, up to 15 seconds, even if the test didn't wait for it; one that doesn't end leaves the result uncached, since what ran isn't known. What ran is named as it was when the run started: a save during a run (another session's, on the same branch) that replaces code it executed leaves its result with the code it ran, so it holds for that version, not the new one. Code no name follows (a build packed before a change, say) counts through the code that ran referring to it; if none does, the result isn't kept (code a codebase the test made runs from that codebase's own cache is that codebase's, and doesn't count). Runs an earlier tcm kept another way aren't reused. Each suite keeps its passes for the last eight versions of the code it ran, so code back as it was (another branch's tests run between, on the same codebase) is cached again. A test still running after ten minutes (something it started keeps it alive) fails as timed out. Only passes are kept, as for test definitions: a program can fail for what's around it (its environment, the machine's state) rather than its code, so a suite that failed runs again next time, changed or not.

tcm's own end-to-end tests are its suites (tcmts.cli.test.e2e): a change to the mirror's code runs the tests that used a mirror again, and leaves the rest cached.

11.1.1 One budget for the machine

A suite is a program of several processes (tcm, the TypeScript checker, Vite, a browser), so runs side by side (agents' sessions testing, a merge's run, a verify's) would each start as many as there are cores, together many times that. Instead every tcm on the machine takes its work from one budget of slots: a suite runs once it has a slot, and gives it back when it ends; a run of test definitions takes one for each of its Vitest projects (Node's, and the browser's, together), and runs that many workers. The rest of tcm's heavy work takes a slot the same way: a build of an app (tcm build, and the builds tcm start and a deploy make), each of a sync's checker processes while it checks (sync), and each of pnpm verify's own programs (tcm verify, packing, npm, the far-away codebase) and its docs build. Each keeps about a core busy as it goes. However many runs there are, no more of that runs at once between them than there are slots.

How many: TCM_SLOTS (TCM_SUITE_SLOTS, its name before, still works), or else one for each performance core and one for every three efficiency cores (every core, on a chip with one kind), at most one for each 2 GB of memory, and at least 2: 6 on a Mac with 4 performance and 6 efficiency cores and 32 GB. A run given a smaller budget (TCM_SLOTS=2 tcm test) uses the first that many slots, shared with the rest: everything in those stays within it, and the others go on using theirs.
Waiting is first come, first served, and said briefly, on stderr (a verify says its own stages' waits in its output): a run says for what it waits and on whom when it first has to, naming the others by branch and kind of work (with their codebase's directory if it's another codebase, and their pid only where two have the same name): waiting for a slot: 6 of 6 in use (3 by session/2026-11-02-b933ee, 2 by session/2026-11-02-4ad254's test definitions, 1 by main's verify), 3 ahead in line (TCM_SLOTS sets how many). It says so again only when who holds the slots changes or how many are ahead changes much (a minute after it last said, at the soonest), or after five minutes of the same (still waiting for a slot after 6m20s: …), and once it has a slot, how long it took (got a slot after 6m30s); its suites waiting again soon after aren't said. The summary says how long, in all, it had a suite waiting: suites: 233 passed, 0 failed (19 cached), 6m12s waiting for slots. A tool's brief report keeps a run's last waiting line, or just the summary's total. A run waiting for several slots (Vitest's projects) gets them together, so two such runs never each hold some. A suite's ten minutes start once it runs.
The slots are files, in a directory of the user's (tcm-suite-slots-<uid> in the temporary directory, or TCM_SLOTS_DIR): slot-<i> holds the pid of the tcm using it, and line-… is a place in line. One whose process is gone (stopped, crashed, killed) is free for the next to look: there's nothing to clean up.
Inside a slot, what the work runs takes no more: what a suite runs (a tcm test of a codebase of its own, as tcm's end-to-end tests do, a build, a sync) is in its suite's slot, and runs a suite at a time with one checker; what a run of test definitions' workers start is in the run's slots; and work a tcm does inside work it took a slot for (a build during a verify's docs build) runs in that one. So nothing waits behind itself.

11.2 Showing it fails before the fix

A fix comes with a test, and the test should fail without the fix. A test definition shows it by running before the fix is saved. An end-to-end test runs a build of a command, so it's run against the build from before the fix instead (--bin). Working on tcm itself, the builds pnpm verify keeps (developing) are that, by name:

known-good: main's build, from the last passing pnpm verify, which pnpm tcm and agents' MCP servers run. A change on a session's branch, or not yet verified, isn't in it: compare against it while you fix.
previous, and older ones by id: the known-good builds before it (tcm builds lists them, with what changed in each), to find when a bug came in, or once the fix is in the known-good one. tcm builds make <entry> makes one from any history entry.

pnpm tcm test tcmts.cli.test.e2e.merge --bin tcm=known-good   # the new test, on the tcm without the fix: it fails
pnpm tcm test tcmts.cli.test.e2e.merge                        # on the codebase's own tcm, with the fix: it passes
pnpm tcm --build previous -C /tmp/repro view x                # a command, with an older tcm, on a codebase made for the repro

An agent does the same with tcm_test's bin ({"tcm": "known-good"}) and tcm_cli's build ("build": "previous", "cwd": "/tmp/repro"). A build without the fix's code runs the test again, whatever passed with the codebase's own.

11.3 Tests in a project you import

Tests are definitions, so they can live beside the code they test in an imported project too: write them as exports named test… in a test/ (or tests/) directory, and tcm import --tests brings them in with everything else (importing). Helpers there are plain definitions. A *.test.ts file next to them can register the same exports with another runner:

// test/math.ts
import assert from "node:assert/strict";
import { double } from "../src/math.ts";

export const testDouble = () => assert.equal(double(21), 42);

11.4 Where a test runs

A test runs in Node, unless it needs a browser: it (or something it uses or renders) touches the DOM or is a component that keeps state or handles events. Those run in Vitest's browser mode, in Chrome (the one installed; nothing is downloaded):

import { expect } from "vitest";
import { fireEvent, render, screen } from "@testing-library/react";

export const testCounter = () => {
  render(<Counter />);                             // Counter keeps state: this test runs in a browser
  fireEvent.click(screen.getByRole("button"));
  expect(screen.getByRole("button").textContent).toBe("clicked 1");
};

A function the test's code only writes out as source text (${init} into a generated file) doesn't make it a browser test: it doesn't run there (placement). Nor does a DOM type named only in types ((el: Element) => …, x as HTMLElement): types are erased, so code that also uses node:fs runs in Node. A browser test that can't load (it also uses Node: node:fs, say) fails saying what put it in the browser ("The test runs in a browser, since it uses lib.chart, which uses IntersectionObserver"), as tcm view says of the code.

Tests go through Vite, as apps do. In a browser test, server functions the code calls are reached by hash, as in an app: the test's server answers them. Testing libraries like @testing-library/react are imports like any other.

11.5 Testing app code in one process

In a Node test, everything runs locally, the counterpart of Unison's local Remote handler:

Server functions are plain function calls. withRequest(init, fn) from the server library runs code as if answering a request (method, URL, headers, cookies, IP), so login flows test with no server.
Tables live in memory, one store per test run, shared by the run's tests: give each test its own rows.
Jobs: runJobs() runs the queued jobs that are due, inline; runJobs({ all: true }) runs all of them.

11.6 The test framework's version

Vitest comes with tcm, as its browser side (Playwright, driving the Chrome that's installed) does, at the versions tcm was tested with: the vitest tests import is tcm's own, linked into the codebase's cache, so the tests and the runner are one Vitest. The lock has its version (tcm packages). tcm upgrade vitest@<range> chooses another for the codebase, installed in its cache like any package, as is the Vitest of tests written against a range tcm's isn't in. Since results are cached by its version too, a new one (a new tcm's) runs the tests again.