20. Sharing code

Because a hash identifies code completely, sharing a codebase means sending only the hashes the other side lacks, and running code remotely means sending its missing closure.

20.1 Remotes

tcm remote add <name> <url|dir> [--token t] [--fingerprint sha256] adds a tcm serve URL or another codebase directory: the token is the server's (its maintainer's) or a contributor's, and a public server needs none to pull. Adding a remote of the same name again replaces it (with a new token, say). tcm remote lists them; tcm remote rm <name> removes one. A URL used where a remote's name could be (tcm pull https://…, a library's source) is asked with no token: a token goes only to the remote it was added with.

20.2 Push and pull

• Sync is by hash, history included: what's sent is what the other side lacks of a branch's history (its entries and their namespaces, node by node: a node the other side has, it has everything under), the definitions those name, and the package lock entries they use (versions and tarball hashes; see packages). Both sides end up with the same history, hash for hash.

• tcm pull [remote] [branch] mirrors into <remote>/<branch> (e.g. origin/main). If your branch's history is part of theirs, it fast-forwards; if theirs is part of yours, there's nothing to do; otherwise it merges from your common ancestor.

• tcm push [remote] [--as branch] [--force] sends this branch's history and moves the remote branch to it. It only moves a remote forward: if the remote has history you don't, it refuses (pull first) unless --force, which replaces the remote branch's head.

• What a push sends is history: every version of every definition its entries named. A branch whose sessions landed squashed (as tcm_merge lands them, agents) sends what each landing ended with; one merged with its history (tcm merge without --squash) sends every step too.

• Pushing to a server running tcm start deploys (production).

• Proposing changes for review is a contribution: tcm contribute pushes a branch and opens one on the remote (contributions).

• Test results travel with the code. A push sends the results of the branch's tests the remote doesn't have (under the Node and Vitest versions, and platform, they ran on), and a pull fetches the ones this machine can use, so tests that passed there don't run again here (testing). So do lint findings, under the linter's version and configuration: what was linted there isn't linted again here (linting). A remote running an older tcm, which doesn't keep them, is pushed to and pulled from as before. Production builds, watch values and kept constants stay each machine's own: a kept constant's value is code, which isn't taken on trust.

Everything named travels: code, and the codebase's files (pages, images, configuration), which are history like code. A file from elsewhere is checked by its content: its text and links must be what its hash says, and bytes must match their digest.

20.3 Serving a codebase

tcm serve [--port 7737] [--host 127.0.0.1] [--token t] [--no-run] [--public] [--tls [--tls-cert f --tls-key f]] [--insecure] [--site dir [--moved path]] [--proxy path=url,…] serves the codebase over HTTP(S).

• The server's token is its maintainer's. Clients with it can push any branch, pull, merge contributions, manage contributors and, unless --no-run, run code. Without --token or $TCM_TOKEN it makes one up and prints it; one it was given isn't printed (so it stays out of a host's logs).
• Contributors have tokens of their own, as on Unison Share: tcm user add <handle> (the maintainer's, from any machine with the server's token, or on a codebase directory) makes one and prints it once. A contributor's token pulls, pushes only branches under @<handle>/ (tcm push and tcm contribute name them so: cubes is pushed as @ada/cubes), opens contributions of them, comments, and closes or reopens its own; what it writes is signed @<handle> by the server, whatever the client says. It can't push other branches, merge, release, run code or send test results and lint findings, which are taken on trust (its pushes leave them out). tcm user says who you are on a remote (and, to its maintainer, lists its contributors); tcm user rm <handle> stops a token working. The server keeps each token's SHA-256 digest, not the token, in the served codebase's settings.
• --public lets anyone pull, and read contributions, without a token: what anyone can download, as the docs site is. Writing still takes one, and a wrong token is refused rather than read as none. tcm's own server is public (developing).
• --tls uses --tls-cert/--tls-key, or a self-signed certificate it creates, and prints the certificate's fingerprint. Clients pin it with tcm remote add … --fingerprint, and the token is only written after the pin checks out.
• Serving beyond localhost without TLS is refused unless --insecure.
• /forge/ is the web UI: the codebase's pages, search, history, and its contributions' review, started the first time it's asked for. / goes to it, on a server with no --site.
• --site <dir> answers GET requests from a directory of static files, to anyone: a docs site published beside the codebase. / and directories are their index.html, /cli is cli.html (before a directory of that name), and nothing outside the directory is served. /forge/ is the web UI's, not the site's. A path with no file gets the nearest 404.html above it: a static site's own, under its path, renders the app's other paths. A directory asked without its slash (/docs) goes to it with one (a 301), so its page's relative links are its own. Everything else still needs a token (or --public, to read). tcm's own server serves its docs this way (developing). A browser gets each file once: its ETag is its content, so asking again is a 304; a file named by its content (style.<digest>.css, the first 12 or more hex digits of its SHA-256) is kept for good (immutable); and text is sent compressed (brotli or gzip, compressed once while the server runs). The docs site's build names its stylesheet that way, and its pages ask Chrome to prerender a page when a link to it is pointed at.
• --moved /docs says the site's pages moved under a path: a GET for a path the site has no file for, that it has under /docs (or that an app it proxies serves there), is redirected there (a 301, its query kept), so links from before the move keep working. tcm's own server moved its docs so (developing).
• --proxy /examples/notes=http://127.0.0.1:7801,… passes every request under a path, to anyone, to a server beside it (an app tcm start --base /examples/notes serves there), or to whichever of several answers (|-separated, =http://127.0.0.1:7801|http://127.0.0.1:7901: an app's version and its next, while one replaces the other; the one that answered last is asked first, and a request one refuses goes to the next, its body read first so it can): method, headers (with the Host the browser asked, and x-forwarded-for/-proto/-host) and body, streamed both ways; what comes back as text the app didn't compress (pages, server components' trees, JSON, scripts, styles, SVG) comes gzipped to a browser that takes it, each piece as it comes, so a page that streams still streams. The longest path that holds a request wins. While its server isn't answering (starting, or built again after a deploy) a request gets a 503 with Retry-After: 5, and a page asked for says so and reloads by itself.
• Big bodies travel compressed (gzip). A server's replies say it takes compressed requests, and a client that has seen this compresses a big one (definitions, mostly: their JSON packs about eightfold); it asks for compressed replies too. A compressed request is held to the server's size limit (64 MB) unpacked. With an older server or client on the other side, everything goes as it is.
• Definitions travel lean to a tcm that says it takes them (x-tcm-records: lean, both ways: a server's replies say so, a client's requests do): without what the receiver derives itself anyway, their IR past its head (the tag, lineage salt and JSX runtime) and their type shapes. That's about half of what a push uploads. A file goes whole (its IR is its content), and an older tcm gets whole records.
• A push sends definitions in batches of up to 40 MB, and waits up to 10 minutes for each to be checked; other requests, a minute.
• Idle connections are kept open for 65 seconds, and a client resends a request once if the connection it reused was closed meanwhile, so a client re-hashing a big pull, or a proxy in front of the server, doesn't break it.

20.4 Running remotely

tcm run <name> [args] --on <remote|url|dir> sends the definition's missing closure, runs it there, and streams stdout/stderr back.

20.5 Sync never trusts the sender

Received definitions are checked before anything is stored:

• each component must arrive complete;
• every reference must resolve, to a definition it has or one that came with it;
• rendering and analyzing them locally (resolving every name they use, deriving their IR) and hashing must reproduce the claimed hashes.

That's every sender, the server's maintainer included: what's stored under a hash is the code that hash is of.

What's derived from the source is what the local analysis says, never what the sender does: the IR (what a hash is of), dependencies, signatures, schemas and how a definition renders. Nothing received is executed during checking, not even a pure constant whose value tcm keeps (that's computed when the code is materialized to run), and npm installs use --ignore-scripts. Tampered source sent under an existing hash is rejected, and IR sent beside honest source isn't kept (the source's own is); tcmts.cli.test.e2e.sync checks both.

Definitions are checked a batch at a time, whole components each after the ones it uses, so a long history needn't fit in memory at once: a push sends them so, and a pull checks what it fetched so (set aside on disk as it comes, until its batch is checked). A batch is shared among checker processes, each compiling and analyzing a run of its components at once: TCM_CHECKERS says how many (half the machine's cores, up to 4, unless it says; 1 checks in tcm's own process, as a sync inside a suite does). Each takes a slot of the machine's while it checks, shared with every tcm's tests, builds and verifies (testing). What's stored is the same either way; tcmts.cli.test.e2e.sync checks it.

Receiving doesn't work out a term's type shape either (what search by type matches, and how a signature names types renamed since): most of what a sync brings is versions only history names. A term gets its shape as a branch comes to name it.

Types are judged where code is merged, not where it's received. That a definition typechecks is a property of code that's already what it claims, not part of what it is, so receiving doesn't ask. Whether received code typechecks:

• a branch its owner pushes was typechecked where it was saved: tcm saves only code that typechecks, and the server doesn't check it again;
• a contribution is typechecked when it's merged: what it brings, under each definition's recorded check profile. A merge that doesn't typecheck is refused, with the errors, so nothing ill-typed reaches the branch it targets that way;
• code no tcm would save can still be posted by hand, over HTTP: a server keeps it on the branch it was pushed to, and merging is where it's caught (limitations). tcm verify on that branch reports its type errors.