Libraries from other codebases — tcm-ts

9. Libraries from other codebases

A library is part of another codebase, used from yours as it was released there: its names are mounted read-only under lib, as Unison's are, and your code uses them like its own.

$ tcm lib add ../alice geo                        # geo's newest release, from another codebase's directory
added app.lib.geo: geo@1.0.0 from ../alice (3 names; 3 definitions received, verified by re-hashing)
$ tcm lib add https://alice.dev geo@1.0.0         # a release, from a server (tcm serve)
$ tcm lib add origin '#476qg45r…' --as geo        # a namespace by its whole hash, from a remote's name (--as: what to call it)
$ tcm lib add tcm react                           # a library tcm ships, from the tcm you're running
$ tcm lib
app.lib.geo  geo@1.0.0  from ../alice
import { area } from "#codebase/app/lib/geo";
export const total = () => area(2);

9.1 What a library is

  • A namespace, by its hash. A release names its package's namespace in the history entry it was made at (releases); <package> alone is its newest release, <package>@<version> one of them, and #<hash> a namespace by its whole hash, released or not. Only that namespace comes, with everything its code uses and the package versions it runs on (packages): not the history behind it.
  • Checked as it arrives. What's received is re-hashed, as a pull is (sync), so a source can't send other code under a hash: where it came from matters only for whether it answers.
  • Mounted at <root>.lib.<name>: in your project's root namespace (its one top-level namespace, if it has one, not counting the libraries mounted already), lib.<name> otherwise; --at <name> mounts it elsewhere. The place must have no names yet.
  • Called after its package: <name> is the package's name, as a name can be (@alice/geo-tools is geo_tools); --as <name> calls it another, as one added by hash, which names no package, must be.
  • Recorded beside it. <namespace>.libraries, a JSON file, says what each library mounted in that namespace is: its namespace's hash, the release it is, and its sources. Adding one is one history entry (lib add app.lib.geo), naming its definitions and the record, so a library travels with your branch: pushed, pulled, merged, reset, reverted like any change, and anyone with your codebase has it.
  • Its own libraries come along: they're part of its namespace (app.lib.geo.lib.shapes), each by hash, so two libraries using the same one share its definitions.
  • What it uses outside its namespace arrives unnamed. A package's code may use its codebase's other definitions; a library brings them by hash, but only its namespace's names, so they show as unnamed versions in yours. A package made to be added this way says so (tcm libraries), and then holds everything it uses.

9.1.1 Making one: a tcm library

A package whose manifest says npm: false is a tcm library: one others add with tcm lib add, not an npm package.

// tcm: namespace geo
export const manifest = { name: "geo", npm: false };
  • It holds everything it uses. Its release is refused while its code uses a definition outside its namespace, naming them (tcm release check geo says so too): move it in, or pass it in, so a codebase that adds it has every definition named.
  • It isn't packed for npm (tcm pack geo is refused), and another package of its codebase that uses it carries its code, rather than depending on an npm package of that name.
  • Its versions follow its API, with the release checks, as any package's do (releases).
  • What tcm treats specially, it says: roles names definitions tcm knows by their hash (a component linking a stylesheet, a function making a table or a recurring job), and host the version of tcm's runtime interface its code is written against, so a tcm that doesn't serve it says so rather than building an app that breaks (publishing).

tcm's own libraries are made so, in tcm's codebase: records (tcmts.records), the records libraries are announced and talked about with: codes.tcm.release, the attestations (codes.tcm.review, codes.tcm.advisory, codes.tcm.yank) and codes.tcm.star, their lexicons and keys, and each read from a record field by field; react (tcmts.react), React's routes, links and data for an app (React), speaking host protocol 1 with Stylesheet in the stylesheet role; and server (tcmts.server), the server API (the request, cookies, sessions, rate limits, tables, transactions, jobs), on tcm's server host #tcm/host (host protocol 1), with table and every in their roles. tcm.codes announces them (tcm lib add @tcm.codes/react), and each one's README says how to use it.

The libraries tcm ships come with it, as a source called tcm: react and server are in tcm's package (each its newest release when that tcm was made), so tcm lib add tcm react mounts it with no network, at <root>.lib.react, and tcm lib add tcm server at <root>.lib.server. Their sources are tcm, so tcm lib upgrade react moves it to the release the tcm you run now ships. A remote you named tcm is asked instead (publishing).

9.2 What it reaches

Adding a library says where its code reaches outside itself, worked out from the code (as tcm works out what to keep, execution), not from what its author says: the clock, randomness, the network, files, the environment, other processes, output. Each with the definitions that do it, and the package, where it's a package's code that does:

$ tcm lib add ../alice geo
added app.lib.geo: geo@1.0.0 from ../alice (5 names; 5 definitions received, verified by re-hashing)
  reaches the clock: now, today (date-fns)
  reaches the environment: home
  • An upgrade says what's new. tcm lib upgrade (and --check) says what the new release reaches that the old didn't (now reaches the network: load), and what it no longer does.
  • tcm lib says what each library reaches, in a word each (reaches the clock, the environment).
  • What it could reach, not what a run did: a definition that calls fetch anywhere counts. Its tests don't.
  • A package whose code couldn't all be followed (a computed import, eval) is named as one tcm can't tell for.

9.3 Read-only

A library's names stay as it was released. tcm update, patch, mv, rm, alias onto its names, fmt, and anything else that changes one name, refuse, as does editing its record:

$ tcm patch app.lib.geo.area --old 'side * side' --new 'side ** 2'
app.lib.geo.area is in the library app.lib.geo (geo@1.0.0), which is read-only: to change it, copy it into a
namespace of your own (`tcm alias app.lib.geo.area <name>`) and change the copy, or `tcm lib rm app.lib.geo`
  • An update elsewhere leaves it alone. Its code uses only its own definitions, so changing yours doesn't change it, even where yours is the same code as one of its own (the same hash): that library's code keeps using its own.
  • A copy you change is yours. tcm alias a definition of it into your namespace and change that: the library keeps its own, and your code that used it follows the change, as it follows any update.
  • Its packages are its own. A manifest in it makes it a package where it came from, not in your codebase: tcm release, pack, releases and package suites don't see it.
  • Its tests run with yours. tcm test app runs the tests under app.lib.geo too, cached by hash as any are, so a release's run once.

9.4 Upgrading

tcm lib upgrade geo moves a library to its package's newest release (or the one given: geo@2.0.0, or a namespace by #<hash>), from where it came (--source another place, kept for next time). What changed in its API is said first, as a release says it (semver checks); --check stops there.

$ tcm lib upgrade geo
app.lib.geo: geo@1.1.0 → geo@2.0.0: breaking changes

  major  area accepts less for a parameter: side: number → { side: number; }
         function_parameter_type_narrowed: arguments the old version accepted may not be accepted

upgraded app.lib.geo to geo@2.0.0 (2 replaced, 0 added, 0 gone)

  1 dependent definition(s) no longer typecheck; see `tcm todo`
  • It's one update. The library's names become the new release's, and each of its definitions that the new one has another at the same name is replaced by it, as tcm update replaces one: your code using it follows, re-hashed where the API kept its types and checked again where it didn't. What no longer typechecks is to do, on the old version until it's fixed (fixing what an update broke). One history entry (lib upgrade app.lib.geo), so tcm reset 1 undoes it.
  • Refused while your code uses what the new release doesn't have (a definition it removed): change that code first, or add the new release beside the old one (tcm lib add ../alice geo@3.0.0 --at app.lib.geo3) and move to it bit by bit.

9.5 Sources and mirrors

A library is fetched from its sources, tried in order: where it came from first, then any added with tcm lib sources. What any of them sends is re-hashed as it arrives, so a mirror needs no trust, only to answer.

$ tcm lib sources geo --add https://mirror.example
app.lib.geo comes from ../alice, then https://mirror.example
$ tcm lib upgrade geo
  tried ../alice: no tcm codebase here; run `tcm init` first
app.lib.geo: geo@1.0.0 → geo@1.1.0: additions (from https://mirror.example)
  • Any codebase can be a mirror. One that has a library, mounted or as its own code, serves it by its namespace's hash (tcm serve, or its directory). A release by name (geo, geo@1.1.0) is found only where releases are: the author's codebase, or one that pulled its history.
  • The first that has it is used, and each that didn't answer is said. tcm lib upgrade --source <s> tries s first, and keeps it first.
  • tcm lib sources geo --rm <s> stops fetching from one; a library keeps at least one source.
  • Sources are history, in the record, like the rest of it: a teammate pulling your branch fetches from them too.

9.6 Announced releases

An author can announce a release on the AT Protocol (tcm release --announce, releases): a record in their repository saying which namespace hash the version is, and which servers have it. Such a release is added by their handle, with no source to know:

$ tcm lib add @alice.dev/geo@1.2.0          # or @alice.dev/geo: the highest version she announced
added app.lib.geo: geo@1.2.0 by @alice.dev from https://tcm.alice.dev (3 names; 3 definitions received, verified by re-hashing)
$ tcm lib
app.lib.geo  geo@1.2.0  by @alice.dev, from https://tcm.alice.dev · reaches nothing outside itself
  • Who announced it is a DID. The handle is looked up (DNS or HTTPS) and must be named back by the DID's document; then the account's codes.tcm.release records are read from the PDS that document names, and the version picked (@<DID>/geo works too). The record used is checked against their signed repository (com.atproto.sync.getRecord, the commit's signature by the DID's key): a PDS that lists another namespace for the version than the repository signs is refused. Its package must be an npm name and its version semver.
  • A library's code doesn't move your package versions: what it runs on is taken only for packages your codebase hasn't locked, and only versions within their ranges.
  • The code comes by hash from the servers the record names, as any library does, and is re-hashed as it arrives: a server, or the record, can't send other code under that hash.
  • The record entry keeps the DID ("did", and the handle as it was, for showing), and the record's servers as its sources. tcm lib upgrade geo reads that DID's announcements for a newer version, never the handle again, so whoever holds the handle later can't stand in for its author; it fetches from the new record's servers, then the library's other sources (--source first), which it keeps.
  • A version announced in another hash format is refused: it names hashes this tcm doesn't make.

tcm lib search asks a hub what its libraries hold: by name, words in docs, or a type, as tcm find searches a codebase, across each library's newest version. Each library comes with what matched in it and the line that adds it.

$ tcm lib search area --hub https://hub.example
@alice.dev/geo 1.2.0  4 dependents · ★ 12
  Shapes and their areas.
  geo.area  (side: number) => number  A square's area.
  tcm lib add @alice.dev/geo@1.2.0
1 definition in 1 library, from https://hub.example
$ tcm lib search ':(s: Shape) => number'     # the hub it asked last time
  • The hub is kept (a setting of the codebase): --hub once, and later searches ask it. Without one, it asks tcm's own, https://tcm.codes/hub (not kept: a codebase that never chose a hub follows tcm's); tcm hub runs another.
  • It asks GET <hub>/api/search?q=…, whose JSON is each library's author (DID and handle), package, newest version, description, dependents, stars, add line and what matched (20 at most, and how many more). A hub leaves out what its authors yanked (hub): the newest version is the newest not yanked, and a library yanked through isn't found.
  • What it says is the hub's word: the add line fetches the release by its author's announcement, checked by hash as any announced release is, so a hub can point at a library but not change what it is.

9.7 What people say: reviews, advisories, yanks

Anyone can say something about a release, as a record signed in their own AT Protocol repository: a review (what they checked), an advisory (it's vulnerable, or badly wrong: how bad, and what fixes it), or, from its author alone, a yank (don't use it). Each is about the code by its namespace hash, so it holds for that code wherever it's released or mirrored.

$ tcm lib attest @alice.dev/geo@1.2.0 review -m "Read all of geo.project; it matches proj4."
reviewed geo@1.2.0 (#x7k3q2mf) as @maya.fyi: at://did:plc:…/codes.tcm.review/3l…
$ tcm lib attest geo advisory --severity high --fixed-in 1.2.1 -m "project divides by zero at the poles"
$ tcm lib attest geo@1.2.0 yank -m "use 1.2.1"          # its author, by the release's name in their codebase
  • What's attested is a library here (its name or mount), a release someone announced (@alice.dev/geo@1.2.0), a release of this codebase (geo@1.2.0), or a namespace by its whole hash (#…).
  • Signing in is the same as announcing's (releases): one sign-in, kept, does for both (--as picks the account).
  • A yank counts only from the release's author: anyone else is refused, and one written anyway isn't shown. A hub stops listing what's yanked (hub).
  • The records are codes.tcm.review, codes.tcm.advisory (with severity: low, moderate, high or critical, and fixedIn) and codes.tcm.yank, each holding its subject (namespace, and the release and its author for showing), its body (markdown, at most 30,000 bytes) and createdAt; --fixed-in is at most 64 characters.

tcm lib add, tcm lib upgrade (of the version it moves to) and tcm lib say what's been said of a library: yanks first, then advisories, then reviews.

$ tcm lib
app.lib.geo  geo@1.2.0  by @alice.dev, from https://tcm.alice.dev · reaches nothing outside itself
  yanked by its author @alice.dev: use 1.2.1
  advisory (high) from @alice.dev: project divides by zero at the poles (fixed in 1.2.1)
  reviewed by @maya.fyi: Read all of geo.project; it matches proj4.
  • Whose: each library's author's, and those of the people the codebase trusts. tcm lib trust --add @maya.fyi trusts someone (--rm stops); tcm lib trust lists them. The list is a file of the codebase, <root>.lib.trust, so it's history: everyone working on the codebase sees what the same people say. Everyone's shows on a hub.

9.8 Removing one

tcm lib rm geo (or its mount, tcm lib rm app.lib.geo) removes its names and its entry in the record (the record too, once it's empty). It's refused while code outside it uses something only it names:

$ tcm lib rm geo
app.lib.geo is used by app.total: change them not to use it first

tcm lib notes a library whose names aren't the namespace it was added as (a merge can leave that).

built from c482a14e