TODL Architecture

Audience: engineers new to the codebase. This document is a map — what the system does, how the pieces fit, and where to look. It describes the code as it stands today (@pragmatic-tech-ai/todl v0.36.x). For the language itself — types, meta-models, taxonomies, operators, and the model compiler — see The TODL language. TODL is built on todl-runtime.

Table of contents

  1. What TODL is
  2. The two-minute mental model
  3. Repository map
  4. Core concepts
  5. The compiler (compiler-services)
  6. Manifest and reflection
  7. Consuming a model (model-data, reflection-client, codegen, authoring)
  8. Publish and packages
  9. Projects and solutions
  10. The build system
  11. The runnable app
  12. Tooling (LSP, CLI, migrate)
  13. Package surface and dependencies
  14. Testing and conventions
  15. End-to-end walkthroughs

1. What TODL is

TODL is the TypeScript rebuild of the Typed Object Language — a typed substrate for authoring and reasoning over ontologies and taxonomies. In plain terms: you write .todl files that declare a domain’s meta-model (what a component is, what fields and relationships it has, how things are classified), then author concrete models (instances of those concepts wired together), and TODL compiles → validates → emits them into a reflective typed graph you can query, publish as a package, generate typed TypeScript clients from, and even compile into a runnable single-page app.

Three properties are worth internalising up front:

Deep dive: What TODL is — the full article.

2. The two-minute mental model

                       .todl sources
                            │
                 ┌──────────▼───────────┐
                 │  compiler-services    │   parse → load → validate
                 │  (the front end)      │
                 └──────────┬───────────┘
                            │ Repository  (the reflective typed graph)
          ┌─────────────────┼──────────────────────────────┐
          │                 │                               │
     emit/json         emit/manifest                    emit/todl
   (TodlDocument)   (binary + logical)                 (.todl text)
          │                 │                               
          │          manifest + reflection            
          │           (browser-safe read API)         
          │                 │                          
   ┌──────▼─────────────────▼───────┐        ┌─────────────────────────┐
   │  consume                        │        │  publish                │
   │  model-data / reflection-client │        │  compilePackage →       │
   │  codegen (typed DTO classes)    │        │  CompiledPackage →      │
   │  authoring (ModelDraft)         │        │  package stores/registry│
   └──────┬──────────────────────────┘        └───────────┬────────────┘
          │                                                 │
          │            build systems (solution-services)    │
          │        ┌────────────────────────────────────────▼──────────┐
          │        │  npm-package build  →  publishable package layout   │
          │        │  html-bundle build  →  runnable single-page app     │
          │        └───────────────────────┬─────────────────────────────┘
          │                                 │ index.html (self-contained)
          │                                 ▼
          └───────────────────────►  browser: TodlAppBootstrap mounts a
                                      mural Application over the model DTO

Read it top to bottom: text becomes a graph, the graph is serialised a few ways, and those serialisations are consumed (typed clients), published (packages), or compiled into an app.

Deep dive: The two-minute mental model — the full article.

3. Repository map

Everything ships from src/. Sibling folders cli/, examples/, shared/ are the demo/corpus suite and are not published (files: ["dist", "README.md"]).

Path Responsibility
src/compiler-services/ The compiler front end: parse, load, validate, emit, prelude, public check/checkAgainst API, predicate evaluation.
src/manifest/ The on-wire metadata format (a MoF-style table+heap binary) and its reader/writer/validator.
src/manifest/reflection/ A read-only, lazy System.Reflection-style API over a loaded manifest.
src/model-data/ Browser-safe runtime model access: ModelDataSource, ModelRegistry, pluggable connectors.
src/reflection-client/ ReflectedEntity — the typed read lens generated clients extend.
src/codegen/ Generates typed TypeScript clients (DTO package + per-concept entity classes) from a compiled model.
src/authoring/ The write path: ModelDraft stages instances over frozen bases and serialises back to .todl.
src/publish/ Pure compile-and-persist spine: compilePackage, CompiledPackage, package stores.
src/domain/ The multi-manifest runtime host (Domain, FrozenGraph) for versioned, cross-package reflection.
src/graph-api/ Read-only query surface (GraphQuery, Snapshot) and, under browser/, the app bootstrap.
src/solution-services/ The build systems, package sources/registries, project types, and multi-project solution builder.
src/application/ Composition/host layer (ApplicationBootstrapper, MuralHost); the generic model-browser view here is legacy.
src/runtime/ An internal handle-based consumption surface (TODL.ComposeGraph) — not the npm todl-runtime package (see §13).
src/language-server/ The LSP (stdio entry todl-language-server).
src/language-service/ Pure whole-project analysis behind the LSP (completion, hover, references, …).
src/migrate/ Mechanical rewriter for legacy sources → current surface.
src/index.ts The package’s root barrel (public API surface).

Deep dive: Repository map — the full article.

4. Core concepts

These are the terms the rest of the codebase assumes you know.

The typed graph

Defined in src/compiler-services/model/graph.ts.

Graph wraps a swappable GraphStore (default InMemoryGraphStore, which keeps dual adjacency plus type/metaKind indexes) and emits a changed signal on every mutation. Repository (model/model.ts) is the read+construct façade over a graph: resolve(id), instancesOf(concept), effectiveSchema(concept), supertypesOf, represents, termsOf, attr, ref/refs, validate(), and so on. EntityBase (model/entity.ts) is a lazy per-node read lens used by the compiler side.

Declarations

Fields vs relationships — the type-directed rule

Whether x = foo becomes a scalar attr or a graph edge is decided by the declared member type, never the surface syntax:

This is why a bare calls = api becomes a Relationship edge when calls is concept-typed. The oracle lives in parse/loader.ts (realizeValue, isReferenceMember). References may be forward — api can be declared later.

Namespaces, models, viewpoints, bound vocabulary

Classes and instanceof (clabjects)

A node with isClass is a partial, fixed-value definition — a clabject that is both a class and an instance. Instances declare instanceof <term-or-class> to inherit fixed values. effectiveFields/effectiveSchema merge class-fixed and own values (class-wins for the ontology view).

Bases, closure, and the prelude

Deep dive: Core concepts — the full article.

5. The compiler (compiler-services)

Pipeline: source text → parse (AST) → load (staged passes) → Repository → validate → emit. Public entry is api.ts.

parse/

parse(source, uri) turns text into a dumb-data AST (NamespaceNode with a flat declaration list) plus syntax diagnostics. The AST mirrors syntax verbatim — no reference resolution, no attr-vs-edge decisions. Every node carries two discriminants: DeclKind (Primitive, Concept, Taxonomy, Model, Instance, …) and, for values, ValueKind (String, Name, List, Composite, Boolean, Object, Edge). A bare identifier is always a NameValue; the loader decides its meaning later. Spans (diagnostics/span.ts) hang off most nodes for go-to-definition and diagnostics.

load (parse/loader.ts)

load(sources) / loadInto(model, sources, reserved, …) is the front door that populates a Repository. It stages in dependency order so forward references work; each pass opens a fresh Builder and commits before the next reads it:

  1. Parse & flatten sources into units; drop prelude redeclarations; flag orphan concrete objects declared outside a model {}.
  2. Resolve pre-pass — walk every reference, gated by namespace visibility; rewrite qualified/bare names to flat ids; collect undefinedIds (their edges are dropped at commit). Normalise and kind-check uses/libraries/conforms.
  3. Pass 1 — type shells: primitives, concepts (+ explicit extends), taxonomies + terms, viewpoints, annotations, operators.
  4. Pass 2a — members: concept fields/relationships, annotation params; invariant predicates parsed and queued; operators validated against schemas.
  5. Pass 2b — instances & models: materialise instances; classify each value attr-vs-edge from the declared member type; synthesise inline objects and reified operator edges through the same applyInstance machinery.
  6. Applications & finalize: annotation applications, application-root resolution (entrypoint), invariant registration, span recording.

Output: a populated Repository, spanned Diagnostic[], and a provenance map (nodeId → source uri).

validate/

validate(model) walks the committed graph and returns Diagnostic[]. The rule families:

emit/

api.ts — the public entry points

Deep dive: The compiler — the full article.

6. Manifest and reflection

Once compiled, a model is serialised into a compact manifest and read back through a reflection API that never touches the compiler — this is the browser-safe, runtime read path.

The compiler Repository and this reflection API are two worlds bridged by the emitter: Repository is the live authoring/compile graph; Manifest/TypeInfo are the packed, read-only runtime view. Parity between them is asserted in manifest/reflection/tests/repository-parity.test.ts.

For versioned, cross-package scenarios, src/domain/ (Domain, FrozenGraph, Heap) hosts many manifests at once and resolves cross-manifest type references.

Deep dive: Manifest and reflection — the full article.

7. Consuming a model

model-data, reflection-client, codegen, authoring.

model-data — browser-safe access

src/model-data/ is the runtime data layer, with no compiler dependency. ModelDataSource (abstract) is a typed façade over a single package’s reflection graph. It is fed by an IModelDataConnector (BundledModelDataConnector for an embedded document, DocumentModelDataConnector for one handed in at runtime), or synchronously via loadDocument(doc) / the static fromJSON(doc) that generated clients expose. ModelRegistry (a service-keyed lookup) indexes multiple sources by model id and designates a root. GenericModelDataSource is the untyped concrete source the model browser reads; typed codegen subclasses ModelDataSource directly.

reflection-client — the entity read lens

src/reflection-client/ReflectedEntity is the runtime analog of EntityBase: a typed read lens over one reflected instance. It exposes protected field(name), ref(member), refs(member); generated per-concept subclasses add typed getters over those primitives. References resolve through the identity-mapped host (EntityHost, implemented by ModelDataSource) so a reference always returns the same cached sibling object. MirrorReader/TermReader adapt reflection mirrors and taxonomy terms into the EntityReader shape.

codegen — typed clients

src/codegen/read-client.ts generateReadClient(repo, options) emits deterministic .ts: a class <Name> extends ModelDataSource (per-concept collection getters like get technologies(): readonly Technology[], a static fromJSON(doc), a createEntity switch) and one class <Concept> extends ReflectedEntity per concept with typed scalar/reference getters. model-package.ts wraps that into a runnable package that embeds each model’s document shard and an AppRegistry.Create() factory. naming.ts provides pascalCase/camelCase/pluralize/allocateNames (collision-checking). The generated collection names (pluralize(camelCase(conceptId))) are a load-bearing contract — the generated app.mu binds to exactly those names (§10, §11).

authoring — the write path

src/authoring/ModelDraft is a mutable overlay over frozen bases: add(descriptor) stages typed instances (fail-fast on dangling refs), and toTodl() / toTodlByFile() serialise the delta back to source. The InstanceDescriptor record it consumes is exactly what codegen’s authoring constructors emit. TodlFileStore is the save/load seam over an injected FileIO.

Deep dive: Consuming a model — the full article.

8. Publish and packages

src/publish/ is the pure compile-and-persist spine (compute is I/O-free; persistence is a seam).

Package sources (src/solution-services/todl-build-system/) form the resolution chain a build reads through, all implementing IPackageSource.TryGet(ref): SourcedPackage | undefined keyed by id@version: CompositePackageSource (first hit wins), CachingPackageSource (read-through, the only writer), SolutionCacheSource (IStorage cache laid out <id>/<version>/model.json), RegistrySource (terminal — fetches an npm tarball and reads it via TarReader). StoragePackageStore unifies publish-write and resolve-read.

Registries (package-manager/): IPackageRegistry is the backend contract; LocalNpmRegistry implements it over an IStorage directory; PackageRegistryClient is the rich per-connection surface (publish(dir), resolveClosure, …). The authored project.plexus manifest is transformed to an npm package.json by toPackageJson, which pins every base as an exact scoped dependency and embeds a todl block identifying the package kind and id independent of scope.

Deep dive: Publish and packages — the full article.

9. Projects and solutions

A project is a directory with a project.plexus manifest. ProjectType is MetaModel, Library, or Architecture (package-manager/manifest.ts). Base dependencies live on the manifest as ProjectBaseModelBindings (metaModels / libraries / architectures, each { id, version }).

src/solution-services/project-services/ owns the three project factories (TodlProjectFactory base + meta-model/library/architecture subclasses): create/open/save, manifest read/write, and the agent scaffold under .claude/. Meta-model and library projects are base-producing (they compile their own base document via IBaseProducingProjectFactory.compileToDocument); an architecture project is a terminal consumer that publishes nothing on its own.

RecursiveProjectReferencesResolver.Resolve(source, bindings) walks the bindings plus each package’s recorded dependencies (BFS, deduped) to reassemble the full base closure, collecting any unpublished binding as a problem rather than throwing.

Solutions (solution/solution-build-manager.ts) build many projects together: SolutionBuildManager derives a dependency-edge map from the base bindings, topo-sorts it (rejecting cycles), and runs each project through TodlProjectBuildManager. After each success it captures the freshly built model.json into an in-memory BuildOutputSource that is prepended to the source chain, so a dependent resolves its siblings’ just-built packages.

Deep dive: Projects and solutions — the full article.

10. The build system

Two layers: a generic engine and its todl-coupled realisation.

build-system-core — the generic engine

src/solution-services/build-system-core/ names zero todl types (hosts bind them through generics). The contract:

todl-build-system — the realisation

src/solution-services/todl-build-system/ binds the core to TodlBuildContext (core context + Manifest + Source) and ships two build systems, pre-registered by TodlBuildSystemRegistry.

npm-package (applies to MetaModel ∨ Library ∨ Architecture) — produces a publishable package layout. Seven actions: ResolveBasesAction → CompileModelAction → CompileMuralAction → StampResourceKeysAction → BakeResourcesAction → EmitBundleAction → EmitPackageLayoutAction. It resolves the base closure, compiles to a CompiledPackage, compiles any .mu, conditionally stamps resource keys and bakes presentation (through TODL’s own DefaultPresentationBaker, always present — a host may override it via PresentationBakerKey), emits bundle.json for producer projects, and stages package.json + model.json + src/ + a browser-safe handle module + resources/ into the sandbox. A second flavor, npm-publish, appends a terminal PublishPackageAction that packs the sandbox and pushes it to the registry threaded from the solution manager; with no registry associated it fails fast and writes nothing.

html-bundle (Architecture only) — compiles a project into a runnable single-page app. This is the per-project application compiler. Before any action runs, its BuildFlavor.Requires precondition checks that the project’s four generator-owned files already exist — src/app.mu, src/main.ts, generated/model.ts, generated/data.ts — and fails fast with a hint naming the generator to run if any is missing. Its constructor takes an IBundler, the one seam that lets the same build run headless or split across processes. The ordered action list (html-bundle-build-system.ts):

  1. ResolveBasesAction → ResolvedBases.
  2. CompileModelAction → CompiledModel (the full closure is on .fullDocument).
  3. EmitEntryAction → writes entry.ts into the sandbox root (build glue, never committed): imports app from the compiled src/app.mu.js (constructing the Application), imports the view-model class and the rehydrated model, does new <App>() (self-registering the view-model), then TodlAppBootstrap.Mount(app, model). The import/new order is load-bearing.
  4. CompileMuralAction → compiles every .mu under the project (hand-authored and the project’s own src/app.mu alike) via the shared MuralCompiler with MuralOutputLayout.Sibling, so src/app.mu → src/app.mu.js in the sandbox. Detects output collisions up front and stops on any compile error.
  5. BundleAppAction → collects the staged src/ + generated/ source and the compiled modules and delegates to the injected IBundler, producing the bundle string in AppBundle. The default EsbuildBundler runs esbuild (iife / browser / es2020 / keepNames) with a single-mural dedup plugin and a development-vs-dist condition probe; in Plexus’s renderer an IpcBundler ships the request to the main process instead.
  6. EmitBundledHostAction → renders a single self-contained index.html into the sandbox via HtmlShell.Render(JSON.stringify(fullDocument), bundle).

HtmlArtifacts holds the keys those actions pass (CompiledUi, AppEntry, AppBundle).

History note. This target has had three shapes: a single frozen 3.5 MB runtime bundle with only model data injected; then a per-project compiler whose view lived in one generated app.mu; and now an editable src/ application (src/app.mu + a paired src/main.ts view-model, with the DTO and data in generated/), with generation moved entirely into the content generators and the one node-only step moved behind the IBundler seam. The build requires those files rather than creating any.

Deep dive: The build system — the full article. See also Project content generators — the generator subsystem behind the project’s src/ application and generated/ DTO.

11. The runnable app

The build output index.html is fully self-contained. At runtime:

  1. The page holds <div id="todl-app-root">, then window.__TODL_APP__ = { …the compiled TodlDocument… }, then the app bundle <script>. HtmlShell (.../html-bundle/html-shell.ts) emits this; note it lives in the build system, not in graph-api/browser/, but shares the todl-app-root id contract with the bootstrap.
  2. The bundle’s entry.ts runs in a fixed order: evaluating the compiled src/app.mu.js constructs the mural Application; generated/data.ts has already rehydrated the DTO as model; then new <App>() runs the view-model constructor, registering the instance into the Application’s service container. src/app.mu, src/main.ts, generated/model.ts, and generated/data.ts are project content, produced by the content generators (§10); entry.ts itself is fixed build glue, emitted fresh into the build sandbox and never committed.
  3. It calls TodlAppBootstrap.Mount(app, model) (src/graph-api/browser/todl-app-bootstrap.ts). Mount guards against no-DOM (so the same entry is inert under Node), finds #todl-app-root, and calls app.initialize(new HtmlTarget(host), { theme: Pragmatic, autoScheme: { light, dark }, dataContext }). The bootstrap carries no view knowledge — the view is the project’s src/app.mu.
  4. mural renders: the Pragmatic theme resolves control styles; the x:root ContentPresenter resolves its $service(<App>) content from the container, the key-less DataTemplate typed to the view-model is auto-selected, and its $HelloText / $ConceptSummary bindings paint the view-model’s state.

Legacy path. src/application/MuralViewContribution + ModelBrowserVM + MuralHost mount a generic built-in model-browser view for any model. The new per-project path (TodlAppBootstrap + a compiled app.mu) does not go through it. ApplicationBootstrapper/ApplicationEntryPoint/ModelRegistryContribution remain the shared composition machinery; only the view contribution diverges.

Deep dive: The runnable app — the full article.

12. Tooling

Deep dive: Tooling — the full article.

13. Package surface and dependencies

src/index.ts is a large root barrel re-exporting across every layer (compiler model, model-data, application, codegen, authoring, publish, predicate, validate, parse, migrate, and the solution-services surface). Beyond the root, the package exposes focused subpath exports: ./language-service, ./language-server, ./package-manager (+ /connections), ./build-system-core, ./todl-build-system, ./domain, ./graph-api.

Every subpath’s import has two conditions: development → raw ./src/**/*.ts and default → compiled ./dist/**/*.js. In-repo tooling and tests run with tsx --conditions=development, importing TypeScript source directly; published consumers get dist. The published package ships only dist and README.md.

Dependencies: @pragmatic-tech-ai/mural (the UI/visual-engine framework — Application, HtmlTarget, Pragmatic theme, Observable) and @pragmatic-tech-ai/todl-runtime (the DI + reactive substrate — CompositionRoot, ServiceProvider, Signal, Disposable). The root barrel re-exports Signal/Disposable and the prompt types from todl-runtime, so a todl consumer gets the runtime primitives without importing that package directly.

Two things called “runtime” — do not confuse them.

Deep dive: Package surface and dependencies — the full article.

14. Testing and conventions

Deep dive: Testing and conventions — the full article.

15. End-to-end walkthroughs

A. A .todl file becomes a validated graph

check(sources) seeds a Repository with the prelude, parses each source to an AST, resolves names (visibility-gated), stages types → members → instances through a Builder (classifying each value attr-vs-edge by declared type), applies annotations, then runs validate. Out comes a populated Repository plus spanned diagnostics — query it via Entity, or emit it.

B. A meta-model or library becomes a published package

TodlProjectBuildManager.Build({ …, BuildSystemId: "npm-package" }) runs ResolveBases (reassemble the closure through the package-source chain) → CompileModel (compilePackage → CompiledPackage) → CompileMural → StampResourceKeys → BakeResources (conditional) → EmitBundle (producer bundle.json) → EmitPackageLayout (stage package.json + model.json + src/ + resources into the sandbox). On success the sandbox is promoted to the output — a complete package. The npm-publish flavor adds a terminal PublishPackageAction that packs the sandbox and pushes it to the IPackageRegistry threaded from the solution manager, from where downstream projects resolve it.

C. An architecture project becomes a runnable single-page app

The html-bundle build requires the project’s src/app.mu, src/main.ts, generated/model.ts, and generated/data.ts to already exist — content the project generators own, not this build (§10) — and fails fast if any is missing. It resolves + compiles the closure, emits entry.ts (wiring) into the sandbox, compiles every .mu with mural (sibling layout), bundles the staged source through the IBundler (default esbuild: keepNames, IIFE), and emits one self-contained index.html inlining the model as window.__TODL_APP__ and the app as a script.

D. What happens when someone opens that index.html

The page evaluates the bundle: the compiled src/app.mu.js constructs the mural Application, generated/data.ts rehydrates the DTO as model, new <App>() registers the view-model into the service container, and TodlAppBootstrap.Mount mounts the app into #todl-app-root with the Pragmatic theme. mural resolves the x:root ContentPresenter’s $service(<App>) content and paints the key-less DataTemplate bound to the view-model’s $HelloText / $ConceptSummary.

Deep dive: End-to-end walkthroughs — the full article.

This document is generated from the in-repo docs/ARCHITECTURE.md. Keep the two in sync as the architecture evolves.