Architecture
The core catalogue database, section data, federation and the no-duplication rule.
Three layers
| Layer | Where it lives | What it holds |
|---|---|---|
| Section source | One branch per section | The section's pages, styles, scripts and its own data files (JSON, CSV, Markdown). |
| Assembled site | Branch site | Every section copied under /<id>/, links rewritten, shared files stored once in /_blob/. |
| Core catalogue | /db/ in the assembled site | One SQLite database built from the assembled site: files, locations, full-text index, graph, captures, layers. |
The core catalogue database
Built by tools/build_db.py on every deploy. Size at the last build: 672 MB, split into 2,565 chunk files of 256 KB (/db/cedwyn.sqlite3.000 onwards). Counts: 11,829 artifacts, 18,857 locations, 4,042 stored blobs, 1,166 nodes, 271,768 edges, 8,481 full-text rows, 183 captures. Live figures: /db/stats.json.
| Table | One row per | Key columns |
|---|---|---|
artifacts | unique file (by sha256) | id, sha, kind, mime, bytes, title, year_min, year_max, has_blob |
locations | place a file appears | aid → artifacts.id, theme (section id), path, url |
blobs | stored file body | sha, data (raw bytes: images, PDFs, data files up to 2.5 MB each) |
fts | page or text file | FTS5 index: title, body, aid, theme, path |
nodes, edges | graph node / relation | page→embeds/links→artifact, artifact→mentions→year, artifact→cites→host, artifact→in_theme→section |
captures | archived web address | key, url, title, latest, captured |
layers, layer_items, layer_refs, layer_deps | reading over the evidence | items point at artifact ids; they never copy content |
section_records | record from a section's own database | theme, kind, id, seq, data (JSON) |
evidence_grades | evidence record | ev, grade, origin, path, note |
themes, meta | section / build fact | section id, title, source, file and page counts; build statistics |
Shared without duplication
- Content addressing. A file that appears in several sections is stored once, at
/_blob/<sha>.<ext>, and once inblobs. Each appearance is a row inlocations. - References, not copies. A layer such as 1000 Cuts stores artifact ids. A section that needs a core record links to it or reads it from the database.
- Served from the database. After the build, data and evidence files whose exact bytes are in the database are removed from the deploy and served from the database chunks by the service worker (
/cedwyn-sw.js) and the page fallback (/_cedwyn/runtime.jswith/_cedwyn/dbfs.js). Files that browsers fetch before a service worker runs stay as files.
Federation: section data and the core
Each section is the source of truth for its own data. The convention for a section's own database: the branch ships public/_db/section.sqlite, a small SQLite file with one table
CREATE TABLE records(kind TEXT, id TEXT, seq INTEGER, data TEXT, -- data = one JSON object
PRIMARY KEY(kind, id));
CREATE TABLE meta(k TEXT, v TEXT);
holding every word and record the section shows (interface strings are rows too, for example kind = 'words'). The section's database works on its own and is served at /<id>/_db/section.sqlite. On every build, tools/build_db.py merges each section database into the core table section_records(theme, kind, id, seq, data) and adds graph edges: in_theme, derived_from (a record made from a core artifact) and shows (an image the record displays). The section's pages read their rows back from the core: SELECT kind, id, data FROM section_records WHERE theme = ? ORDER BY seq.
So a section owns its data, the core holds one merged copy for reading, search, graph and export, and a record that already exists in the core (an image, a document) is referenced by path or sha256, never copied into the section. Sections with live writes keep a small store of their own (see Sections). Every section can be taken out as a slice: /db/export/sections/<id>.json.gz holds its row, its own database rows and every artifact it shows (see Data export).
Why one built database rather than a live database per section: it costs nothing to serve, it cannot drift out of sync, and it stays inside the free limits of the host. Live per-section databases add servers, quota and replication with no gain at the current scale.
Graph and GraphQL
The nodes and edges tables are a ready graph for retrieval and analysis. A GraphQL schema is resolved in the browser against the same database: see API.