Data export
The whole database as portable files: CSV, JSON, SQLite and one slice per section.
Everything in the core catalogue can be taken away and opened with ordinary tools, independently of any front end. The export is rebuilt with the database on every deploy.
One URL
/db/export/manifest.json lists every table with its row count and columns, every export file with its size and sha256, and the size and sha256 of the full database.
Files
| File | Contents |
|---|---|
/db/export/<table>.csv.gz | One CSV per table (artifacts, locations, section_records, nodes, edges, captures, fts, layers, layer_items, layer_refs, layer_deps, evidence_grades, themes, meta). Large tables continue in -001, -002 parts. |
/db/export/cedwyn-data.json.gz | Every table up to 60,000 rows in one JSON document: {"table": [rows]}. |
/db/export/cedwyn-metadata.sqlite3.gz | The database without blob bytes and without the full-text index, as one SQLite file (written when it fits under 24 MB). |
/db/export/sections/<id>.json.gz | One slice per section: its row in themes, its own database rows (records) and every artifact it shows, with paths. |
/<id>/_db/section.sqlite | A section's own database, where the section ships one (for example /nft-art/_db/section.sqlite). |
/db/export/fetch-full.sh | Rebuilds the complete database, blobs included, from the chunk files, and prints the expected sha256. |
One command
# the complete database, every table and every stored file
curl -sO https://cedwyn.bucklerfamily.estate/db/export/fetch-full.sh && sh fetch-full.sh
sqlite3 cedwyn.sqlite3 "SELECT theme, count(*) FROM locations GROUP BY theme"
# everything except file bodies, as one small SQLite file
curl -s https://cedwyn.bucklerfamily.estate/db/export/cedwyn-metadata.sqlite3.gz | gunzip > cedwyn-metadata.sqlite3
# one section
curl -s https://cedwyn.bucklerfamily.estate/db/export/sections/wayback.json.gz | gunzip | jq '.artifacts | length'File bodies
Stored images and documents are in the blobs table of the full database and are also served one by one at /_blob/<sha>.<ext>. Files over 2.5 MB and video stay with the Wayback media stores and are referenced, not stored.