Wayback Websites Archive

• Background: many of the estate sites cite pages held by public bodies and archives (heritage records, archive catalogues, newspapers, council pages). Those sites go offline (for example, the Welsh HER portal returned Cloudflare 522 errors from 13:21 UTC on 24 October 2026) and change without notice.
• Objective: one place, the Wayback Vault (datro branch wayback), that holds every archived web page for the programme, with a stable "latest" address per page and numbered versions behind it.
• Scope in: the 211 pages that Watchtower tracks as concerning the farm, Tŷ Mawr, Llandough and the 1987 case. Text, structure and a screenshot where one was taken.
• Scope out: GGATE003317 (a different Great House Farm, Michaelston Road, Cardiff); the Great House Farm at Mamhilad; paywalled or login-only pages; WhatDoTheyKnow request pages until they are redacted.
• Approach: reuse the existing captures (internetarchive branch: GitHub runner and box captures), convert each to a self-contained static page, then move capture and change detection so that they write straight into the Wayback Vault.

• Address: /websites/<host>/<page>/latest.html is always the newest copy (the stable link target; /websites/<host>/<page>/, latest.htm and #latest resolve to it). Each version is also kept as v1.0.0.html, v1.1.0.html and so on.
• Record: meta.json per page (source URL, title, first captured, versions with date and SHA-256 of the normalised text, current version) and /websites/index.json for the whole set.
• Versioning: first capture v1.0.0; a change to the readable text is a minor version (v1.1.0); a change to markup or layout only is a patch (v1.0.1); an identical capture adds no version.
• Static and self-contained: scripts, trackers, forms, iframes, external styles and hot-linked images removed. Links that leave the estate are disabled; a link to another archived page points to that page's latest copy instead.
• Banner: each copy opens with the source URL as plain text, the capture date, the version and one "Live website" link, the only live outbound link on the page.
• Websites tab: grouped by site, collapsible; each page is a card with a thumbnail, title, capture date and version, with a "Live website" link directly under the thumbnail. Tapping the card previews the copy inside the Vault.

• Every in-scope page with at least one capture has latest.html, a version file and meta.json, and appears in index.json and the Websites tab.
• No archived copy contains a working link off the estate other than the banner's "Live website" link; no script, iframe or form action survives.
• The landing is minimal with details collapsible, and reads correctly at 390 px wide.
• Neutral, dated wording; no personal data of private individuals; source and capture date shown on every copy.
• Three sample latest.html addresses verified live after each release, with a cache-bust.
• S2: an unchanged recapture adds no version; a changed one adds exactly one, with the correct bump. S3: a link audit before and after, with counts per site.

• W1 — Sites block automated capture (403, bot challenges, 522 outages): capture from the box and with a real browser where allowed; keep Internet Archive copies as seeds; record unreachable pages rather than dropping them.
• W2 — robots.txt: respect disallow rules; record pages excluded for that reason.
• W3 — Upload limits: the upload path refuses requests over about 1 MB; files go one at a time, screenshots as JPEG under 650 KB, commits in small parts.
• W4 — Disk space on the working machine: stream page by page; stop below 700 MB free.
• W5 — Copyright: copies are kept for reference and research, each with its source, capture date and a link to the live page; a rights holder's request is handled by withdrawing the copy.
• W6 — Personal data in third-party pages: private individuals' names, emails and postal addresses redacted before publication.
• W7 — Two archives drifting apart during S2: the internetarchive site stays read-only once S2 starts, and is redirected only after the Vault copy is verified.

• Time: S1 within one working session; S2 and S3 each within two.
• Cost: no spending.
• Scope: no page outside the farm list without a recorded reason; no deletion of internetarchive content before the Vault is verified.
• Quality: zero live outbound links in archived copies other than "Live website".
• Risk: any personal-data exposure stops publication of that page at once.

Outcome (24 Oct 2026). Delivered. Website snapshots: wayback-html (182 pages, design kept). Images: wayback-img (490). PDFs and documents: wayback-pdf (2196). Video: wayback-video (1). Text: wayback-text (611). Old addresses forward: 183 Vault page folders and 1066 Internet Archive capture paths (301). Link check: 7134 archive addresses, 14 landing links and 200 sampled references from other sites, 0 broken.

Scope. Keep each snapshot's design; give every record a short readable address with its file extension; split the Vault by media type into five sites on separate branches; forward every old address.

Products. Design-kept snapshots of the archived farm pages; a readable-name map (old address → new address) generated from the manifests; five landing pages with search and media loaders; the hub page; a redirect table; a crawl report showing zero broken links.

Redirect plan. Old Websites addresses (/websites/<host>/<slug>/latest.html, the folder, latest.htm, vX.Y.Z.html, #websites/… fragments) and the internetarchive site's /<timestamp>/… and /latest/<key>/ paths answer with a permanent redirect (301) to the readable address; fragment links are forwarded by a small script on the hub. Existing media file addresses on the hub keep serving, so nothing already linked breaks; the five sites add readable names on top.

Risks. Cloudflare Pages project count and build quota (checked: room for five more; one deploy per site per change); the 1 MB request limit on repository writes (one file per request, images compressed to about 200 KB, no file over 650 KB); disk space on the build machine (work streamed, caches deleted, stop under 700 MB); some sources block copying of their stylesheets (fallback: the dated archive copy of the stylesheet; otherwise a plain readable layout is kept and noted); links breaking during the move (mitigated by the redirect table and a full crawl before sign-off).

Tolerances. Time: same day. Cost: nil. Quality: crawl must show zero broken links; three sample snapshots checked at 390 px and desktop against the source design.

Rollback. The hub and its branch are changed only by adding redirects and links; reverting those commits, or rolling back the hub's deployment, restores the previous state. The five new sites can be detached without affecting the hub.

• v1.0.0 — 24 October 2026: plan created; S1 started.
• v1.1.0 — 24 October 2026: change C1 added (design-kept snapshots, readable addresses, media split into five sites, redirect plan); stages S4–S6.