Story 026: a UAT archive instance (#139). Purpose uat at slot 1, bulk writers allowed; the Taskfile works on UAT, production only by promote

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
hum3 2026-11-04 21:08:39 +00:00
commit 4dceb5fda9
17 changed files with 156 additions and 70 deletions

View file

@ -1,5 +1,5 @@
# Copy to .env (gitignored). Both are optional.
# The archive (or mirror) task build and task serve read. Default: ~/.local/share/bytestone/archive
#BYTESTONE_ARCHIVE=/home/me/.local/share/bytestone/archive
# The archive (or mirror) task build and task serve read. Default: ~/.local/share/bytestone/uat, the mirror of UAT (story 026)
#BYTESTONE_ARCHIVE=/home/me/.local/share/bytestone/uat
# Where task serve listens. Default: localhost:8321+slot, the instance's slot address
#BYTESTONE_ADDR=localhost:8321

View file

@ -125,14 +125,16 @@ are to items the same person holds; another's is shown as a bare id.
| Brand page | `docs/brand.html`: the palette and ramp, the type, the stone, tiles and the day's margin as the build draws them (`life-weaver brand DIR`) |
| Archive | The primary store: a directory per person ([docs/archive.md](docs/archive.md)) |
| Archive instance | One archive tree, named by its `archive.yaml` (`name`, `purpose`, `slot`); `build` and `serve` report which they are on |
| Slot | An instance's place among the ten that may serve at once, 0 to 9, and so its address: `serve` listens on `localhost:8321+slot`. Production is slot 0, so its clients default to `localhost:8321`; a scratch or scale instance names its own |
| Production | The one instance on the NAS, `purpose: production`; bulk writers refuse it |
| Slot | An instance's place among the ten that may serve at once, 0 to 9, and so its address: `serve` listens on `localhost:8321+slot`. Production is slot 0, so its clients default to `localhost:8321`, and UAT is slot 1, said or not; a scratch or scale instance names its own, 2 to 9 |
| Production | The archive of record, on the NAS, `purpose: production`; bulk writers refuse it, and nothing writes it but a promotion |
| UAT | The staging instance beside production on the NAS, `purpose: uat`, slot 1: a full copy, people and items, that every depositor writes and i.bytestone.uk is built from; bulk writers may write it (story 026) |
| Promotion | `task promote`: production becomes exactly what UAT is now, tree and site |
| Item | One archived thing with its own id, kept as a bag in its home person's directory |
| Kind | What an item is: a *photo*, a print scanned here with `photo.yaml` beside it, or a *page*, with `page.yaml`. Read off the bag; the receipt names it |
| Page | A file received from elsewhere (a copy emailed, a phone photograph of prints), kept unaltered as an item of its own; the prints on it are photos cut from it, as from a bed scan |
| Cut from | A photo's tie to the page it was cut from and where on it the print lay; the page lists its photos |
| Supersedes | A later scan's tie to the copy of the same print it replaces; the site shows the scan in the copy's place and the copy stays |
| Mirror | The laptop's copy of the archive's people tree without items and renditions, which the build reads |
| Mirror | The laptop's copy of UAT's people tree without items and renditions, which the build reads |
| Site | The static output: `/` the front page, `/<id>/` page, `/<id>/<file>` downloads, `/<id>/<name>/` document pages |
| Front page | The featured life with their family as the hourglass chart, the count of lives, the slogan and the search (stories 006, 020) |
| Featured life | The person the front page opens with: the best-recorded (portrait and bio) with the most kin, else the best-connected |
@ -167,25 +169,29 @@ page there (story 025).
## Usage
```sh
task archive:pull # NAS → mirror (~/.local/share/bytestone/archive): archive.yaml and the people tree, less items and renditions
task uat:seed # once: UAT on the NAS copied from production entire, named uat
task archive:pull # UAT on the NAS → mirror (~/.local/share/bytestone/uat): archive.yaml and the people tree, less items and renditions
task person:pull ID=0 # one person entire, items included
task build # mirror → site/, then the front page's wasm (task wasm)
task archive:push # mirror → NAS (adds and updates; never deletes on the NAS)
task archive:push # mirror → UAT on the NAS (adds and updates; never deletes on the NAS)
task person:push ID=0 # one person entire
task serve # the archive service's API over the mirror (below)
task preview # preview site/ locally
task publish # mirror → NAS, NAS → mirror, then docs and site to their hosts (tp pages deploy)
task promote # i.bytestone.org becomes what i.bytestone.uk serves now
task publish # mirror → UAT, UAT → mirror, then docs and site to their hosts (tp pages deploy)
task promote:diff # what promote would change in production, deletions included
task promote # production becomes what UAT is now: the tree on the NAS, and i.bytestone.org what i.bytestone.uk serves
task check # fmt, vet, lint, test
task perf # the search ladder: GET /people?q= at each scale of register
```
Edit a record in the mirror, then `task archive:push` or `person:push`.
`task publish` brings the mirror and the NAS level, builds and sends
`site/` and `docs/` to their hosts. The site it sends is i.bytestone.uk,
where a publish is looked at; `task promote` then copies it, as served,
over i.bytestone.org, the production site of lives, on the host itself.
Production changes only by a promotion.
The mirror is UAT's. Edit a record in it, then `task archive:push` or
`person:push`. `task publish` brings the mirror and UAT on the NAS level,
builds and sends `site/` and `docs/` to their hosts. The site it sends is
i.bytestone.uk, where a publish is looked at; `task promote` then makes
production what UAT is, the tree on the NAS, deletions included, and the
site as served copied over i.bytestone.org, the production site of
lives, on the host itself. Production changes only by a promotion, and
`task promote:diff` shows what one would do.
`.env` (gitignored; `.env.example`) may set `BYTESTONE_ARCHIVE`, the
archive the build and the service read, and `BYTESTONE_ADDR`, where the
@ -196,8 +202,8 @@ Every archive instance names itself in `archive.yaml` at its root, and
```yaml
name: bytestone
purpose: production # or scratch, or scale
slot: 0 # production's; a scratch or scale instance takes 1 to 9
purpose: production # or uat, scratch or scale
slot: 0 # production's; uat is 1; a scratch or scale instance takes 2 to 9
```
`serve` listens on `localhost:8321+slot` unless `-addr` says otherwise,
@ -207,11 +213,11 @@ refuses, naming what has the address. So several instances serve at once,
each where its slot puts it, and a scratch run cannot take production's
address by accident.
The mirror is production's copy and carries production's file, so it is
production for this purpose. Anything that writes in bulk (a synthetic
generator, a mass import) calls `archive.Load(root)` and then
`Instance.Bulk()`, which refuses `production`; a scratch or scale
instance is any other tree with its own file.
Anything that writes in bulk (a synthetic generator, a mass import)
calls `archive.Load(root)` and then `Instance.Bulk()`, which refuses
`production`. UAT takes bulk writes: that is what it is for, and the
mirror is UAT's, carrying UAT's file. A scratch or scale instance is any
other tree with its own file.
## The archive service

View file

@ -4,17 +4,23 @@ version: '3'
dotenv: ['.env']
vars:
# The bytestone archive: the primary store, on the NAS. See docs/archive.md.
# The bytestone archive, on the NAS (docs/archive.md). PRODUCTION is the
# archive of record, written only by task promote; ARCHIVE is UAT, the
# staging copy every depositor writes and the site is built from (story
# 026), which the archive tasks work on.
NAS: humphrey@10.0.0.1
NAS_PORT: 5522
ARCHIVE: /volume1/H3Archive/bytestone
# The archive the build and the service read: the laptop's mirror unless .env says otherwise.
MIRROR: '{{.BYTESTONE_ARCHIVE | default (printf "%s/.local/share/bytestone/archive" .HOME)}}'
PRODUCTION: /volume1/H3Archive/bytestone
ARCHIVE: /volume1/H3Archive/bytestone-uat
# The archive the build and the service read: the laptop's mirror of UAT unless .env says otherwise.
MIRROR: '{{.BYTESTONE_ARCHIVE | default (printf "%s/.local/share/bytestone/uat" .HOME)}}'
# Where task serve listens.
ADDR: '{{if .BYTESTONE_ADDR}}-addr {{.BYTESTONE_ADDR}}{{end}}'
RSYNC: rsync -az --chmod=D755,F644 -e "ssh -p {{.NAS_PORT}} -o BatchMode=yes"
# What the mirror leaves on the NAS.
BULK: --exclude items/ --exclude renditions/
# What promotion leaves alone: each instance keeps its own archive.yaml, and Synology's index directories stay where they are.
PROMOTE: --exclude archive.yaml --exclude @eaDir
tasks:
fmt:
@ -46,14 +52,23 @@ tasks:
cmds:
- go test -run '^$' -bench Search -benchmem ./internal/archive/
uat:seed:
desc: 'Seed UAT on the NAS from production, once: the whole tree copied, people and items, and its archive.yaml made UAT''s; refused when the UAT tree exists (story 026)'
cmds:
- |
ssh -p {{.NAS_PORT}} -o BatchMode=yes {{.NAS}} 'set -e
if [ -e {{.ARCHIVE}} ]; then echo "{{.ARCHIVE}} exists: UAT is seeded once, from production" >&2; exit 1; fi
cp -a {{.PRODUCTION}} {{.ARCHIVE}}
printf "name: bytestone-uat\npurpose: uat\nslot: 1\n" > {{.ARCHIVE}}/archive.yaml
cat {{.ARCHIVE}}/archive.yaml'
archive:pull:
desc: Mirror the archive from the NAS (archive.yaml, records and files; not items or renditions), the NAS winning
desc: Mirror UAT from the NAS (archive.yaml, records and files; not items or renditions), the NAS winning
cmds:
- mkdir -p {{.MIRROR}}/people
- '{{.RSYNC}} {{.NAS}}:{{.ARCHIVE}}/archive.yaml {{.MIRROR}}/archive.yaml'
- '{{.RSYNC}} --delete {{.BULK}} {{.NAS}}:{{.ARCHIVE}}/people/ {{.MIRROR}}/people/'
archive:push:
desc: Send the mirror's archive.yaml and people tree to the NAS (adds and updates; never deletes there)
desc: Send the mirror's archive.yaml and people tree to UAT on the NAS (adds and updates; never deletes there)
cmds:
- 'ssh -p {{.NAS_PORT}} {{.NAS}} mkdir -p {{.ARCHIVE}}/people'
- '{{.RSYNC}} {{.MIRROR}}/archive.yaml {{.NAS}}:{{.ARCHIVE}}/archive.yaml'
@ -93,14 +108,19 @@ tasks:
- GOOS=js GOARCH=wasm go build -ldflags=-s -trimpath -o site/kinship.wasm ./cmd/kinship
- cp "$(go env GOROOT)/lib/wasm/wasm_exec.js" site/
publish:
desc: 'Publish: mirror → NAS and NAS → mirror (archive:push, archive:pull), then build and send docs/ and site/ to their hosts (tp pages deploy); the stage-1 form of #41'
desc: 'Publish: the mirror → UAT on the NAS and back (archive:push, archive:pull), then build and send docs/ and site/ to their hosts (tp pages deploy): i.bytestone.uk is UAT; the stage-1 form of #41'
cmds:
- task: archive:push
- task: archive:pull
- task-plus pages deploy
promote:
desc: 'Promote: i.bytestone.org, the production site of lives, becomes what i.bytestone.uk serves now; a copy made on the host, no build (#103)'
promote:diff:
desc: What promote would change in production on the NAS (deletions included); nothing is written
cmds:
- ssh -p {{.NAS_PORT}} -o BatchMode=yes {{.NAS}} 'rsync -n -i -a --delete {{.PROMOTE}} {{.ARCHIVE}}/ {{.PRODUCTION}}/'
promote:
desc: 'Promote: production becomes what UAT is now. The UAT tree is copied over production''s on the NAS, people and items, deletions included (run promote:diff first), and i.bytestone.org becomes what i.bytestone.uk serves, a copy made on the host, no build (#103, story 026). The only task that writes production'
cmds:
- ssh -p {{.NAS_PORT}} -o BatchMode=yes {{.NAS}} 'rsync -a --delete {{.PROMOTE}} {{.ARCHIVE}}/ {{.PRODUCTION}}/'
- ssh -o BatchMode=yes deploy@docs.bytestone.uk 'rsync -a --delete /srv/sites/i/ /srv/sites/i-org/'
serve:
desc: Run the archive service's API over the archive (BYTESTONE_ARCHIVE, BYTESTONE_ADDR from .env)

View file

@ -1,9 +1,9 @@
// Package archive describes an archive instance: one tree laid out as
// docs/archive.md says, which names itself in archive.yaml at its root.
// Several instances exist, production on the NAS, scratch copies and a
// scale instance for large imports, and the file is what tells them
// apart, so a bulk writer refuses production by asking the tree rather
// than trusting the operator's flag.
// Several instances exist, production on the NAS, UAT beside it (story
// 026), scratch copies and a scale instance for large imports, and the
// file is what tells them apart, so a bulk writer refuses production by
// asking the tree rather than trusting the operator's flag.
package archive
import (
@ -22,7 +22,8 @@ const File = "archive.yaml"
type Purpose string
const (
Production Purpose = "production" // the one instance on the NAS
Production Purpose = "production" // the archive of record, on the NAS
UAT Purpose = "uat" // the staging copy every depositor writes; production is promoted from it
Scratch Purpose = "scratch" // a copy to try things on
Scale Purpose = "scale" // large imports and performance tests
)
@ -32,15 +33,18 @@ type Instance struct {
Name string `yaml:"name"`
Purpose Purpose `yaml:"purpose"`
// Slot is the instance's place among the Slots that may serve at
// once, and so its address: production is slot 0, said or not; a
// scratch or scale instance says its own, 1 to Slots-1.
// once, and so its address: production is slot 0 and UAT is UATSlot,
// said or not; a scratch or scale instance says its own, UATSlot+1 to
// Slots-1.
Slot int `yaml:"slot"`
}
// BasePort is slot 0's port, production's, which the archive's clients
// default to; slot n serves on BasePort+n. Slots is how many there are.
// default to; slot n serves on BasePort+n. UATSlot is UAT's, so its
// clients find it without reading a tree. Slots is how many there are.
const (
BasePort = 8321
UATSlot = 1
Slots = 10
)
@ -62,27 +66,47 @@ func Load(root string) (Instance, error) {
}
return Instance{}, err
}
var in Instance
if err := yaml.Unmarshal(data, &in); err != nil {
var file struct {
Name string `yaml:"name"`
Purpose Purpose `yaml:"purpose"`
Slot *int `yaml:"slot"` // nil when the file does not say
}
if err := yaml.Unmarshal(data, &file); err != nil {
return Instance{}, fmt.Errorf("%s: %w", path, err)
}
in := Instance{Name: file.Name, Purpose: file.Purpose}
if in.Name == "" {
return Instance{}, fmt.Errorf("%s: no name", path)
}
named := fmt.Sprintf("%s, %s, %s or %s", Production, UAT, Scratch, Scale)
switch in.Purpose {
case Production, Scratch, Scale:
case Production, UAT, Scratch, Scale:
case "":
return Instance{}, fmt.Errorf("%s: no purpose (%s, %s or %s)", path, Production, Scratch, Scale)
return Instance{}, fmt.Errorf("%s: no purpose (%s)", path, named)
default:
return Instance{}, fmt.Errorf("%s: purpose %q is not %s, %s or %s", path, in.Purpose, Production, Scratch, Scale)
return Instance{}, fmt.Errorf("%s: purpose %q is not %s", path, in.Purpose, named)
}
// Production and UAT have fixed slots, said or not; the others name
// their own among the rest.
fixed := map[Purpose]int{Production: 0, UAT: UATSlot}
if slot, ok := fixed[in.Purpose]; ok {
if file.Slot != nil && *file.Slot != slot {
return Instance{}, fmt.Errorf("%s: %s is slot %d, not %d", path, in.Purpose, slot, *file.Slot)
}
in.Slot = slot
return in, nil
}
if file.Slot == nil {
return Instance{}, fmt.Errorf("%s: a %s instance needs a slot of its own, %d to %d (slot 0 is production's, %d is UAT's)", path, in.Purpose, UATSlot+1, Slots-1, UATSlot)
}
in.Slot = *file.Slot
switch {
case in.Purpose == Production && in.Slot != 0:
return Instance{}, fmt.Errorf("%s: production is slot 0, not %d", path, in.Slot)
case in.Purpose != Production && in.Slot == 0:
return Instance{}, fmt.Errorf("%s: a %s instance needs a slot of its own, 1 to %d (slot 0 is production's)", path, in.Purpose, Slots-1)
case in.Slot == 0:
return Instance{}, fmt.Errorf("%s: slot 0 is production's; a %s instance takes %d to %d", path, in.Purpose, UATSlot+1, Slots-1)
case in.Slot == UATSlot:
return Instance{}, fmt.Errorf("%s: slot %d is UAT's; a %s instance takes %d to %d", path, UATSlot, in.Purpose, UATSlot+1, Slots-1)
case in.Slot < 0 || in.Slot >= Slots:
return Instance{}, fmt.Errorf("%s: slot %d is not 0 to %d", path, in.Slot, Slots-1)
return Instance{}, fmt.Errorf("%s: slot %d is not %d to %d", path, in.Slot, UATSlot+1, Slots-1)
}
return in, nil
}
@ -92,6 +116,7 @@ func (in Instance) String() string { return fmt.Sprintf("%s (%s)", in.Name, in.P
// Bulk is what a bulk writer (a synthetic generator, a mass import) asks
// before it starts: ErrProduction, naming the instance, for production.
// UAT takes bulk writes: that is what it is for (story 026).
func (in Instance) Bulk() error {
if in.Purpose == Production {
return fmt.Errorf("%s: %w", in, ErrProduction)

View file

@ -13,8 +13,10 @@ func TestLoad(t *testing.T) {
want Instance
}{
{"production", Instance{Name: "bytestone", Purpose: Production}}, // slot 0, said or not
{"scratch", Instance{Name: "nas-copy", Purpose: Scratch, Slot: 1}},
{"scratch", Instance{Name: "nas-copy", Purpose: Scratch, Slot: 2}},
{"scale", Instance{Name: "ladder", Purpose: Scale, Slot: 2}},
{"uat", Instance{Name: "bytestone-uat", Purpose: UAT, Slot: 1}}, // story 026: UAT is slot 1
{"uatnoslot", Instance{Name: "bytestone-uat", Purpose: UAT, Slot: 1}}, // said or not
}
for _, c := range cases {
got, err := Load(filepath.Join("testdata", c.dir))
@ -39,6 +41,9 @@ func TestLoadRefuses(t *testing.T) {
{"noslot", "slot"}, // #71: a scratch or scale instance needs a slot of its own
{"badslot", "slot"}, // 12 is past the ten
{"prodslot", "slot 0"}, // production is slot 0
{"uatslot0", "slot 1"}, // story 026: UAT is slot 1
{"uatslot2", "slot 1"},
{"scaleslot1", "UAT"}, // slot 1 is UAT's; scratch and scale take 2 to 9
}
for _, c := range cases {
_, err := Load(filepath.Join("testdata", c.dir))
@ -61,7 +66,7 @@ func TestBulk(t *testing.T) {
if err == nil || !strings.Contains(err.Error(), "bytestone") {
t.Errorf("production: %v does not name the instance", err)
}
for _, p := range []Purpose{Scratch, Scale} {
for _, p := range []Purpose{Scratch, Scale, UAT} {
if err := (Instance{Name: "x", Purpose: p}).Bulk(); err != nil {
t.Errorf("%s: %v", p, err)
}
@ -77,4 +82,7 @@ func TestAddrFollowsTheSlot(t *testing.T) {
if got := (Instance{Name: "h3", Purpose: Scale, Slot: 2}).Addr(); got != "localhost:8323" {
t.Errorf("slot 2: %s", got)
}
if got := (Instance{Name: "bytestone-uat", Purpose: UAT, Slot: 1}).Addr(); got != "localhost:8322" {
t.Errorf("UAT: %s", got)
}
}

View file

@ -0,0 +1,3 @@
name: x
purpose: scale
slot: 1

View file

@ -1,3 +1,3 @@
name: nas-copy
purpose: scratch
slot: 1
slot: 2

3
archive/testdata/uat/archive.yaml vendored Normal file
View file

@ -0,0 +1,3 @@
name: bytestone-uat
purpose: uat
slot: 1

View file

@ -0,0 +1,2 @@
name: bytestone-uat
purpose: uat

View file

@ -0,0 +1,3 @@
name: x
purpose: uat
slot: 0

View file

@ -0,0 +1,3 @@
name: x
purpose: uat
slot: 2

1
changes/139.added.md Normal file
View file

@ -0,0 +1 @@
Story 026, a UAT archive instance (#139): a fourth purpose, `uat`, at slot 1 as production is slot 0, said or not, that bulk writers may write; scratch and scale instances take 2 to 9. UAT is a full copy of production on the NAS beside it, people and items, seeded once by `task uat:seed`, mirrored to the laptop at `~/.local/share/bytestone/uat`, and what every depositor writes and i.bytestone.uk is built from: `archive:pull`, `archive:push`, `build`, `serve` and `publish` work on it. Production becomes what `task promote` makes it, exactly UAT, the tree with deletions after `task promote:diff` and the site as before; the only task that writes production

View file

@ -46,7 +46,7 @@ func writeBag(t *testing.T, root, id string, payload map[string]string) string {
// not sound; it is the monthly fixity run.
func TestFixityAuditsEveryHeldItem(t *testing.T) {
root := t.TempDir()
write(t, filepath.Join(root, "archive.yaml"), []byte("name: scratch\npurpose: scratch\nslot: 1\n"))
write(t, filepath.Join(root, "archive.yaml"), []byte("name: scratch\npurpose: scratch\nslot: 2\n"))
writeBag(t, root, "0", map[string]string{"front.tif": "II*\x00one", "photo.yaml": "batch: 1\n"})
rotten := writeBag(t, root, "1", map[string]string{"front.tif": "II*\x00two"})
if err := os.MkdirAll(filepath.Join(root, "people/0/00/items/2.partial"), 0o755); err != nil {

View file

@ -13,7 +13,7 @@ import (
// naming the squatter; something that is not life-weaver, refuse as
// taken.
func TestServeRefusesAnAddressAlreadyServing(t *testing.T) {
h3 := archive.Instance{Name: "h3", Purpose: archive.Scale, Slot: 1}
h3 := archive.Instance{Name: "h3", Purpose: archive.Scale, Slot: 2}
geni := archive.Instance{Name: "geni", Purpose: archive.Scale, Slot: 2}
for _, c := range []struct {
there archive.Instance

View file

@ -10,15 +10,17 @@ builder. Proposed 2026-10-24; stage 1 is life-weaver#22.
| Term | Meaning |
|---|---|
| Archive | The primary store: one directory tree laid out as below |
| Archive instance | One such tree, named by `archive.yaml` at its root (`name`, `purpose`: `production`, `scratch` or `scale`, `slot`: 0 for production, 1 to 9 for the others, which puts the service on `localhost:8321+slot`); the `archive` package reads it, `build` and `serve` report it |
| Production | The one instance on the NAS. A bulk writer (a synthetic generator, a mass import) asks the instance first and refuses production |
| Archive instance | One such tree, named by `archive.yaml` at its root (`name`, `purpose`: `production`, `uat`, `scratch` or `scale`, `slot`: 0 for production, 1 for UAT, 2 to 9 for the others, which puts the service on `localhost:8321+slot`); the `archive` package reads it, `build` and `serve` report it |
| Production | The archive of record, on the NAS. A bulk writer (a synthetic generator, a mass import) asks the instance first and refuses production; nothing writes production but a promotion |
| UAT | The staging instance beside production on the NAS (story 026): a full copy, people and items, seeded once from production, that every depositor writes and i.bytestone.uk is built from. Bulk writers may write it |
| Promotion | `task promote`: production becomes exactly what UAT is now, the tree copied over with deletions and the site copied over i.bytestone.org; `task promote:diff` lists the changes first |
| Register | The person records, `people/<dir>/person.yaml` (README) |
| Person directory | Everything of one person under `people/<dir>/`, found from the id; the unit of work, sync, publish and audit |
| Files | What a record names, under the person's `files/`; the lecture's images and outputs are Victoria Drummond's |
| Item | One archived thing with its own id, a BagIt bag in its home person's `items/<item-id>/` (life-weaver#19; the format statement). Its kind is read off the bag: a *photo* (`photo.yaml`) or a *page* (`page.yaml`: a file received from elsewhere, kept as sent; the prints on it are photos cut from it, #72) |
| Rendition | A derived access copy of an item, under the person's `renditions/<item>/<recipe>/`: `display.jpg`, `thumb.jpg`, `tiny.jpg`, `recipe.yaml`, the back's under `back/`. Made by the depositor at a recipe version and carried with the bag (#39); rebuildable, not preserved |
| Derived | Outputs of a per-person activity (maps, thumbnails) under `derived/`, each with its recipe version; never regenerated by a code release |
| Mirror | The laptop's copy of production's `archive.yaml` and people tree without `items/` and `renditions/`, the build's input until the build runs on the NAS; it carries production's file, so it is production to a bulk writer |
| Mirror | The laptop's copy of UAT's `archive.yaml` and people tree without `items/` and `renditions/`, the build's input until the build runs on the NAS; it carries UAT's file, so it takes bulk writes as UAT does |
| Site | The read copy the public sees, built from the archive and rsynced to its host |
## Endpoint and layout
@ -26,13 +28,13 @@ builder. Proposed 2026-10-24; stage 1 is life-weaver#22.
| | |
|---|---|
| Host | `humphrey@10.0.0.1`, SSH port 5522 (the Synology NAS, HDTLServer) |
| Root | `/volume1/H3Archive/bytestone` |
| Root | `/volume1/H3Archive/bytestone` (production); `/volume1/H3Archive/bytestone-uat` (UAT) |
| Transport | rsync over SSH. DSM must have the rsync service enabled for the account, or its rsync answers *Permission denied* after a successful login |
| Capacity | volume1 has 5.5 TB free; the layout is meant to hold 1–2 TB before it is restructured |
| Capacity | volume1 has 5.5 TB free; the layout is meant to hold 1–2 TB before it is restructured, twice over now that UAT is a full copy |
```
bytestone/
archive.yaml name: bytestone, purpose: production (#36)
bytestone/ production; bytestone-uat/ is laid out the same
archive.yaml name: bytestone, purpose: production (#36); name: bytestone-uat, purpose: uat, slot: 1 in UAT (story 026)
people/<dir>/
person.yaml the record; its file paths are relative to this directory
files/ what the record names
@ -49,9 +51,9 @@ holds more than 1024 entries, a directory is either a shard or a person,
and nothing caps the id's length. `pid.Dir` and `life-weaver dir` compute
it.
The Taskfile's `NAS`, `ARCHIVE` and `MIRROR` vars are the one place the
endpoint is written. Moving the archive to a dedicated share later is a
`mv` on the NAS and a change to `ARCHIVE`.
The Taskfile's `NAS`, `PRODUCTION`, `ARCHIVE` (UAT) and `MIRROR` vars are
the one place the endpoint is written. Moving the archive to a dedicated
share later is a `mv` on the NAS and a change to those.
## Flow
@ -62,7 +64,7 @@ depositors: Depositors {
lecture: victoria-drummond-lecture
laptop: records edited on the laptop
}
archive: Archive, on the NAS {
uat: UAT, on the NAS {
person: "people/<dir>/" {
record: person.yaml
files
@ -71,13 +73,19 @@ archive: Archive, on the NAS {
derived
}
}
production: Production, on the NAS
site: i.bytestone.uk
depositors -> archive: "stage 1: rsync over SSH\nstage 2: the archive service's API"
archive -> site: "stage 1: pull to the mirror, build, tp pages deploy\nstage 3: build on the NAS, rsync out"
prod_site: i.bytestone.org
depositors -> uat: "stage 1: rsync over SSH\nstage 2: the archive service's API"
uat -> site: "stage 1: pull to the mirror, build, tp pages deploy\nstage 3: build on the NAS, rsync out"
uat -> production: "task promote: the tree, with deletions"
site -> prod_site: "task promote: the site as served"
```
Writes go from depositors to the archive; publishing goes from the archive
to the site. Nothing goes from the site back.
Writes go from depositors to UAT; publishing goes from UAT to
i.bytestone.uk; a promotion makes production and i.bytestone.org what
UAT and its site are. Nothing goes from a site back, and nothing writes
production but the promotion.
## Stages
@ -227,4 +235,8 @@ not to look up; the receipt stays the cheap call. bytestone-scan's
| Where the build runs | Stage 1: the laptop, from the mirror. Stage 3: the NAS | Always the laptop | At 1–2 TB the published files cannot pass through a laptop. The site build copies every published file into `site/`; on the NAS (Btrfs) that copy can be a reflink |
| Who makes renditions | The depositor, at its recipe version; the service refuses a deposit without them and places them beside the bag | The archive develops them from the master and `photo.yaml`, with the recipe in a package both import | The scanner has the recipe and the operator's eye on the result already; the archive stays an image-agnostic store that holds, checks and serves. Cost: non-scanner photographs (`files/`) need their own thumbnail route (#3), and re-development at a new recipe (bytestone-scan#18) is a depositor's job over the archive |
| A tree with no `archive.yaml` | Refused by `build`, `serve` and the loader | Assumed `scratch` | The guard must not be defeatable by forgetting the file; the mirror gets its copy from `archive:pull`, and a scratch or scale tree is made by writing its file first |
| What UAT is (story 026) | A full archive instance of its own on the NAS, items included, mirrored to the laptop as production was | A site-only stage built from production's archive, as before | A bulk load has to be an archive before it is a site, and the loader must never write production; the scanner's items will go to UAT too. Cost: a second copy of the data on volume1 |
| What promotion does | Production becomes exactly UAT: copied with deletion, each instance keeping its own `archive.yaml`, after `promote:diff` | Adds and updates only, as `archive:push` | A record withdrawn from UAT (a living person, a suspect tie) must not outlive it in production; exactness is what makes "reviewed in UAT" mean anything. The dry run and a human running it are the guard |
| Slots after UAT | UAT is slot 1, fixed, as production is 0 | Any of 1 to 9 | Clients find UAT without reading a tree; scratch and scale instances take 2 to 9 |
| Seeding UAT | Once, from production, refused thereafter | Copying production to UAT on demand | After the seed nothing reaches production that did not come through UAT; a reverse copy would hide that |
| Records' history | Not kept yet | A git repo in `people/`, as the format statement keeps bag sidecars | Needed, but the service of stage 2 decides who commits; not before |

View file

@ -1,6 +1,6 @@
# 026 — A UAT archive instance (#139)
**Status:** proposed
**Status:** done locally
**Roadmap:** none — the archive's stage 2 (docs/archive.md) gains a staging instance; the bytestone-loader loads and bytestone-loader #2 depend on it
## Why

View file

@ -1,3 +1,3 @@
name: fixture
purpose: scratch
slot: 1
slot: 2