jon.recoil.org
A fork of mtelver's day10 project
Add garbage collection design for layers and universes
- Layer GC: aggressive cleanup after each run, delete any layer
not referenced by current solutions
- Universe GC: store universe refs in each package's docs directory
(universes.json), derive live universes from blessed packages,
delete unreferenced universe directories
The universe refs move atomically with the docs via the same .new/.old
swap mechanism, so failed rebuilds keep old universes alive.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
+75
+75
docs/plans/2026-02-03-fresh-docs-design.md
+75
docs/plans/2026-02-03-fresh-docs-design.md
···
218
218
219
219
Logs include stdout, stderr, exit code, timing, and retry attempts.
220
220
221
+
### Garbage Collection
222
+
223
+
GC runs after each successful batch run to clean up stale artifacts.
224
+
225
+
#### Layer GC (Aggressive)
226
+
227
+
Layers in the cache directory become stale when packages update (new opam file → new layer hash). Clean up aggressively since regeneration is fast.
228
+
229
+
After each run:
230
+
1. Collect all layer hashes referenced by current solutions
231
+
2. List all layers in cache directory
232
+
3. Delete any layer not in the referenced set
233
+
234
+
```ocaml
235
+
let gc_layers ~cache_dir ~current_solutions =
236
+
let referenced =
237
+
current_solutions
238
+
|> List.concat_map (fun sol -> sol.layer_hashes)
239
+
|> String.Set.of_list
240
+
in
241
+
let all_layers = Sys.readdir (cache_dir / "layers") in
242
+
Array.iter (fun layer ->
243
+
if not (Set.mem layer referenced) then
244
+
rm_rf (cache_dir / "layers" / layer)
245
+
) all_layers
246
+
```
247
+
248
+
#### Universe GC (Preserve Until Replaced)
249
+
250
+
Universe directories (`html/u/{universe-hash}/...`) contain docs for specific dependency combinations. A universe stays alive as long as at least one blessed package references it.
251
+
252
+
**Universe references stored with package docs:**
253
+
254
+
Each blessed package's docs directory includes a `universes.json` listing which universes it references:
255
+
256
+
```
257
+
html/p/{package}/{version}/
258
+
├── index.html
259
+
├── Pkg_module/index.html
260
+
└── universes.json # {"universes": ["abc123", "def456"]}
261
+
```
262
+
263
+
This file is written during doc generation and moves atomically with the docs (same `.new`/`.old` swap). If a rebuild fails, the old `universes.json` stays in place, keeping old universe references alive.
264
+
265
+
After each run:
266
+
1. Scan all `html/p/*/*/universes.json` files
267
+
2. Collect all referenced universe hashes
268
+
3. Delete any universe directory not referenced by any blessed package
269
+
270
+
```ocaml
271
+
let gc_universes ~html_dir =
272
+
(* Collect all universe refs from all blessed packages *)
273
+
let referenced =
274
+
Glob.find (html_dir / "p" / "*" / "*" / "universes.json")
275
+
|> List.concat_map (fun path ->
276
+
let json = Yojson.Safe.from_file path in
277
+
json |> member "universes" |> to_list |> List.map to_string
278
+
)
279
+
|> String.Set.of_list
280
+
in
281
+
282
+
(* Delete unreferenced universes *)
283
+
Sys.readdir (html_dir / "u")
284
+
|> Array.iter (fun hash ->
285
+
if not (Set.mem hash referenced) then
286
+
rm_rf (html_dir / "u" / hash)
287
+
)
288
+
```
289
+
290
+
Benefits:
291
+
- Universe refs move atomically with the docs (same swap mechanism)
292
+
- Failed rebuild keeps old `universes.json`, so old universes stay alive
293
+
- No separate manifest that could get out of sync
294
+
- Truth derived from actual docs structure
295
+
221
296
## Implementation Changes Required
222
297
223
298
### day10 Core