jon.recoil.org
forked from
anil.recoil.org/monopam-myspace
My aggregated monorepo of OCaml code, automaintained
Add implementation plan: day10 uses jtw opam for per-package artifacts
6 tasks: add findlib name helper, replace container script, remove
post-container copying from linux.ml, delete dead code, e2e test, cleanup.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
+274
+274
docs/plans/2026-02-20-day10-use-jtw-opam-plan.md
+274
docs/plans/2026-02-20-day10-use-jtw-opam-plan.md
···
1
+
# day10 uses `jtw opam` for per-package artifacts — Implementation Plan
2
+
3
+
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
4
+
5
+
**Goal:** Eliminate code duplication by having day10's container script call `jtw opam` instead of reimplementing artifact generation.
6
+
7
+
**Architecture:** Replace `jtw_container_script` (which generates raw `js_of_ocaml compile` commands) with a `jtw opam` invocation. Remove post-container `.cmi`/META copying and `dynamic_cmis.json` generation from `linux.ml` since `jtw` now does all of this inside the container. Delete dead code from `jtw_gen.ml`.
8
+
9
+
**Tech Stack:** OCaml, dune, day10 container infrastructure, jtw CLI
10
+
11
+
**Design doc:** `docs/plans/2026-02-20-day10-use-jtw-opam-design.md`
12
+
13
+
---
14
+
15
+
### Task 1: Derive findlib package names from installed_libs
16
+
17
+
`jtw opam` takes findlib package names (positional args). day10 has `installed_libs` — a list of relative paths like `hmap/hmap.cmi`, `hmap/META`. We need to extract findlib package names from these paths.
18
+
19
+
**Files:**
20
+
- Modify: `day10/bin/jtw_gen.ml`
21
+
22
+
**Step 1: Add `findlib_names_of_installed_libs` function**
23
+
24
+
Add after the existing `jtw_result_to_yojson` at the end of `jtw_gen.ml`:
25
+
26
+
```ocaml
27
+
(** Extract top-level findlib package names from installed lib file paths.
28
+
Each path is relative to lib/ (e.g., "hmap/hmap.cmi", "hmap/META").
29
+
A findlib package is identified by the presence of a META file.
30
+
Returns deduplicated, sorted list of findlib directory names containing META. *)
31
+
let findlib_names_of_installed_libs installed_libs =
32
+
List.filter_map (fun rel_path ->
33
+
if Filename.basename rel_path = "META" then
34
+
Some (Filename.dirname rel_path)
35
+
else None
36
+
) installed_libs
37
+
|> List.sort_uniq String.compare
38
+
```
39
+
40
+
**Step 2: Build and verify compilation**
41
+
42
+
Run: `dune build day10/bin/main.exe 2>&1 | head -20`
43
+
Expected: Compiles successfully (function is unused for now, but no errors)
44
+
45
+
**Step 3: Commit**
46
+
47
+
```bash
48
+
git add day10/bin/jtw_gen.ml
49
+
git commit -m "day10: add findlib_names_of_installed_libs helper"
50
+
```
51
+
52
+
---
53
+
54
+
### Task 2: Replace `jtw_container_script` with `jtw opam` invocation
55
+
56
+
The current `jtw_container_script` filters `installed_libs` for `.cma` files and generates raw `js_of_ocaml compile` shell commands. Replace with a `jtw opam` call.
57
+
58
+
**Files:**
59
+
- Modify: `day10/bin/jtw_gen.ml:127-152` (replace `jtw_container_script`)
60
+
61
+
**Step 1: Rewrite `jtw_container_script`**
62
+
63
+
Replace the existing function with:
64
+
65
+
```ocaml
66
+
(** Build the shell script to run inside the container for jtw generation.
67
+
Calls `jtw opam` to handle all per-package artifact generation:
68
+
.cmi copying, .cma -> .cma.js compilation, dynamic_cmis.json, findlib_index.json. *)
69
+
let jtw_container_script ~pkg ~installed_libs =
70
+
let pkg_name = OpamPackage.name_to_string pkg in
71
+
let findlib_names = findlib_names_of_installed_libs installed_libs in
72
+
if findlib_names = [] then
73
+
"true"
74
+
else
75
+
let libs = String.concat " " (List.map Filename.quote findlib_names) in
76
+
String.concat " && " [
77
+
"eval $(opam env)";
78
+
Printf.sprintf "echo 'JTW: Building %s via jtw opam (%d findlib packages)'" pkg_name (List.length findlib_names);
79
+
Printf.sprintf "jtw opam --path %s --no-worker -o /home/opam/jtw-output %s"
80
+
(Filename.quote pkg_name) libs;
81
+
"echo 'JTW: Done'";
82
+
]
83
+
```
84
+
85
+
**Step 2: Delete `jsoo_compile_command`**
86
+
87
+
Delete `jsoo_compile_command` (lines 118-123) — no longer referenced.
88
+
89
+
**Step 3: Build and verify**
90
+
91
+
Run: `dune build day10/bin/main.exe 2>&1 | head -20`
92
+
Expected: Compiles. If `jsoo_compile_command` is referenced elsewhere, grep first:
93
+
`grep -r jsoo_compile_command day10/`
94
+
95
+
**Step 4: Commit**
96
+
97
+
```bash
98
+
git add day10/bin/jtw_gen.ml
99
+
git commit -m "day10: replace jtw_container_script with jtw opam invocation"
100
+
```
101
+
102
+
---
103
+
104
+
### Task 3: Remove post-container .cmi/META copying and dynamic_cmis.json generation from linux.ml
105
+
106
+
Currently `linux.ml:run_jtw_in_container` does three things after the container exits:
107
+
1. Copies `.cma.js` output from container (lines 715-723) — **keep**, jtw writes to container output dir
108
+
2. Copies `.cmi` and `META` from build layer (lines 724-735) — **delete**, jtw now does this inside container
109
+
3. Generates `dynamic_cmis.json` outside container (lines 736-759) — **delete**, jtw now does this inside container
110
+
111
+
**Files:**
112
+
- Modify: `day10/bin/linux.ml:714-759`
113
+
114
+
**Step 1: Delete post-container .cmi/META copying (lines 724-735)**
115
+
116
+
Remove the block:
117
+
```ocaml
118
+
(* Also copy .cmi and META from the build layer to the jtw layer *)
119
+
let build_lib = Path.(build_layer_dir / "fs" / "home" / "opam" / ".opam" / "default" / "lib") in
120
+
List.iter (fun rel_path ->
121
+
...
122
+
) installed_libs;
123
+
```
124
+
125
+
**Step 2: Delete post-container dynamic_cmis.json generation (lines 736-759)**
126
+
127
+
Remove the block:
128
+
```ocaml
129
+
(* Generate dynamic_cmis.json for each lib subdirectory that has .cmi files *)
130
+
let jtw_lib_dir = Path.(jtw_layer_dir / "lib") in
131
+
if Sys.file_exists jtw_lib_dir then begin
132
+
let rec scan_dirs base rel =
133
+
...
134
+
scan_dirs jtw_lib_dir ""
135
+
end;
136
+
```
137
+
138
+
**Step 3: Update the doc comment for `run_jtw_in_container` (line 655)**
139
+
140
+
Change from:
141
+
```
142
+
(** Run jtw generation in a container: compile .cma -> .cma.js, copy .cmi, META *)
143
+
```
144
+
to:
145
+
```
146
+
(** Run jtw generation in a container using jtw opam *)
147
+
```
148
+
149
+
**Step 4: Build and verify**
150
+
151
+
Run: `dune build day10/bin/main.exe 2>&1 | head -20`
152
+
Expected: Compiles successfully.
153
+
154
+
**Step 5: Commit**
155
+
156
+
```bash
157
+
git add day10/bin/linux.ml
158
+
git commit -m "day10: remove post-container artifact generation, jtw handles it"
159
+
```
160
+
161
+
---
162
+
163
+
### Task 4: Delete dead code from jtw_gen.ml
164
+
165
+
With `jtw_container_script` no longer generating js_of_ocaml commands and `linux.ml` no longer calling `generate_dynamic_cmis_json`, these functions are dead code.
166
+
167
+
**Files:**
168
+
- Modify: `day10/bin/jtw_gen.ml`
169
+
170
+
**Step 1: Verify no remaining references**
171
+
172
+
Run:
173
+
```bash
174
+
grep -r 'generate_dynamic_cmis_json\|jsoo_compile_command' day10/
175
+
```
176
+
Expected: No matches (both should be unreferenced after Tasks 2 and 3).
177
+
178
+
**Step 2: Delete `generate_dynamic_cmis_json` (lines 9-40)**
179
+
180
+
Remove the entire function.
181
+
182
+
**Step 3: Build and verify**
183
+
184
+
Run: `dune build day10/bin/main.exe 2>&1 | head -20`
185
+
Expected: Compiles successfully.
186
+
187
+
**Step 4: Commit**
188
+
189
+
```bash
190
+
git add day10/bin/jtw_gen.ml
191
+
git commit -m "day10: delete generate_dynamic_cmis_json, now handled by jtw opam"
192
+
```
193
+
194
+
---
195
+
196
+
### Task 5: End-to-end test with day10
197
+
198
+
Verify the change works by running day10 with a fresh cache.
199
+
200
+
**Step 1: Build day10**
201
+
202
+
Run: `dune build day10/bin/main.exe`
203
+
204
+
**Step 2: Run day10 batch on a test package**
205
+
206
+
Run:
207
+
```bash
208
+
dune exec day10/bin/main.exe batch cmdliner.1.3.0 -- \
209
+
--cache /tmp/day10-jtw-refactor-test \
210
+
--with-jtw --jtw-output /tmp/day10-jtw-refactor-test/jtw-output \
211
+
--jtw-tools-repo https://github.com/nickcannata/monopam.git \
212
+
--jtw-tools-branch universe-builder
213
+
```
214
+
215
+
**Step 3: Verify output**
216
+
217
+
Check that the jtw output has the expected structure:
218
+
```bash
219
+
# Should have findlib_index.json with "meta_files" key
220
+
cat /tmp/day10-jtw-refactor-test/jtw-output/u/*/findlib_index.json | python3 -m json.tool
221
+
222
+
# Should have dynamic_cmis.json files in lib dirs
223
+
find /tmp/day10-jtw-refactor-test/jtw-output -name dynamic_cmis.json | head -5
224
+
225
+
# Should have .cma.js files
226
+
find /tmp/day10-jtw-refactor-test/jtw-output -name '*.cma.js' | head -5
227
+
228
+
# Should have .cmi files in p/ directories
229
+
find /tmp/day10-jtw-refactor-test/jtw-output/p -name '*.cmi' | head -5
230
+
231
+
# Should have META files
232
+
find /tmp/day10-jtw-refactor-test/jtw-output/p -name META | head -5
233
+
234
+
# Should NOT have any old-style findlib_index (no extension)
235
+
find /tmp/day10-jtw-refactor-test/jtw-output -name findlib_index -not -name '*.json'
236
+
```
237
+
238
+
**Step 4: Also test with jtw opam mode (regression)**
239
+
240
+
Run:
241
+
```bash
242
+
dune exec js_top_worker/bin/jtw.exe opam -- -o /tmp/jtw-opam-test --no-worker cmdliner
243
+
ls /tmp/jtw-opam-test/findlib_index.json
244
+
ls /tmp/jtw-opam-test/lib/cmdliner/dynamic_cmis.json
245
+
```
246
+
247
+
**Step 5: Run node tests**
248
+
249
+
Run: `dune build @js_top_worker/test/node/runtest`
250
+
Expected: All tests pass.
251
+
252
+
---
253
+
254
+
### Task 6: Clean up /tmp test dirs and final commit
255
+
256
+
**Step 1: Clean up temp directories**
257
+
258
+
```bash
259
+
rm -rf /tmp/day10-jtw-refactor-test /tmp/jtw-opam-test
260
+
```
261
+
262
+
**Step 2: Verify all changes compile cleanly**
263
+
264
+
```bash
265
+
dune build day10/bin/main.exe js_top_worker/bin/jtw.exe
266
+
```
267
+
268
+
**Step 3: Review the diff**
269
+
270
+
```bash
271
+
git diff HEAD~4 --stat
272
+
```
273
+
274
+
Expected: Changes only in `day10/bin/jtw_gen.ml` and `day10/bin/linux.ml`. No changes to `js_top_worker/` files (the whole point of this refactor).