jon.recoil.org
this repo has no description
Add design doc and implementation plan for interactive OCaml tutorials
System design for web-based, client-side OCaml tutorials authored in
.mld files. Covers universe structure, authoring format with tagged
code blocks, thin odoc plugin approach, x-ocaml cell modes, and
widget/FRP bridge experiments.
Implementation plan has four parallel streams: odoc-interactive-extension
plugin, x-ocaml cell modes, universe builder improvements, and
Lwd vs Note FRP experiments.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
+350
docs/plans/2026-02-20-ocaml-interactive-tutorials-design.md
+350
docs/plans/2026-02-20-ocaml-interactive-tutorials-design.md
···
1
+
# Interactive OCaml Tutorials — System Design
2
+
3
+
## Overview
4
+
5
+
A system for authoring web-based, purely client-side interactive OCaml
6
+
tutorials and exercises. Authors write `.mld` files using odoc, run
7
+
`dune build @doc`, and get HTML pages where code blocks are live,
8
+
editable, and executable — backed by a Web Worker running the OCaml
9
+
toplevel.
10
+
11
+
The system supports three use cases:
12
+
13
+
1. **Scrollycode tutorials** — step-by-step, scroll-driven code
14
+
walkthroughs (already prototyped via `odoc-scrollycode-extension`)
15
+
2. **Exercises and assessment** — editable skeleton code with
16
+
immutable test cells, in the style of Jupyter/nbgrader (e.g.,
17
+
Cambridge "Foundations of Computer Science")
18
+
3. **Interactive widgets** — reactive UI elements (sliders, plots,
19
+
mini-apps) driven by an FRP library running in the Worker
20
+
21
+
There is no server-side component beyond serving static files over
22
+
HTTP.
23
+
24
+
## Architecture (Approach C — Thin Plugin + Smart WebComponent)
25
+
26
+
The odoc plugin is deliberately thin. Its job is to translate
27
+
`{@ocaml ...}` tagged code blocks into `<x-ocaml>` HTML elements with
28
+
data attributes. All interactive behaviour lives in the x-ocaml
29
+
WebComponent and the js_top_worker backend.
30
+
31
+
```
32
+
Author writes .mld odoc plugin x-ocaml + worker
33
+
───────────────────── ──> ──────────────── ──> ─────────────────
34
+
{@ocaml exercise <x-ocaml WebComponent reads
35
+
id=factorial mode="exercise" data attrs, manages
36
+
[let facr n = ...]} data-id="factorial"> UI, sends code to
37
+
... worker for execution
38
+
</x-ocaml>
39
+
```
40
+
41
+
This approach means:
42
+
- The plugin stays simple — pure data transformation
43
+
- Interactive behaviour can be iterated by updating x-ocaml.js without
44
+
re-running odoc
45
+
- The system works outside odoc (hand-written HTML, other doc
46
+
generators)
47
+
48
+
## 1. Universe Structure
49
+
50
+
A **universe** is a self-consistent set of compiled OCaml packages,
51
+
discoverable via a `findlib_index.json` file. OCaml requires that all
52
+
libraries in a universe are built with exactly the same versions of
53
+
all transitive dependencies — you cannot mix libraries from different
54
+
build environments.
55
+
56
+
This constraint means a central host (e.g., ocaml.org) cannot serve a
57
+
single universal set of libraries. Different tutorials may need
58
+
different package combinations. The system supports both self-hosted
59
+
universes and centrally-hosted ones.
60
+
61
+
Additionally, OCaml and OxCaml are fundamentally incompatible — each
62
+
requires its own universe.
63
+
64
+
### Directory layout
65
+
66
+
```
67
+
universe/
68
+
├── findlib_index.json # lists META paths for all packages
69
+
├── worker.js # js_top_worker compiled to JS
70
+
├── x-ocaml.js # WebComponent runtime
71
+
├── stdlib/
72
+
│ ├── META
73
+
│ ├── dynamic_cmis.json
74
+
│ ├── stdlib.cma.js
75
+
│ └── *.cmi
76
+
├── cmdliner/
77
+
│ ├── META
78
+
│ ├── dynamic_cmis.json
79
+
│ ├── cmdliner.cma.js
80
+
│ └── *.cmi
81
+
└── .../
82
+
```
83
+
84
+
### Discovery mechanism
85
+
86
+
The existing js_top_worker discovery mechanism (findlibish) is the
87
+
runtime — no new protocol needed:
88
+
89
+
- **findlib_index.json** — JSON file listing META file paths and
90
+
pointers to other universes
91
+
- **META files** — standard findlib metadata with `requires`,
92
+
`archive`, `directory` fields
93
+
- **dynamic_cmis.json** — per-library file listing available modules
94
+
and CMI file prefixes for on-demand loading
95
+
- **Universe linking** — findlib_index.json can reference other
96
+
universes via a `universes` field; the worker resolves dependencies
97
+
transitively from META files
98
+
99
+
### Building a universe
100
+
101
+
**From an opam switch (common case):** A CLI tool walks the switch,
102
+
and for each installed package: copies its META file, compiles the
103
+
`.cma` to `.cma.js` (via js_of_ocaml), generates `dynamic_cmis.json`
104
+
from the `.cmi` files, and writes the `findlib_index.json` listing all
105
+
packages. Tooling for this partially exists and needs to be connected
106
+
into a coherent tool.
107
+
108
+
**From day10 (at scale):** The CI pipeline builds universes across
109
+
multiple compiler versions and OS targets, producing the same layout
110
+
for each coherent package set. day10 manages many co-existing
111
+
universes with its filesystem hierarchy.
112
+
113
+
### ocaml.org integration
114
+
115
+
The doc build pipeline produces both HTML docs and the JS/cmi
116
+
artifacts needed for interactivity. Each package's tutorial gets a
117
+
universe consisting of that package plus its transitive dependencies,
118
+
all already compiled as part of the doc pipeline.
119
+
120
+
## 2. Authoring Format
121
+
122
+
### Page-level configuration
123
+
124
+
Custom tags at the top of the `.mld` file configure the page:
125
+
126
+
```
127
+
@x-ocaml.universe https://ocaml.org/universe/5.3.0
128
+
@x-ocaml.requires cmdliner, astring
129
+
@x-ocaml.auto-execute false
130
+
@x-ocaml.merlin false
131
+
```
132
+
133
+
- **universe** — URL where the `findlib_index.json` lives. Falls back
134
+
to relative `./universe/` if omitted.
135
+
- **requires** — packages to load at initialization, before any cells
136
+
run.
137
+
- **auto-execute** — whether cells run automatically on page load
138
+
(default: true).
139
+
- **merlin** — whether Merlin-based LSP feedback is enabled (default:
140
+
true). Can be overridden per-cell.
141
+
142
+
### Cell types
143
+
144
+
Code blocks use odoc's tagged code block syntax:
145
+
`{@ocaml <attributes> [...code...]}`.
146
+
147
+
| Attribute | Purpose | Editable? | Visible? |
148
+
|---------------|--------------------------------------|-----------|----------|
149
+
| `interactive` | Demo/example cell | No | Yes |
150
+
| `exercise` | Skeleton for student to fill in | Yes | Yes |
151
+
| `test` | Immutable test assertions | No | Yes |
152
+
| `hidden` | Setup code, runs but not shown | No | No |
153
+
154
+
### Additional per-cell attributes
155
+
156
+
| Attribute | Purpose |
157
+
|----------------|--------------------------------------------|
158
+
| `id=name` | Name this cell for explicit linking |
159
+
| `for=name` | Link a test cell to a specific exercise |
160
+
| `env=name` | Named execution environment |
161
+
| `merlin` | Override page-level merlin setting |
162
+
163
+
### Exercise linking
164
+
165
+
Test cells are linked to exercise cells by two mechanisms:
166
+
167
+
- **Positional (default)** — a test cell applies to the nearest
168
+
preceding exercise cell.
169
+
- **Explicit** — use `id` and `for` attributes when the test is
170
+
distant or ambiguous.
171
+
172
+
### Example: assessment worksheet
173
+
174
+
```
175
+
{@ocaml hidden [
176
+
(* Setup code the student doesn't see *)
177
+
let check_positive f =
178
+
assert (f 0 = 1);
179
+
assert (f 1 = 1)
180
+
]}
181
+
182
+
Write an OCaml function [facr] to compute the factorial by recursion.
183
+
184
+
{@ocaml exercise id=factorial [
185
+
let rec facr n =
186
+
(* YOUR CODE HERE *)
187
+
failwith "Not implemented"
188
+
]}
189
+
190
+
{@ocaml test for=factorial [
191
+
assert (facr 10 = 3628800);;
192
+
assert (facr 11 = 39916800);;
193
+
]}
194
+
```
195
+
196
+
### What the odoc plugin emits
197
+
198
+
The plugin maps attributes directly to HTML data attributes:
199
+
200
+
```html
201
+
<x-ocaml mode="hidden">
202
+
(* Setup code the student doesn't see *)
203
+
let check_positive f = ...
204
+
</x-ocaml>
205
+
206
+
<p>Write an OCaml function <code>facr</code> to compute the factorial
207
+
by recursion.</p>
208
+
209
+
<x-ocaml mode="exercise" data-id="factorial">
210
+
let rec facr n =
211
+
(* YOUR CODE HERE *)
212
+
failwith "Not implemented"
213
+
</x-ocaml>
214
+
215
+
<x-ocaml mode="test" data-for="factorial">
216
+
assert (facr 10 = 3628800);;
217
+
assert (facr 11 = 39916800);;
218
+
</x-ocaml>
219
+
```
220
+
221
+
The plugin also injects a `<script>` tag for x-ocaml.js (once per
222
+
page) and a `<meta>` tag for the universe URL.
223
+
224
+
## 3. odoc Plugin
225
+
226
+
The plugin is deliberately minimal:
227
+
228
+
1. **Parse** `{@ocaml <tags> [...]}` blocks
229
+
2. **Emit** `<x-ocaml>` HTML elements with tags mapped to data
230
+
attributes
231
+
3. **Inject** a `<script>` tag for x-ocaml.js (once per page)
232
+
4. **Emit** `<meta>` tags for page-level configuration
233
+
(`@x-ocaml.*` custom tags)
234
+
235
+
The plugin does **not** handle:
236
+
- Exercise grouping logic (x-ocaml's job)
237
+
- Universe discovery (WebComponent + worker)
238
+
- Widget wiring (FRP bridge)
239
+
- Cell execution or state management
240
+
241
+
## 4. Runtime (x-ocaml WebComponent)
242
+
243
+
The x-ocaml WebComponent is where interactive behaviour lives. It:
244
+
245
+
- Reads `mode`, `data-id`, `data-for`, `data-requires` etc. from its
246
+
HTML attributes
247
+
- Discovers the universe from `<meta name="x-ocaml-universe">` or
248
+
falls back to `./universe/`
249
+
- Manages the js_top_worker Web Worker lifecycle
250
+
- Provides CodeMirror editing for `exercise` cells
251
+
- Provides Merlin integration (configurable)
252
+
- Handles cell dependency ordering and execution
253
+
- Resolves test-to-exercise linking (positional + explicit)
254
+
255
+
### Execution environments
256
+
257
+
Cells sharing an `env` attribute see each other's definitions. By
258
+
default, all cells on a page share one environment. Named environments
259
+
allow isolation when needed.
260
+
261
+
## 5. Widget/FRP Bridge (Experimental)
262
+
263
+
### Architecture
264
+
265
+
Backend-authoritative, inspired by Marimo's model. The Worker owns all
266
+
state. View descriptions flow out via postMessage, user events flow
267
+
back in.
268
+
269
+
```
270
+
Worker Main Thread (x-ocaml)
271
+
┌─────────────────┐ ┌─────────────────────┐
272
+
│ OCaml code │ │ │
273
+
│ ↓ │ view │ │
274
+
│ FRP library │ desc. │ Render to DOM │
275
+
│ ↓ │ ────────> │ ↓ │
276
+
│ Serializable │ │ Real DOM │
277
+
│ view description │ events │ ↓ │
278
+
│ │ <──────── │ User interaction │
279
+
└─────────────────┘ └─────────────────────┘
280
+
```
281
+
282
+
### Reactivity library (to be determined)
283
+
284
+
Two candidates:
285
+
286
+
**Lwd** (Frederic Bour) — lightweight incremental computation.
287
+
`Lwd.var` (mutable inputs) and `Lwd.t` (derived values) with
288
+
monadic/applicative composition. Natural for building tree-structured
289
+
view descriptions. Actively maintained (v0.4, May 2025). Multiple
290
+
existing backends (terminal, web).
291
+
292
+
**Note** (Daniel Bunzli) — classic FRP with events (`E`) and signals
293
+
(`S`). Designed explicitly for js_of_ocaml (no weak references).
294
+
Cleaner model for discrete user interactions.
295
+
296
+
### View description format
297
+
298
+
A custom serializable type — no closures, no JS object references.
299
+
Event handlers represented as symbolic descriptors. Inspired by
300
+
ocaml-vdom's pure `Vdom` module and TyXML's functorial architecture.
301
+
302
+
Optional: instantiate TyXML's functors over the serializable type for
303
+
type-safe HTML construction.
304
+
305
+
### Proposed experiments
306
+
307
+
1. **Experiment A — Counter with Lwd**: Minimal counter (button +
308
+
display). Lwd in Worker produces serializable view, main thread
309
+
renders, click events sent back.
310
+
311
+
2. **Experiment B — Counter with Note**: Same counter using Note's
312
+
signals and events. Compare code ergonomics.
313
+
314
+
3. **Experiment C — Richer widget**: Multiple interacting controls
315
+
(e.g., two sliders controlling a computed value). Tests composition
316
+
in each library.
317
+
318
+
4. **Experiment D — TyXML integration**: Instantiate TyXML's functors
319
+
over a serializable backend. Evaluate whether the type safety is
320
+
worth the complexity.
321
+
322
+
**Evaluation criteria:**
323
+
- Code ergonomics
324
+
- Serialization format (what does the view description look like on
325
+
the wire?)
326
+
- Event round-trip latency
327
+
- Bundle size impact on worker.js
328
+
329
+
## Summary of Components
330
+
331
+
| Component | Status | Purpose |
332
+
|------------------------------|----------------|--------------------------------------------|
333
+
| js_top_worker | Exists | OCaml toplevel in a Web Worker |
334
+
| x-ocaml | Exists | WebComponent for interactive code cells |
335
+
| odoc (fork) | Exists | Doc generator with extension plugin system |
336
+
| odoc-scrollycode-extension | Exists | Scroll-driven code tutorials |
337
+
| odoc-interactive-extension | **To build** | Thin plugin: tags → x-ocaml elements |
338
+
| Universe builder (opam) | **To build** | opam switch → hostable artifacts |
339
+
| Universe builder (day10) | Partially exists | At-scale universe management |
340
+
| Widget/FRP bridge | **To design** | Experiments needed (Lwd vs Note) |
341
+
342
+
## Open Questions
343
+
344
+
- **Exercise grouping**: Should cells be explicitly grouped into named
345
+
exercises, or is implicit grouping from document structure
346
+
sufficient? Needs prototyping.
347
+
- **FRP library choice**: Lwd vs Note — to be resolved by experiments.
348
+
- **View serialization format**: Depends on FRP library choice.
349
+
Virtual DOM diffs vs structured widget protocol vs something else.
350
+
- **TyXML integration**: Worth the complexity? Experiment D will tell.
+773
docs/plans/2026-02-20-ocaml-interactive-tutorials-plan.md
+773
docs/plans/2026-02-20-ocaml-interactive-tutorials-plan.md
···
1
+
# Interactive OCaml Tutorials — Implementation Plan
2
+
3
+
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
4
+
5
+
**Goal:** Build a system where tutorial authors write `.mld` files with tagged code blocks, run `dune build @doc`, and get HTML pages with interactive, executable OCaml cells.
6
+
7
+
**Architecture:** Thin odoc plugin (Approach C) translates `{@ocaml ...}` tags to `<x-ocaml>` HTML elements with data attributes. The x-ocaml WebComponent handles all interactive behaviour. Package universes are built from opam switches via the existing `jtw` tool and discovered at runtime via `findlib_index.json`.
8
+
9
+
**Tech Stack:** OCaml, js_of_ocaml, odoc (fork with extension API), dune-site plugins, x-ocaml WebComponent, js_top_worker
10
+
11
+
**Repo:** `/home/jons-agent/workspace/mono` (monorepo managed by monopam)
12
+
13
+
---
14
+
15
+
## Work Streams
16
+
17
+
The work is organised into four independent streams. Streams 1-3 can
18
+
proceed in parallel. Stream 4 (FRP experiments) is exploratory and
19
+
independent.
20
+
21
+
### Stream 1: odoc-interactive-extension (the thin plugin)
22
+
### Stream 2: x-ocaml cell modes (exercise/test/hidden/interactive)
23
+
### Stream 3: Universe builder improvements (findlib_index.json)
24
+
### Stream 4: Widget/FRP bridge experiments
25
+
26
+
---
27
+
28
+
## Stream 1: odoc-interactive-extension
29
+
30
+
Create a new odoc extension plugin that handles `{@ocaml ...}` code
31
+
blocks with interactive attributes and `@x-ocaml.*` custom tags.
32
+
33
+
### Current state
34
+
35
+
- The odoc extension API exists at `odoc/src/extension_api/odoc_extension_api.ml`
36
+
- `odoc-scrollycode-extension/` is a working reference implementation
37
+
- Code blocks with attributes like `{@ocaml foo=bar [...]}` are
38
+
already parsed by odoc's parser into `code_block_meta` with
39
+
`language` and `tags` fields
40
+
- The `Code_Block_Extension` module type handles code block
41
+
transformation
42
+
- The `Extension` module type handles custom `@tag` processing
43
+
- Resources (JS/CSS URLs) are collected and injected into page `<head>`
44
+
45
+
### Key files to reference
46
+
47
+
- `odoc-scrollycode-extension/src/scrollycode_extension.ml` — reference
48
+
plugin implementation
49
+
- `odoc-scrollycode-extension/src/dune` — dune config for plugin
50
+
registration
51
+
- `odoc/src/extension_api/odoc_extension_api.ml` — the API to implement
52
+
- `odoc/src/extension_registry/odoc_extension_registry.ml` — registry internals
53
+
- `odoc/src/document/comment.ml:257-307` — where code block extensions
54
+
are dispatched
55
+
56
+
### Task 1.1: Scaffold odoc-interactive-extension directory
57
+
58
+
**Files:**
59
+
- Create: `odoc-interactive-extension/dune-project`
60
+
- Create: `odoc-interactive-extension/src/dune`
61
+
- Create: `odoc-interactive-extension/src/interactive_extension.ml`
62
+
63
+
**Step 1: Create dune-project**
64
+
65
+
```
66
+
(lang dune 3.18)
67
+
(using dune_site 0.1)
68
+
(name odoc-interactive-extension)
69
+
(generate_opam_files true)
70
+
71
+
(package
72
+
(name odoc-interactive-extension)
73
+
(synopsis "Interactive OCaml code cells for odoc documentation")
74
+
(depends
75
+
(ocaml (>= 4.14))
76
+
odoc))
77
+
```
78
+
79
+
**Step 2: Create src/dune**
80
+
81
+
Follow the scrollycode pattern exactly:
82
+
83
+
```
84
+
(library
85
+
(public_name odoc-interactive-extension.impl)
86
+
(name interactive_extension)
87
+
(libraries odoc.extension_api))
88
+
89
+
(plugin
90
+
(name odoc-interactive-extension)
91
+
(libraries odoc-interactive-extension.impl)
92
+
(site (odoc extensions)))
93
+
```
94
+
95
+
**Step 3: Create minimal interactive_extension.ml**
96
+
97
+
Start with a skeleton that registers both a Code_Block_Extension (for
98
+
`{@ocaml ...}`) and an Extension (for `@x-ocaml.*` tags):
99
+
100
+
```ocaml
101
+
open Odoc_extension_api
102
+
103
+
(* Page-level config accumulated during processing *)
104
+
let universe_url = ref None
105
+
let requires = ref []
106
+
let auto_execute = ref true
107
+
let merlin_enabled = ref true
108
+
109
+
module X_ocaml_config : Extension = struct
110
+
let prefix = "x-ocaml"
111
+
112
+
let to_document ~tag content =
113
+
(* tag will be "x-ocaml.universe", "x-ocaml.requires", etc. *)
114
+
let subtag = match String.split_on_char '.' tag with
115
+
| _ :: rest -> String.concat "." rest
116
+
| _ -> tag
117
+
in
118
+
let text = (* extract text from content *) "" in
119
+
(match subtag with
120
+
| "universe" -> universe_url := Some text
121
+
| "requires" -> requires := String.split_on_char ',' text
122
+
| "auto-execute" -> auto_execute := (text <> "false")
123
+
| "merlin" -> merlin_enabled := (text <> "false")
124
+
| _ -> ());
125
+
(* Emit nothing visible *)
126
+
{ content = []; overrides = []; resources = []; assets = [] }
127
+
end
128
+
129
+
module X_ocaml_code : Code_Block_Extension = struct
130
+
let prefix = "ocaml"
131
+
132
+
let to_document meta code =
133
+
let tags = meta.tags in
134
+
(* Extract mode: interactive, exercise, test, hidden *)
135
+
let mode = (* ... find mode tag ... *) "interactive" in
136
+
let id_attr = (* ... find id=xxx binding ... *) None in
137
+
let for_attr = (* ... find for=xxx binding ... *) None in
138
+
let merlin_attr = (* ... find merlin tag ... *) None in
139
+
(* Build data attributes string *)
140
+
let attrs = String.concat " "
141
+
(List.filter_map Fun.id [
142
+
Some (Printf.sprintf "mode=\"%s\"" mode);
143
+
Option.map (Printf.sprintf "data-id=\"%s\"") id_attr;
144
+
Option.map (Printf.sprintf "data-for=\"%s\"") for_attr;
145
+
Option.map (fun _ -> "data-merlin") merlin_attr;
146
+
])
147
+
in
148
+
let html = Printf.sprintf "<x-ocaml %s>\n%s\n</x-ocaml>" attrs code in
149
+
Some {
150
+
content = [{ attr = []; desc = Raw_markup ("html", html) }];
151
+
overrides = [];
152
+
resources = [
153
+
(* x-ocaml.js — URL will come from universe config *)
154
+
Js_url "x-ocaml.js";
155
+
];
156
+
assets = [];
157
+
}
158
+
end
159
+
160
+
let () =
161
+
Registry.register (module X_ocaml_config);
162
+
Registry.register_code_block (module X_ocaml_code)
163
+
```
164
+
165
+
**Step 4: Build to verify compilation**
166
+
167
+
Run: `opam exec -- dune build -p odoc-interactive-extension`
168
+
169
+
Expected: Compiles without errors (may need adjustments to match
170
+
exact API signatures — check `odoc_extension_api.mli` for precise
171
+
types)
172
+
173
+
**Step 5: Commit**
174
+
175
+
```bash
176
+
git add odoc-interactive-extension/
177
+
git commit -m "feat: scaffold odoc-interactive-extension plugin"
178
+
```
179
+
180
+
### Task 1.2: Implement @x-ocaml.* config tag handling
181
+
182
+
**Files:**
183
+
- Modify: `odoc-interactive-extension/src/interactive_extension.ml`
184
+
185
+
**Step 1: Study how scrollycode extracts text from tag content**
186
+
187
+
Read `odoc-scrollycode-extension/src/scrollycode_extension.ml` to see
188
+
how `to_document ~tag content` extracts text from the
189
+
`nestable_block_element list`. The content is structured AST, not
190
+
plain text.
191
+
192
+
**Step 2: Implement text extraction from tag content**
193
+
194
+
Write a helper that extracts plain text from the nestable block
195
+
element list (the tag payload).
196
+
197
+
**Step 3: Implement meta tag emission**
198
+
199
+
The `@x-ocaml.universe` tag should emit a `<meta>` tag in the page.
200
+
Since odoc resource injection happens in `<head>`, this may need to
201
+
use `Raw_markup` in the content, or a `Js_inline` resource containing
202
+
a meta tag injection script. Investigate which approach works.
203
+
204
+
**Step 4: Build and test with a sample .mld file**
205
+
206
+
Create a test `.mld` file with `@x-ocaml.universe` and verify the
207
+
generated HTML contains the right meta tag.
208
+
209
+
**Step 5: Commit**
210
+
211
+
### Task 1.3: Implement code block transformation
212
+
213
+
**Files:**
214
+
- Modify: `odoc-interactive-extension/src/interactive_extension.ml`
215
+
216
+
**Step 1: Study the Code_Block_Extension API precisely**
217
+
218
+
Read `odoc/src/extension_api/odoc_extension_api.ml` — look at the
219
+
`code_block_meta` type and `code_block_tags` to understand how
220
+
`{@ocaml exercise id=factorial [...]}` is parsed. The tags will be a
221
+
list of `Tag` (bare) and `Binding` (key=value) entries.
222
+
223
+
**Step 2: Implement tag extraction**
224
+
225
+
Write helpers to extract:
226
+
- Mode: first bare tag that matches `interactive|exercise|test|hidden`
227
+
- `id`: from `Binding("id", value)`
228
+
- `for`: from `Binding("for", value)`
229
+
- `merlin`: bare tag presence
230
+
- `env`: from `Binding("env", value)`
231
+
232
+
**Step 3: Implement HTML emission**
233
+
234
+
Generate `<x-ocaml>` elements with the appropriate data attributes.
235
+
The code content goes as the text content of the element. Make sure
236
+
to HTML-escape the code content.
237
+
238
+
**Step 4: Handle the script tag injection**
239
+
240
+
The x-ocaml.js script should be injected once per page via
241
+
`resources`. The URL should reference the universe base URL if
242
+
configured, otherwise use a relative path. Investigate whether the
243
+
resource URL can be dynamic (based on the `@x-ocaml.universe` config)
244
+
or must be static.
245
+
246
+
**Step 5: Build and test with sample .mld**
247
+
248
+
Create a test `.mld` file with various cell types:
249
+
250
+
```
251
+
@x-ocaml.universe ./universe
252
+
253
+
{@ocaml hidden [let helper x = x + 1]}
254
+
255
+
{@ocaml interactive [let x = helper 41]}
256
+
257
+
{@ocaml exercise id=double [
258
+
let double x =
259
+
(* YOUR CODE HERE *)
260
+
failwith "Not implemented"
261
+
]}
262
+
263
+
{@ocaml test for=double [assert (double 5 = 10)]}
264
+
```
265
+
266
+
Run `dune build @doc` and inspect the generated HTML to verify:
267
+
- Hidden cell emits `<x-ocaml mode="hidden">`
268
+
- Interactive cell emits `<x-ocaml mode="interactive">`
269
+
- Exercise cell emits `<x-ocaml mode="exercise" data-id="double">`
270
+
- Test cell emits `<x-ocaml mode="test" data-for="double">`
271
+
- Script tag is injected once
272
+
- Meta tag for universe URL is present
273
+
274
+
**Step 6: Commit**
275
+
276
+
### Task 1.4: Add to monorepo build
277
+
278
+
**Files:**
279
+
- Modify: `dune-project` (root) — add package
280
+
- Modify: `sources.toml` — if needed for monopam
281
+
282
+
**Step 1: Add the package to the root dune-project**
283
+
284
+
The monorepo's root `dune-project` needs to know about the new
285
+
package. Check how other extension packages are registered.
286
+
287
+
**Step 2: Verify it builds in the monorepo context**
288
+
289
+
Run: `opam exec -- dune build`
290
+
291
+
**Step 3: Commit**
292
+
293
+
---
294
+
295
+
## Stream 2: x-ocaml cell modes
296
+
297
+
Extend the x-ocaml WebComponent to support `mode`, `data-id`,
298
+
`data-for`, `data-env`, `data-merlin`, and `data-auto-execute`
299
+
attributes.
300
+
301
+
### Current state
302
+
303
+
- x-ocaml currently reads attributes only from the `<script>` tag,
304
+
not from individual `<x-ocaml>` elements
305
+
- Per-element, only `run-on` is read (click vs load)
306
+
- Cells form a linked list (`prev`/`next`) with implicit sequential
307
+
execution
308
+
- All cells are fully editable with CodeMirror
309
+
- No concept of modes, hidden cells, or test cells
310
+
- Cell status is: `Not_run | Running | Run_ok | Request_run`
311
+
312
+
### Key files
313
+
314
+
- `x-ocaml/src/x_ocaml.ml` — main entry, WebComponent registration,
315
+
global state
316
+
- `x-ocaml/src/cell.ml` — individual cell UI, editor, execution
317
+
- `x-ocaml/src/editor.ml` — CodeMirror wrapper
318
+
- `x-ocaml/src/webcomponent.ml` — custom element definition, shadow
319
+
DOM, attribute reading
320
+
- `x-ocaml/src/backend.ml` — worker communication abstraction
321
+
322
+
### Task 2.1: Read mode and data attributes from elements
323
+
324
+
**Files:**
325
+
- Modify: `x-ocaml/src/cell.ml`
326
+
- Modify: `x-ocaml/src/x_ocaml.ml`
327
+
328
+
**Step 1: Read the current cell initialisation code**
329
+
330
+
Read `cell.ml` — specifically `Cell.init` and `Cell.start` — to
331
+
understand how cells are created and what data they hold. Read
332
+
`x_ocaml.ml` to see how the `connectedCallback` creates cells.
333
+
334
+
**Step 2: Add mode type and reading**
335
+
336
+
Add a `mode` type to `cell.ml`:
337
+
338
+
```ocaml
339
+
type mode = Interactive | Exercise | Test | Hidden
340
+
```
341
+
342
+
Read `mode` from the element's `mode` attribute in the
343
+
`connectedCallback`. Default to `Interactive` if not present.
344
+
345
+
**Step 3: Read data attributes**
346
+
347
+
Read `data-id`, `data-for`, `data-env`, `data-merlin` from the
348
+
element. Store them in the cell record.
349
+
350
+
**Step 4: Build and verify**
351
+
352
+
Run: `opam exec -- dune build`
353
+
354
+
**Step 5: Commit**
355
+
356
+
### Task 2.2: Implement hidden cell behaviour
357
+
358
+
**Files:**
359
+
- Modify: `x-ocaml/src/cell.ml`
360
+
361
+
**Step 1: Skip rendering for hidden cells**
362
+
363
+
In the cell initialization, if `mode = Hidden`:
364
+
- Don't create the CodeMirror editor
365
+
- Don't create the run button
366
+
- Don't attach to shadow DOM (or set `display: none`)
367
+
- Still register in the cell linked list for execution ordering
368
+
- Still hold the code text from the element's textContent
369
+
370
+
**Step 2: Ensure hidden cells execute**
371
+
372
+
Hidden cells should still execute their code (they provide setup
373
+
definitions). When a subsequent cell runs and triggers the linked
374
+
list dependency chain, hidden cells should execute silently.
375
+
376
+
**Step 3: Test manually**
377
+
378
+
Create an HTML file with:
379
+
```html
380
+
<x-ocaml mode="hidden">let helper x = x + 1</x-ocaml>
381
+
<x-ocaml mode="interactive">helper 41</x-ocaml>
382
+
```
383
+
384
+
Verify the hidden cell is not visible and `helper 41` evaluates to
385
+
`42`.
386
+
387
+
**Step 4: Commit**
388
+
389
+
### Task 2.3: Implement exercise cell behaviour
390
+
391
+
**Files:**
392
+
- Modify: `x-ocaml/src/cell.ml`
393
+
394
+
**Step 1: Make exercise cells editable, others read-only**
395
+
396
+
Currently all cells are editable. Change this:
397
+
- `Exercise`: editable (keep current behaviour)
398
+
- `Interactive`: editable but could be made read-only — check design
399
+
doc. Design says "No" for editable, so set CodeMirror to read-only.
400
+
- `Test`: read-only
401
+
- `Hidden`: no editor
402
+
403
+
Use CodeMirror's `readOnly` extension/config to control this.
404
+
405
+
**Step 2: Visual differentiation**
406
+
407
+
Add CSS classes or styles to distinguish exercise cells:
408
+
- Exercise cells could have a light background tint or border
409
+
indicating "edit here"
410
+
- Test cells could have a different tint indicating "assertion"
411
+
412
+
Keep it minimal — just enough to distinguish cell types visually.
413
+
414
+
**Step 3: Test manually**
415
+
416
+
Create an HTML page with exercise + test cells and verify:
417
+
- Exercise cell is editable
418
+
- Test cell is read-only
419
+
- Visual distinction is clear
420
+
421
+
**Step 4: Commit**
422
+
423
+
### Task 2.4: Implement test cell linking
424
+
425
+
**Files:**
426
+
- Modify: `x-ocaml/src/cell.ml`
427
+
- Modify: `x-ocaml/src/x_ocaml.ml`
428
+
429
+
**Step 1: Implement positional linking**
430
+
431
+
After all cells are registered (in the `connectedCallback` or a
432
+
post-registration pass), for each test cell without a `data-for`
433
+
attribute, find the nearest preceding exercise cell and record the
434
+
association.
435
+
436
+
**Step 2: Implement explicit linking**
437
+
438
+
For test cells with `data-for="name"`, find the cell with
439
+
`data-id="name"` and record the association.
440
+
441
+
**Step 3: Implement test execution trigger**
442
+
443
+
When an exercise cell is executed (user clicks Run), automatically
444
+
execute its associated test cells afterwards. Display pass/fail
445
+
results in the test cell's output area.
446
+
447
+
**Step 4: Test manually**
448
+
449
+
Create an HTML page with the factorial exercise example. Verify:
450
+
- Filling in the correct implementation and running it triggers the
451
+
test cell
452
+
- Tests passing shows success
453
+
- Tests failing shows the assertion error
454
+
455
+
**Step 5: Commit**
456
+
457
+
### Task 2.5: Implement per-cell merlin override
458
+
459
+
**Files:**
460
+
- Modify: `x-ocaml/src/cell.ml`
461
+
- Modify: `x-ocaml/src/x_ocaml.ml`
462
+
463
+
**Step 1: Read page-level merlin setting**
464
+
465
+
The page-level `data-merlin` (from `<meta>` tag or `<script>` tag
466
+
attribute) sets the default. Read this in `x_ocaml.ml`.
467
+
468
+
**Step 2: Allow per-cell override**
469
+
470
+
If an `<x-ocaml>` element has `data-merlin="true"` or
471
+
`data-merlin="false"`, override the page default for that cell.
472
+
473
+
**Step 3: Skip merlin initialisation for disabled cells**
474
+
475
+
In `cell.ml`, when creating the Merlin client, check the cell's
476
+
merlin setting. If disabled, don't create the Merlin worker or
477
+
register error/completion callbacks.
478
+
479
+
**Step 4: Test and commit**
480
+
481
+
### Task 2.6: Implement universe discovery from meta tag
482
+
483
+
**Files:**
484
+
- Modify: `x-ocaml/src/x_ocaml.ml`
485
+
486
+
**Step 1: Read universe URL from meta tag**
487
+
488
+
Currently x-ocaml reads `src-worker` from the `<script>` tag. Add
489
+
logic to also check for:
490
+
```html
491
+
<meta name="x-ocaml-universe" content="https://...">
492
+
```
493
+
494
+
If present, derive the worker URL from the universe base
495
+
(`{universe}/worker.js`).
496
+
497
+
**Step 2: Fallback chain**
498
+
499
+
1. `<meta name="x-ocaml-universe">` → use `{url}/worker.js`
500
+
2. `src-worker` attribute on `<script>` tag (existing behaviour)
501
+
3. Relative `./universe/worker.js` (default)
502
+
503
+
**Step 3: Pass universe URL to worker for findlib_index.json**
504
+
505
+
The worker needs to know where to find `findlib_index.json`. This
506
+
may require extending the `init` RPC call to accept the
507
+
findlib_index URL (currently it uses a relative path).
508
+
509
+
**Step 4: Test and commit**
510
+
511
+
---
512
+
513
+
## Stream 3: Universe builder improvements
514
+
515
+
Improve the `jtw opam` command to produce the universe layout from
516
+
the design doc, using `findlib_index.json`.
517
+
518
+
### Current state
519
+
520
+
- `jtw opam` exists in `js_top_worker/bin/jtw.ml` (lines 499-535)
521
+
- It walks an opam switch, copies CMIs, compiles cma.js, generates
522
+
`findlib_index` (not `.json`) and `dynamic_cmis.json`
523
+
- The JSON format currently uses `"metas"` key (not `"meta_files"`)
524
+
- `findlibish.ml` parser accepts both `"metas"` and `"meta_files"`
525
+
- The tool already handles most of what's needed
526
+
527
+
### Key files
528
+
529
+
- `js_top_worker/bin/jtw.ml` — the CLI tool (571 lines)
530
+
- `js_top_worker/bin/ocamlfind.ml` — queries ocamlfind for packages
531
+
- `js_top_worker/lib/findlibish.ml` — findlib_index parser/loader
532
+
533
+
### Task 3.1: Rename findlib_index to findlib_index.json
534
+
535
+
**Files:**
536
+
- Modify: `js_top_worker/bin/jtw.ml`
537
+
- Modify: `js_top_worker/lib/findlibish.ml`
538
+
539
+
**Step 1: Read the current code**
540
+
541
+
Read `jtw.ml` to find where `findlib_index` filenames are generated.
542
+
Read `findlibish.ml` to find where `findlib_index` is expected.
543
+
544
+
**Step 2: Update output filename**
545
+
546
+
In `jtw.ml`, change the output filename from `findlib_index` to
547
+
`findlib_index.json` wherever universes are written.
548
+
549
+
**Step 3: Update default input path**
550
+
551
+
In `findlibish.ml` (or wherever the default findlib_index path is
552
+
set), update to look for `findlib_index.json`. Keep backward
553
+
compatibility: if `findlib_index.json` doesn't exist, try
554
+
`findlib_index`.
555
+
556
+
**Step 4: Standardise JSON key names**
557
+
558
+
If the generated JSON uses `"metas"`, also emit `"meta_files"` for
559
+
consistency with the design doc. The parser already accepts both.
560
+
561
+
**Step 5: Build and test**
562
+
563
+
Run: `opam exec -- dune build`
564
+
565
+
If there are existing tests for jtw, run them.
566
+
567
+
**Step 6: Commit**
568
+
569
+
### Task 3.2: Include worker.js and x-ocaml.js in universe output
570
+
571
+
**Files:**
572
+
- Modify: `js_top_worker/bin/jtw.ml`
573
+
574
+
**Step 1: Study current worker.js generation**
575
+
576
+
The `jtw opam` command has a `--no-worker` flag, meaning it can
577
+
already include worker.js. Read the code to understand how
578
+
`Mk_backend.mk` works and where the worker.js gets placed.
579
+
580
+
**Step 2: Add x-ocaml.js to output**
581
+
582
+
When generating a universe, also copy x-ocaml.js into the output
583
+
directory alongside worker.js. This may require a new flag
584
+
(`--x-ocaml-js PATH`) or auto-detection.
585
+
586
+
**Step 3: Update the design doc if needed**
587
+
588
+
If the implementation differs from the design doc's universe layout,
589
+
update the design doc.
590
+
591
+
**Step 4: Build and test**
592
+
593
+
**Step 5: Commit**
594
+
595
+
### Task 3.3: Add @x-ocaml.requires support to worker init
596
+
597
+
**Files:**
598
+
- Modify: `js_top_worker/lib/impl.ml`
599
+
600
+
**Step 1: Study current requires handling**
601
+
602
+
The `init` RPC already accepts `findlib_requires` in `init_config`
603
+
(impl.ml line 655+). Check whether x-ocaml currently passes
604
+
requires through.
605
+
606
+
**Step 2: Verify the flow**
607
+
608
+
Trace the path from x-ocaml reading `@x-ocaml.requires` (from
609
+
`<meta>` tag) through to the worker's `init`/`setup` calls. If
610
+
x-ocaml already has a way to pass requires, document it. If not,
611
+
identify where the integration point is.
612
+
613
+
**Step 3: Implement if needed**
614
+
615
+
If x-ocaml doesn't currently pass requires to the worker init,
616
+
add this capability.
617
+
618
+
**Step 4: Test and commit**
619
+
620
+
---
621
+
622
+
## Stream 4: Widget/FRP bridge experiments
623
+
624
+
These are exploratory — the goal is to evaluate Lwd and Note for the
625
+
widget architecture, not to build production code.
626
+
627
+
### Task 4.1: Set up experiment scaffolding
628
+
629
+
**Files:**
630
+
- Create: `experiments/widget-bridge/dune-project`
631
+
- Create: `experiments/widget-bridge/README.md`
632
+
633
+
**Step 1: Create experiment directory**
634
+
635
+
Set up a minimal dune project outside the monorepo (or in a
636
+
subdirectory) that can build js_of_ocaml targets and use web workers.
637
+
638
+
**Step 2: Define the serializable view type**
639
+
640
+
Create a simple, serializable view description type:
641
+
642
+
```ocaml
643
+
type event_id = string
644
+
645
+
type attr =
646
+
| Property of string * string
647
+
| Style of string * string
648
+
| Class of string
649
+
| On of string * event_id (* event name, handler id *)
650
+
651
+
type node =
652
+
| Text of string
653
+
| Element of { tag : string; attrs : attr list; children : node list }
654
+
655
+
(* Serialise to JSON for postMessage *)
656
+
val to_json : node -> string
657
+
val of_json : string -> node
658
+
```
659
+
660
+
**Step 3: Define the event message type**
661
+
662
+
```ocaml
663
+
type event_msg = {
664
+
handler_id : event_id;
665
+
event_type : string;
666
+
value : string option; (* for input elements *)
667
+
}
668
+
```
669
+
670
+
**Step 4: Commit**
671
+
672
+
### Task 4.2: Experiment A — Counter with Lwd
673
+
674
+
**Files:**
675
+
- Create: `experiments/widget-bridge/lwd_counter/`
676
+
677
+
**Step 1: Install Lwd**
678
+
679
+
Run: `opam install lwd`
680
+
681
+
**Step 2: Write the worker-side counter**
682
+
683
+
Using Lwd, create a counter:
684
+
- `Lwd.var` for the count
685
+
- Derived `Lwd.t` that produces a `node` view description
686
+
- On "increment" event, update the var
687
+
- On each change, serialize the view and post it
688
+
689
+
**Step 3: Write the main-thread renderer**
690
+
691
+
A small JS/OCaml module that:
692
+
- Receives serialized view descriptions
693
+
- Diffs against current DOM (or just replaces)
694
+
- Attaches event listeners that send messages back
695
+
696
+
**Step 4: Measure and document**
697
+
698
+
- Code size (lines of OCaml)
699
+
- Bundle size impact
700
+
- Round-trip latency (click → updated DOM)
701
+
- Ergonomics notes
702
+
703
+
**Step 5: Commit**
704
+
705
+
### Task 4.3: Experiment B — Counter with Note
706
+
707
+
**Files:**
708
+
- Create: `experiments/widget-bridge/note_counter/`
709
+
710
+
Same as Task 4.2 but using Note's signals and events. Compare:
711
+
712
+
- `Note.S.create` for the count signal
713
+
- `Note.E.create` for click events
714
+
- `Note.S.map` to derive the view
715
+
716
+
**Document comparison with Lwd version.**
717
+
718
+
### Task 4.4: Experiment C — Richer widget (two sliders)
719
+
720
+
**Files:**
721
+
- Create: `experiments/widget-bridge/sliders/`
722
+
723
+
Using whichever library felt better from A/B, build:
724
+
- Two sliders that control x and y
725
+
- A computed display showing x * y
726
+
- Tests composition and multiple interacting inputs
727
+
728
+
### Task 4.5: Experiment D — TyXML backend
729
+
730
+
**Files:**
731
+
- Create: `experiments/widget-bridge/tyxml_backend/`
732
+
733
+
Try instantiating TyXML's `Html_f.Make` functor with a custom
734
+
`Xml_sigs.T` implementation that produces the serializable `node`
735
+
type. Evaluate whether the type safety is worth the boilerplate.
736
+
737
+
---
738
+
739
+
## Dependency graph
740
+
741
+
```
742
+
Stream 1 (odoc plugin) ──────────────────────┐
743
+
│
744
+
Stream 2 (x-ocaml modes) ───────────────────┼──> End-to-end demo
745
+
│
746
+
Stream 3 (universe builder) ─────────────────┘
747
+
748
+
Stream 4 (FRP experiments) ──────────────────────> Decision on widget arch
749
+
```
750
+
751
+
Streams 1, 2, 3 converge in an end-to-end integration test: write
752
+
an `.mld` with exercises, build with `dune build @doc`, serve the
753
+
output alongside a universe, and verify the interactive tutorial
754
+
works in a browser.
755
+
756
+
Stream 4 is independent exploratory work.
757
+
758
+
## End-to-end integration test
759
+
760
+
After Streams 1-3 are complete:
761
+
762
+
1. Build a universe from the current opam switch using `jtw opam`
763
+
2. Write a sample `.mld` tutorial with hidden setup, interactive
764
+
demos, exercises, and tests
765
+
3. Run `dune build @doc` with the interactive extension installed
766
+
4. Serve the generated HTML + universe with a local HTTP server
767
+
5. Open in browser and verify:
768
+
- Hidden cells are invisible but their definitions are available
769
+
- Interactive cells display pre-filled code
770
+
- Exercise cells are editable
771
+
- Running an exercise triggers associated test cells
772
+
- Test pass/fail is clearly displayed
773
+
- Merlin provides completions in exercise cells