# Interactive OCaml Demos — End-to-End Setup

This document explains how to build and serve the interactive OCaml demo
pages that use the `odoc-interactive-extension` and `js_top_worker` (jtw)
to create executable code cells inside odoc documentation.

## Overview

The system has three main components:

1. **odoc-interactive-extension** — an odoc plugin that transforms
   `{@ocaml[...]}` code blocks into `<x-ocaml>` custom elements and
   processes `@x-ocaml.universe` tags.

2. **x-ocaml** — a WebComponent (`<x-ocaml>`) that creates CodeMirror
   editors, connects to a Web Worker for OCaml evaluation, and renders
   output inline.

3. **jtw** (js_top_worker) — builds "universe" directories containing
   compiled OCaml libraries (`.cma.js` files, `.cmi` stubs, findlib
   metadata) that the worker loads at runtime.

### Demo pages

| Page | Description | Universe |
|------|-------------|----------|
| `demo1.html` | Basic expressions + Yojson | `./universe` (yojson from default switch) |
| `demo2_v2.html` | Yojson 2.x API | `./universe-v2` (yojson from demo-yojson-v2 switch) |
| `demo2_v3.html` | Yojson 3.x API | `./universe-v3` (yojson from default switch) |

The v2 and v3 demos show that different library versions can coexist as
separate universes, each page loading its own set of compiled libraries.

## Prerequisites

- OCaml 5.4+ with opam
- The monorepo checked out at `/home/jons-agent/workspace/mono` (or adjust paths)
- Two opam switches:
  - `default` — has yojson 3.x (or latest)
  - `demo-yojson-v2` — has yojson 2.2.2
- Python 3 (for the dev server, or any static file server with CORS)

### Creating the yojson v2 switch (one-time)

```bash
opam switch create demo-yojson-v2 ocaml-base-compiler.5.4.1
eval $(opam env --switch demo-yojson-v2 --set-switch)
opam install yojson.2.2.2
```

## Build Steps

All commands assume you start from the monorepo root.

### Step 1: Build and install the toolchain

The odoc extension must be installed before generating docs, because odoc
loads extensions as plugins at doc-generation time.

```bash
eval $(opam env --switch default --set-switch)
dune build @install
```

This builds and installs (locally) `odoc`, `jtw`, `x-ocaml`, and the
`odoc-interactive-extension` plugin.

### Step 2: Generate odoc HTML

```bash
dune build @doc
```

This generates HTML for all packages. The demo `.mld` files
(`odoc-interactive-extension/doc/demo1.mld`, etc.) are compiled as part
of the `odoc-interactive-extension` package documentation.

The HTML output lands in:
```
_build/default/_doc/_html/odoc-interactive-extension/
├── demo1.html
├── demo2_v2.html
├── demo2_v3.html
└── index.html
```

### Step 3: Copy x-ocaml assets

The extension loads `x-ocaml.js` and `worker.js` from a local
`_x-ocaml/` directory relative to the HTML pages. Copy them from the
dune build output:

```bash
HTMLDIR=_build/default/_doc/_html/odoc-interactive-extension
mkdir -p "$HTMLDIR/_x-ocaml"
cp _build/default/x-ocaml/src/x_ocaml.bc.js "$HTMLDIR/_x-ocaml/x-ocaml.js"
cp _build/default/js_top_worker/lib/.worker.eobjs/jsoo/worker.bc.js "$HTMLDIR/_x-ocaml/worker.js"
```

### Step 4: Build universe directories

Each demo page declares a universe via `@x-ocaml.universe ./universe-name`
in its `.mld` source. We use `jtw opam` to generate each universe
directory.

The `jtw` binary is at `_build/install/default/bin/jtw`.

```bash
JTW=_build/install/default/bin/jtw

# Universe for demo1 (default switch, yojson latest)
eval $(opam env --switch default --set-switch)
$JTW opam --no-worker --output "$HTMLDIR/universe" yojson

# Universe for demo2_v3 (default switch, yojson latest = 3.x)
$JTW opam --no-worker --output "$HTMLDIR/universe-v3" yojson

# Universe for demo2_v2 (demo-yojson-v2 switch, yojson 2.2.2)
eval $(opam env --switch demo-yojson-v2 --set-switch)
$JTW opam --no-worker --switch demo-yojson-v2 --output "$HTMLDIR/universe-v2" yojson

# Switch back to default
eval $(opam env --switch default --set-switch)
```

Key flags:
- `--no-worker` — skip building a worker.js (we provide our own via `_x-ocaml/`)
- `--output DIR` — where to write the universe files
- `--switch NAME` — which opam switch to read packages from
- `yojson` — positional argument: the library to include (plus its deps)

Each universe directory will contain:
```
universe/
├── findlib_index.json       # lists META file locations
└── lib/
    ├── ocaml/
    │   ├── META
    │   ├── dynamic_cmis.json   # lists .cmi files to load on demand
    │   └── *.cmi               # stdlib type interfaces
    ├── yojson/
    │   ├── META
    │   ├── *.cmi
    │   └── yojson.cma.js       # compiled library code
    └── seq/                    # (dependency, if needed)
        └── ...
```

### Step 5: Serve and test

Start a local HTTP server from the HTML root:

```bash
python3 docs/demos/cors_server.py 8080 _build/default/_doc/_html
```

Then open in a browser:
- http://localhost:8080/odoc-interactive-extension/demo1.html
- http://localhost:8080/odoc-interactive-extension/demo2_v2.html
- http://localhost:8080/odoc-interactive-extension/demo2_v3.html

Each page should:
1. Auto-evaluate all code cells on load
2. Show output inline below each expression (e.g., `- : int = 7`)
3. Display stdout output (e.g., `Hello, World!`)

## Writing Demo Pages

Demo pages are standard odoc `.mld` files with two additions:

### Universe tag

Declares the universe directory (relative to the HTML page):

```
@x-ocaml.universe ./universe
```

This makes the extension use the `jtw` backend (js_top_worker) instead of
the builtin merlin-js backend. The worker loads libraries from the
specified universe directory.

### Code cells

Standard odoc `{@ocaml[...]}` blocks become interactive cells:

```
{@ocaml[
1 + 2 * 3
]}
```

For cells that use `#require` or reference libraries:

```
{@ocaml[
#require "yojson"
]}

{@ocaml[
let json = `Assoc [("key", `String "value")]
let () = print_endline (Yojson.Safe.to_string json)
]}
```

Note: The `#require` directive must be in a separate cell before the code
that uses the library, because each cell is evaluated independently and
`#require` loads the library's `.cma.js` file.

### Cell modes (extension syntax)

For the extended `{x@ocaml ... x[...]x}` syntax, additional attributes
control cell behaviour:

```
{x@ocaml hidden x[let helper = 42]x}         (* invisible setup cell *)
{x@ocaml exercise id=solve x[...]x}          (* editable cell *)
{x@ocaml test for=solve x[assert (...)]x}    (* test cell, linked *)
{x@ocaml interactive x[read_only_code]x}     (* default: visible, read-only *)
```

## Architecture Notes

### Same-origin constraint

Web Workers cannot be loaded from a different origin. The extension
therefore always loads `x-ocaml.js` and `worker.js` from the local
`_x-ocaml/` directory (same origin as the page).

Universe directories can technically be cross-origin for fetching
`findlib_index.json` and `.cmi` files (these use `fetch()`), but
`.cma.js` files are loaded via `importScripts()` inside the worker,
which also requires same-origin in modern browsers. For simplicity,
keep universe directories same-origin (relative paths in the
`@x-ocaml.universe` tag).

### How the pieces connect

```
Page Load
  │
  ├─ odoc extension emits:
  │    <meta name="x-ocaml-universe" content="./universe">
  │    <script src="./_x-ocaml/x-ocaml.js" src-worker="./_x-ocaml/worker.js" backend="jtw">
  │    <x-ocaml mode="interactive">code...</x-ocaml>
  │
  ├─ x-ocaml.js (WebComponent):
  │    1. Reads <meta> tags for universe URL
  │    2. Creates Web Worker from worker.js
  │    3. Sends init message with findlib_index URL
  │    4. For each <x-ocaml> cell:
  │       - Creates CodeMirror editor
  │       - Sends eval message to worker
  │       - Renders streaming output as decorations
  │
  └─ worker.js (js_top_worker):
       1. Receives init with findlib_index URL
       2. Fetches findlib_index.json → learns library locations
       3. Fetches dynamic_cmis.json → learns available .cmi files
       4. On eval: compiles OCaml code using toplevel
       5. On #require: loads .cma.js via importScripts
       6. Sends output_at (per-phrase) and output (final) messages
```

### jtw_client.ml — the bridge

`x-ocaml/src/jtw_client.ml` bridges x-ocaml's `X_protocol` with
js_top_worker's message protocol. Key design decisions:

- Uses `Jtw.eval_stream` (not `Jtw.eval`) to capture per-phrase
  `output_at` messages, which contain the actual evaluation results
  (e.g., `val x : int = 3`). The `Jtw.eval` function only returns the
  final `output` message, which often has an empty `caml_ppf`.

- Derives `stdlib_dcs` (the URL for `dynamic_cmis.json`) from the
  `findlib_index` URL, so the worker can locate stdlib CMIs regardless
  of whether the universe is at a relative or absolute path.

## Troubleshooting

**No output after clicking Run / cells don't auto-evaluate**
- Check browser console for worker errors
- Verify `_x-ocaml/worker.js` exists and is accessible
- Verify the universe's `findlib_index.json` is accessible

**"Toplevel not initialized" error**
- The worker.js must be built with `js_of_ocaml-toplevel` and the flags
  `--toplevel +toplevel.js +dynlink.js`

**"Unbound module Yojson" after #require**
- Check that `#require "yojson"` is in a separate earlier cell
- Verify the universe directory contains `lib/yojson/yojson.cma.js`

**Cross-origin errors**
- Keep universe directories same-origin (use relative paths in
  `@x-ocaml.universe`)
- The `_x-ocaml/` directory must be on the same origin as the HTML page

**Console shows 404 for .cmi files**
- Normal if the code doesn't reference those modules; the worker
  fetches CMIs on demand and some 404s are expected for unused modules