this repo has no description

Add content-hashed path support for immutable caching

Add OcamlWorker.fromManifest() to discover content-hashed worker URLs
from manifest.json, and update all test pages to use it instead of
hardcoded compiler paths. This enables Cache-Control: immutable for
all artifact paths except manifest.json and findlib_index.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

+84 -15
+21
client/ocaml-worker.js
··· 112 } 113 114 /** 115 * Create a new OCaml worker client. 116 * @param {string} workerUrl - URL to the worker script (e.g., '_opam/worker.js') 117 * @param {Object} [options] - Options
··· 112 } 113 114 /** 115 + * Create a worker from a manifest.json URL. 116 + * Fetches the manifest to discover content-hashed worker and stdlib_dcs URLs. 117 + * @param {string} manifestUrl - URL to manifest.json (e.g., '/jtw-output/manifest.json') 118 + * @param {string} [ocamlVersion='5.4.0'] - OCaml compiler version 119 + * @param {Object} [options] - Options passed to OcamlWorker constructor 120 + * @returns {Promise<{worker: OcamlWorker, stdlib_dcs: string}>} 121 + */ 122 + static async fromManifest(manifestUrl, ocamlVersion = '5.4.0', options = {}) { 123 + const resp = await fetch(manifestUrl); 124 + if (!resp.ok) throw new Error(`Failed to fetch manifest: ${resp.status}`); 125 + const manifest = await resp.json(); 126 + const compiler = manifest.compilers[ocamlVersion]; 127 + if (!compiler) throw new Error(`OCaml ${ocamlVersion} not found in manifest`); 128 + const baseUrl = new URL(manifestUrl, window.location.href); 129 + const workerUrl = new URL(compiler.worker_url, baseUrl).href; 130 + const stdlibDcs = compiler.stdlib_dcs; 131 + const worker = new OcamlWorker(workerUrl, options); 132 + return { worker, stdlib_dcs: stdlibDcs }; 133 + } 134 + 135 + /** 136 * Create a new OCaml worker client. 137 * @param {string} workerUrl - URL to the worker script (e.g., '_opam/worker.js') 138 * @param {Object} [options] - Options
+46
docs/technical-qa.md
··· 4 5 --- 6 7 ## 2026-01-20: What does `--include-runtime` do in js_of_ocaml? 8 9 **Question**: What does the `--include-runtime` argument actually do when compiling with js_of_ocaml?
··· 4 5 --- 6 7 + ## 2026-02-06: Is js_of_ocaml compilation deterministic? 8 + 9 + **Question**: Is js_of_ocaml compilation deterministic? If we rebuild the same package, will the `.cma.js` file have the same content hash? This matters for using content hashes as cache-busting URLs. 10 + 11 + **Answer**: Yes, js_of_ocaml compilation is deterministic. Given the same inputs (bytecode, debug info, compiler version, flags), it produces byte-for-byte identical JavaScript output. This is confirmed by both the js_of_ocaml maintainer (hhugo) and empirical testing. 12 + 13 + **Evidence**: 14 + 15 + 1. **Maintainer confirmation** (GitHub issue ocsigen/js_of_ocaml#1297): hhugo (Hugo Heuzard, core maintainer) stated: "Js_of_ocaml produces JS from ocaml bytecode and uses debug info (from the bytecode) to recover variable names. The renaming algo is deterministic. You should expect the jsoo build to be reproducible." 16 + 17 + 2. **Source code analysis**: The `js_output.ml` file in the compiler converts internal Hashtbl structures to sorted lists before output generation: 18 + ```ocaml 19 + let hashtbl_to_list htb = 20 + String.Hashtbl.fold (fun k v l -> (k, v) :: l) htb [] 21 + |> List.sort ~cmp:(fun (_, a) (_, b) -> compare a b) 22 + |> List.map ~f:fst 23 + ``` 24 + This ensures deterministic output regardless of Hashtbl iteration order. 25 + 26 + 3. **No embedded non-deterministic data**: Grep of `.cma.js` files found no embedded timestamps, build paths, random values, or other non-deterministic content. 27 + 28 + 4. **Empirical testing** (OCaml 5.4.0, js_of_ocaml 6.2.0): Four consecutive `dune clean && dune build` cycles (including one with `-j 1`) produced byte-for-byte identical `.cma.js` files: 29 + - `stdlib.cma.js`: `496346f4...` (all 4 builds) 30 + - `lwt.cma.js`: `e65a4a54...` (all 4 builds) 31 + - `rpclib.cma.js`: `ffaa5ffc...` (all 4 builds) 32 + - `js_of_ocaml.cma.js`: `4169ea91...` (all 4 builds) 33 + 34 + **Caveats**: 35 + 36 + - **Different OCaml compiler versions** will produce different bytecode, which leads to different `.cma.js` output. Content hashes are stable only when the full toolchain is pinned. 37 + - **Different js_of_ocaml versions** or different compiler flags (e.g., `--opt 3` vs default) will produce different output. 38 + - **Dune parallel build bug** (dune#3863): On OCaml < 4.11, parallel builds could produce non-deterministic `.cmo` files due to debug info sensitivity. This is fixed in OCaml 4.11+ (we use 5.4.0). 39 + - **`dune-build-info`**: If a package uses `dune-build-info`, the VCS revision can be embedded in the binary, but this does not affect `.cma.js` compilation for libraries that don't use it. 40 + 41 + **Conclusion**: Content hashes of `.cma.js` files are safe to use for cache-busting URLs, provided the OCaml toolchain version and js_of_ocaml version are held constant (which they are within a single ohc layer build). 42 + 43 + **Verification Steps**: 44 + - Searched web for "js_of_ocaml deterministic", "js_of_ocaml reproducible build" 45 + - Read GitHub issue ocsigen/js_of_ocaml#1297 and all comments 46 + - Analyzed js_of_ocaml compiler source (`generate.ml`, `js_output.ml`) for non-determinism 47 + - Performed 4 clean rebuilds and compared SHA-256 hashes 48 + - Tested both default parallelism and `-j 1` single-core builds 49 + - Grepped `.cma.js` output for embedded paths, timestamps, dates 50 + 51 + --- 52 + 53 ## 2026-01-20: What does `--include-runtime` do in js_of_ocaml? 54 55 **Question**: What does the `--include-runtime` argument actually do when compiling with js_of_ocaml?
+3 -5
test/ohc-integration/eval-test.html
··· 29 throw new Error('universe parameter required'); 30 } 31 32 - const workerUrl = `/jtw-output/compiler/${compilerVersion}/worker.js`; 33 const findlib_index = `/jtw-output/u/${universe}/findlib_index`; 34 - const stdlib_dcs = `lib/ocaml/dynamic_cmis.json`; 35 - 36 - status.textContent = 'Creating worker...'; 37 - const worker = new OcamlWorker(workerUrl, { timeout: 120000 }); 38 39 try { 40 status.textContent = 'Initializing...';
··· 29 throw new Error('universe parameter required'); 30 } 31 32 + status.textContent = 'Fetching manifest...'; 33 + const { worker, stdlib_dcs } = await OcamlWorker.fromManifest( 34 + '/jtw-output/manifest.json', compilerVersion, { timeout: 120000 }); 35 const findlib_index = `/jtw-output/u/${universe}/findlib_index`; 36 37 try { 38 status.textContent = 'Initializing...';
+6 -2
test/ohc-integration/runner.html
··· 481 482 // Worker cache 483 const workerCache = new Map(); 484 485 async function getWorker(universe) { 486 if (workerCache.has(universe)) return workerCache.get(universe); 487 - const w = new OcamlWorker(`/jtw-output/compiler/5.4.0/worker.js`, { timeout: 120000 }); 488 await w.init({ 489 findlib_requires: [], 490 - stdlib_dcs: 'lib/ocaml/dynamic_cmis.json', 491 findlib_index: `/jtw-output/u/${universe}/findlib_index`, 492 }); 493 workerCache.set(universe, w);
··· 481 482 // Worker cache 483 const workerCache = new Map(); 484 + const manifestPromise = fetch('/jtw-output/manifest.json').then(r => r.json()); 485 486 async function getWorker(universe) { 487 if (workerCache.has(universe)) return workerCache.get(universe); 488 + const manifest = await manifestPromise; 489 + const compiler = manifest.compilers['5.4.0']; 490 + const workerUrl = `/jtw-output/${compiler.worker_url}`; 491 + const w = new OcamlWorker(workerUrl, { timeout: 120000 }); 492 await w.init({ 493 findlib_requires: [], 494 + stdlib_dcs: compiler.stdlib_dcs, 495 findlib_index: `/jtw-output/u/${universe}/findlib_index`, 496 }); 497 workerCache.set(universe, w);
+3 -5
test/ohc-integration/test.html
··· 31 throw new Error('universe parameter required'); 32 } 33 34 - const workerUrl = `/jtw-output/compiler/${compilerVersion}/worker.js`; 35 const findlib_index = `/jtw-output/u/${universe}/findlib_index`; 36 - const stdlib_dcs = `lib/ocaml/dynamic_cmis.json`; 37 - 38 - status.textContent = 'Creating worker...'; 39 - const worker = new OcamlWorker(workerUrl, { timeout: 120000 }); 40 41 try { 42 status.textContent = 'Initializing (loading stdlib)...';
··· 31 throw new Error('universe parameter required'); 32 } 33 34 + status.textContent = 'Fetching manifest...'; 35 + const { worker, stdlib_dcs } = await OcamlWorker.fromManifest( 36 + '/jtw-output/manifest.json', compilerVersion, { timeout: 120000 }); 37 const findlib_index = `/jtw-output/u/${universe}/findlib_index`; 38 39 try { 40 status.textContent = 'Initializing (loading stdlib)...';
+5 -3
test/ohc-integration/tutorials/tutorial.html
··· 160 content.appendChild(sEl); 161 } 162 163 - // Init worker 164 initEl.innerHTML = '<span class="spinner">&#x25E0;</span> Initializing OCaml worker...'; 165 let worker; 166 try { 167 - worker = new OcamlWorker(`/jtw-output/compiler/${tutorial.compiler || '5.4.0'}/worker.js`, { timeout: 120000 }); 168 await worker.init({ 169 findlib_requires: [], 170 - stdlib_dcs: 'lib/ocaml/dynamic_cmis.json', 171 findlib_index: `/jtw-output/u/${tutorial.universe}/findlib_index`, 172 }); 173
··· 160 content.appendChild(sEl); 161 } 162 163 + // Init worker via manifest.json for content-hashed URLs 164 initEl.innerHTML = '<span class="spinner">&#x25E0;</span> Initializing OCaml worker...'; 165 let worker; 166 try { 167 + const { worker: w, stdlib_dcs } = await OcamlWorker.fromManifest( 168 + '/jtw-output/manifest.json', tutorial.compiler || '5.4.0', { timeout: 120000 }); 169 + worker = w; 170 await worker.init({ 171 findlib_requires: [], 172 + stdlib_dcs: stdlib_dcs, 173 findlib_index: `/jtw-output/u/${tutorial.universe}/findlib_index`, 174 }); 175