js_top_worker Architecture#

This document describes the current architecture of js_top_worker and the planned changes.

Overview#

js_top_worker is an OCaml toplevel (REPL) designed to run in a Web Worker or remote process. It enables interactive OCaml execution in browsers for:

• Jupyter-style notebooks
• Interactive documentation
• Educational tools (lecture slides, tutorials)
• Library documentation with live examples

System Architecture#

┌─────────────────────────────────────────────────────────────────┐
│                         Browser                                  │
│  ┌──────────────────┐         ┌──────────────────────────────┐  │
│  │   Frontend       │         │      Web Worker              │  │
│  │                  │         │                              │  │
│  │  ┌────────────┐  │  RPC    │  ┌────────────────────────┐  │  │
│  │  │   Client   │◄─┼────────►│  │      Server           │  │  │
│  │  │ (Lwt/Fut)  │  │ JSON    │  │   (worker.ml)         │  │  │
│  │  └────────────┘  │         │  └──────────┬─────────────┘  │  │
│  │                  │         │             │                │  │
│  │                  │         │  ┌──────────▼─────────────┐  │  │
│  │                  │         │  │     Implementation     │  │  │
│  │                  │         │  │      (impl.ml)         │  │  │
│  │                  │         │  │  - Execute phrases     │  │  │
│  │                  │         │  │  - Type checking       │  │  │
│  │                  │         │  │  - Code completion     │  │  │
│  │                  │         │  └──────────┬─────────────┘  │  │
│  │                  │         │             │                │  │
│  │                  │         │  ┌──────────▼─────────────┐  │  │
│  │                  │         │  │   js_of_ocaml-toplevel │  │  │
│  │                  │         │  │      + Merlin          │  │  │
│  │                  │         │  └────────────────────────┘  │  │
│  └──────────────────┘         └──────────────────────────────┘  │
└─────────────────────────────────────────────────────────────────┘

Package Structure#

Current Communication Layer#

RPC Protocol#

Uses ocaml-rpc with JSON-RPC 2.0:

Client                              Server (Worker)
   │                                     │
   │  ──── JSON-RPC request ────────►    │
   │       {method: "exec",              │
   │        params: ["let x = 1"],       │
   │        id: 1}                       │
   │                                     │
   │  ◄─── JSON-RPC response ────────    │
   │       {result: {...},               │
   │        id: 1}                       │
   │                                     │

RPC Operations#

Type Definitions#

Key types from idl/toplevel_api.ml:

type exec_result = {
  stdout : string option;
  stderr : string option;
  sharp_ppf : string option;      (* # directive output *)
  caml_ppf : string option;       (* Regular output *)
  highlight : highlight option;   (* Error location *)
  mime_vals : mime_val list;      (* Rich output *)
}

type mime_val = {
  mime_type : string;             (* e.g., "text/html" *)
  encoding : encoding;            (* Noencoding | Base64 *)
  data : string;
}

type init_config = {
  findlib_requires : string list; (* Packages to preload *)
  stdlib_dcs : string option;     (* Dynamic CMIs URL *)
  execute : bool;                 (* Allow execution? *)
}

Core Implementation#

Module Structure (lib/impl.ml)#

module type S = sig
  type findlib_t
  val capture : (unit -> 'a) -> unit -> captured * 'a
  val sync_get : string -> string option
  val async_get : string -> (string, [`Msg of string]) result Lwt.t
  val import_scripts : string list -> unit
  val get_stdlib_dcs : string -> dynamic_cmis list
  val findlib_init : string -> findlib_t Lwt.t
  val require : bool -> findlib_t -> string list -> dynamic_cmis list
  val path : string
end

module Make (S : S) : sig
  val init : init_config -> unit Lwt.t
  val setup : unit -> exec_result Lwt.t
  val exec : string -> exec_result Lwt.t
  val typecheck : string -> exec_result Lwt.t
  (* ... *)
end

Execution Flow#

exec(phrase)
    │
    ▼
capture stdout/stderr
    │
    ▼
parse phrase (Ocamltop.parse_toplevel)
    │
    ▼
execute (Toploop.execute_phrase)
    │
    ▼
collect MIME outputs
    │
    ▼
return exec_result

Cell Dependency System#

Cells can depend on previous cells via module wrapping:

(* Cell "c1" defines: *)
let x = 1

(* Internally becomes module Cell__c1 *)

(* Cell "c2" with deps=["c1"]: *)
let y = x + 1

(* Prepended with: open Cell__c1 *)

The mangle_toplevel function handles this transformation.

Library Loading#

findlibish.ml#

Custom findlib-like implementation for WebWorker context:

                          ┌─────────────────┐
                          │ findlib_index.json │ (list of META URLs)
                          └────────┬────────┘
                                   │
                    ┌──────────────┼──────────────┐
                    ▼              ▼              ▼
              ┌─────────┐   ┌─────────┐   ┌─────────┐
              │  META   │   │  META   │   │  META   │
              │ (pkg A) │   │ (pkg B) │   │ (pkg C) │
              └────┬────┘   └────┬────┘   └────┬────┘
                   │             │             │
                   ▼             ▼             ▼
              ┌─────────────────────────────────────┐
              │        Dependency Resolution         │
              └─────────────────┬───────────────────┘
                                │
                   ┌────────────┼────────────┐
                   ▼            ▼            ▼
             ┌──────────┐ ┌──────────┐ ┌──────────┐
             │ .cma.js  │ │ .cma.js  │ │ .cma.js  │
             │ (import) │ │ (import) │ │ (import) │
             └──────────┘ └──────────┘ └──────────┘

Package Loading Process#

1. Fetch findlib_index.json (list of META file URLs)
2. Parse each META file with Fl_metascanner
3. Build dependency graph
4. On #require:
   • Resolve dependencies
   • Fetch dynamic_cmis.json for each package
   • Load .cma.js via import_scripts

Preloaded Packages#

These are compiled into the worker and not loaded dynamically:

• compiler-libs.common, compiler-libs.toplevel
• merlin-lib.*
• js_of_ocaml-compiler, js_of_ocaml-toplevel
• findlib, findlib.top

Merlin Integration#

Code intelligence features use Merlin:

Queries run through Mpipeline with source "mangled" to include cell dependencies.

Planned Architecture Changes#

Phase 1: Communication Redesign#

Replace JSON-RPC with CBOR-based bidirectional channel:

Current:                           Planned:
┌─────────┐     JSON-RPC          ┌─────────┐     CBOR
│ Client  │◄──────────────►       │ Client  │◄──────────────►
│         │  request/response     │         │  bidirectional
└─────────┘                       └─────────┘

                                  Message types:
                                  - Request/Response (like RPC)
                                  - Push (server → client)
                                  - Widget events (bidirectional)

Phase 2: Environment Isolation#

Multiple isolated execution contexts:

┌──────────────────────────────────────────┐
│              Web Worker                   │
│                                          │
│  ┌─────────────┐    ┌─────────────┐     │
│  │   Env "a"   │    │   Env "b"   │     │
│  │             │    │             │     │
│  │ Cell 1      │    │ Cell 1      │     │
│  │ Cell 2      │    │ Cell 2      │     │
│  │ (isolated)  │    │ (isolated)  │     │
│  └─────────────┘    └─────────────┘     │
│                                          │
│  Shared: stdlib, preloaded packages      │
└──────────────────────────────────────────┘

Phase 3: Rich Output & Widgets#

MIME-typed output with bidirectional widget communication:

(* User code *)
let chart = Chart.bar [1; 2; 3; 4] in
Display.show chart

(* Generates *)
{
  mime_type = "application/vnd.widget+json";
  data = {widget_id = "w1"; state = ...}
}

(* Frontend renders widget, sends events back *)
Widget_event {widget_id = "w1"; event = Click {x; y}}

File Reference#

Core Files#

Build Outputs#

Dependencies#

Runtime#

• js_of_ocaml >= 3.11.0
• js_of_ocaml-toplevel
• js_of_ocaml-compiler
• rpclib, rpclib-lwt
• merlin-lib
• compiler-libs
• brr >= 0.0.4

Planned Additions#

• cbort - CBOR codec (tangled.org)
• zarith_stubs_js - JS stubs for zarith
• bytesrw - Streaming I/O

Last updated: 2026-01-20