Add `day10 cascade` to rerun reverse dependencies of recently fixed
packages (those that transitioned to success), and `day10 gc` for
log retention with history compaction and run archival/deletion.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

- rerun: retry failed builds by hash or package name, with --force to clear cached layers
- rdeps: reverse dependency lookup from cached solutions, with --failing filter
- notify: pluggable notification system supporting slack, zulip, telegram, email, stdout

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

- query: show package build details with optional --history flag
- failures: list failing packages with --blessed and --category filters
- changes: show status transitions since last run
- disk: show disk usage breakdown by layer type

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Reads status.json from the cache directory and displays current build
state including run info, blessed/non-blessed totals, changes, and new
packages. Supports both text and JSON output formats via --format flag.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

After history entries are recorded and the batch summary is printed,
call Status_index.generate and write to produce status.json. Also
prints a summary of any status changes and new packages detected.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

1. Extract compiler version from layer.json deps field instead of using
empty string - searches for ocaml-base-compiler or ocaml-variants
2. Record dependency_failure entries for packages that have no build
layer by comparing solutions against scanned layers
3. Deduplicate history entries by checking for existing run_id +
build_hash before appending
4. Record doc success entries for blessed packages, not just failures

Also extract matches_any helper to clean up classify_build_failure and
doc category detection.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Check the doc error message for "link" (case-insensitive) to categorize
doc failures as doc_link_failure vs doc_compile_failure, so both categories
are produced in the history.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Add record_build_result helper and classify_build_failure for pattern-matching
build logs against transient/depext failure categories. Record history entries
in print_batch_summary for build successes, classified build failures, and
blessed doc failures. Record solver failures after the solve phase in run_batch.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Implements Status_index with JSON serialization, generation from
package history files, change detection between runs, and atomic
file writes. Scans package directories, tallies blessed/non-blessed
totals by category, detects status changes and new packages.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Implements append-only JSONL storage for build history entries with
functions for reading, filtering by build_hash/blessed status, and
compacting old entries. This is the foundation for CI status tracking.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Add index.mld files for all packages that were missing them, which
caused odoc to generate default pages titled "<package> index". Also
fix leaf item indentation by always adding a spacer span regardless
of the node kind.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Layer hashes for doc, jtw, and solution workers now depend on the
compiler stack, ensuring cache invalidation when the compiler layers
change. Adds ~compiler_layers parameter to doc_layer_hash, generate_docs,
jtw_layer_hash, generate_jtw, and build_solution_worker across all
platform implementations.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

- Add cppo preprocessing for merlin-js and x-ocaml workers to support
conditional compilation with OXCAML flag
- Guard day10 packages with enabled_if >= 5.3.0 since they need recent OCaml
- Remove fatal odoc warnings from dune-workspace (handled per-package now)
- Bump merlin-js dune lang to 3.17
- Add warning suppression flags where needed (-w -58, -w -67)
- Add interactive extension exercise pages (FOCS 2020/2024/2025, OxCaml
stack allocation)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

- Consolidate js_top_worker and odoc dual-compiler stanzas into single
library stanzas with cppo rules generating impl.ml from impl.cppo.ml
- Per-package findlib_index.json with relative universe paths (../dep)
and implicit stdlib dependency injection
- Add find_stdlib_dcs to Impl.S interface for stdlib CMI lookup via
findlib metadata instead of hardcoded URLs
- Replace jsoo Json.output/Json.unsafe_input with plain JSON.stringify/
JSON.parse for cross-jsoo-version compatibility (6.0.1+ox vs 6.2.0)
- Cross-origin worker support: set __global_rel_url in blob worker,
skip URL rewriting for absolute http(s) URLs
- Fix odoc doc comments and ocamlformat-ignore for cppo files
- Add demo docs, helper scripts, and x-ocaml package-lock.json

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Adds a stable, human-readable entry point for each target package.
The per-package findlib_index contains the same compiler info and
META paths as the universe-level index, but with paths rebased
relative to p/<pkg>/<ver>/. The target package's own META uses a
short content-hashed path; all other paths include full content
hashes for immutability.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Instead of overlaying the pre-built jtw-tools layer (which was compiled
against potentially different dependency versions), install js_of_ocaml
and js_top_worker via opam inside the worker container. Pass
solution_packages version constraints to opam install so shared
dependencies (e.g., yojson) match the solution's exact versions,
eliminating CRC conflicts between universes.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Each solution now gets its own worker.js compiled against the exact
dependency versions in that solution's overlay, ensuring no CRC
mismatches when packages like yojson have different versions across
universes.

Key changes:
- jtw_gen.ml: add jtw_worker_container_script, refactor
assemble_jtw_output to accept per-solution (target, solution,
ocaml_version, worker_output_dir) tuples
- linux.ml: add run_jtw_worker_in_container and build_solution_worker
that run jtw opam (without --no-worker) in a container with the
solution's full dependency overlay
- main.ml: Phase 4 now builds worker.js per-solution before assembly
- jtw_tools.ml: add --no-worker to jtw-tools build (tools only, no
worker.js)
- s.ml + stubs: add build_solution_worker signature and stubs

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Adds --local-repo PATH CLI option (repeatable) that bind-mounts local
directories into build containers and pins opam packages from the local
path instead of from a git URL. Supports both jtw and odoc tool layers.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Implements the core module for --local-repo support with:
- discover_packages: scan directory for *.opam files
- repo_hash: compute cache hash using git state or opam file contents
- find_for_packages: match requested packages against local repos
- validate: check repo paths exist and detect duplicate packages

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Three fixes for day10's JTW integration:

1. Default --jtw-tools-repo changed from bogus github URL to correct
tangled.org URL, default branch from "enhancements" to "main"

2. opam pin commands now use git+ prefix (git+https://...) so opam
treats the URL as a git repo instead of trying to download it as
an archive

3. Container output path fixed: jtw opam --path <pkg> writes to
<output>/<pkg>/lib/ but linux.ml was looking at <output>/lib/.
Now correctly reads from <output>/<pkg>/lib/.

Tested end-to-end with seq.0.3.1: jtw-tools layer builds, per-package
artifacts generated, batch assembly produces correct findlib_index.json
with meta_files key and content-hashed paths.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Subpackage META files (e.g., base/base_internalhash_types/META) were
being returned as nested paths, which aren't valid findlib package
names. Now extracts only the first path component, since jtw opam
discovers subpackages automatically via ocamlfind.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

The jtw opam command now generates .cmi copies, META copies, and
dynamic_cmis.json inside the container, so the post-container code
in linux.ml that duplicated this work is no longer needed.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

- Add findlib_names_of_installed_libs to extract findlib package names
from installed_libs paths by finding directories containing META files
- Replace hand-rolled js_of_ocaml compile commands in jtw_container_script
with a single `jtw opam` call that handles .cmi copying, .cma -> .cma.js
compilation, dynamic_cmis.json, and findlib_index.json generation
- Delete unused jsoo_compile_command helper function

This eliminates code duplication between jtw.ml (opam mode) and
jtw_gen.ml (day10 mode), letting day10 reuse jtw opam's per-package
artifact generation logic.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Writers (jtw.ml opam mode, jtw_gen.ml day10 mode): output filename
changed from findlib_index to findlib_index.json, JSON key from
"metas" to "meta_files".

Reader (findlibish.ml): universe link resolution uses .json extension.
Runtime default (impl.ml): changed from "findlib_index" to
"findlib_index.json".

JS client, test fixtures, HTML test pages, and documentation updated
to match.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Escape curly braces and fix odoc reference syntax to eliminate warnings
with (warnings fatal):
- Replace {dir}, {pkg}, {hash} etc with [dir], [<pkg>], [<hash>] or
{[ code blocks ]} as appropriate
- Fix {!Stdlib.Out_channel.with_open_gen} -> [Out_channel.with_open_gen]
in sherlodoc type_polarity.mli
- Fix @scrolly.<theme> -> [@scrolly.<theme>] in scrollycode extension
- Fix JSON format in findlibish.ml doc comment

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Replace per-universe file copies with content-hashed paths under p/<pkg>/<ver>/<hash>/,
so identical package builds are stored once. Universe dirs now only contain a findlib_index
with relative paths into p/. Compiler dir also gains a content hash level for immutable
caching. Remove blessed/non-blessed distinction. Ensure deterministic output ordering
throughout (sorted modules, metas, find results).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Investigates the problem of finding compatible sets of pre-solved package
universes. Implements and benchmarks 6 CSP algorithms (brute force,
backtracking+FC, AC-3, greedy+MRV, signature clustering, fingerprint hashing).
Brute force with incremental map merging wins decisively at all practical scales.

Includes real-world validation against 18,388 solved universes (all versions of
all 4,491 opam packages): 63.7% pairwise compatible, with deep conflict
classification showing incompatibility is ecosystemic rather than caused by any
single dependency.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

The sudo cp -a preserves container uid ownership, preventing
subsequent cmi/META copies and dynamic_cmis.json generation.
Chown the lib directory after the initial copy.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

The permission-denied messages from the non-sudo rm are noise since
the sudo fallback handles them. Redirect stderr to /dev/null.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

- Replace non-existent 'jtw mk-backend' with 'jtw opam' which handles
both worker.js and stdlib generation in one step
- Pin and install js_top_worker-web (needed by worker.js build)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Layer files created inside containers are owned by root, so plain
rm -rf fails with permission denied. Fall back to sudo rm -rf when
the initial rm fails, matching the pattern used by Os.sudo_rm_rf.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

The js_top_worker packages are published on the fork, not upstream.
Update default --jtw-tools-repo and --jtw-tools-branch accordingly.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

The jtw-tools layer needs js_top_worker which isn't on opam. Add
--jtw-tools-repo and --jtw-tools-branch flags (like doc-tools) so the
build script pins js_top_worker, js_top_worker-rpc, js_top_worker-bin,
and js_top_worker_rpc_def from the configured repo before installing.

The repo/branch are also included in the layer hash for correct
cache invalidation.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Enable ohc to produce JavaScript toplevel artifacts alongside
documentation, so every (package, version, universe) triple gets a
working browser-based OCaml REPL with that package pre-loaded.

Adds --with-jtw and --jtw-output flags to health-check and batch
commands. Creates jtw-tools layers (js_of_ocaml + js_top_worker per
OCaml version), per-package jtw layers (.cma.js, .cmi, META,
dynamic_cmis.json), and assembles output with compiler/, u/, and p/
directory structure.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

- Add lib/progress.ml with progress tracking types and atomic writes
- Write progress.json during batch runs (after solve, during builds)
- Dashboard now shows live progress: phase, targets, build/doc counts
- Live log viewer shows completed builds (finds logs via package symlinks)
- Auto-detect platform from cache directory for web server
- Add accessor functions to Run_log for progress tracking

The dashboard now displays real-time progress during batch runs including
the current phase (solving/building/gc), target list, and completion counts.
Live log viewer now works for both in-progress and completed builds.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Lock system enhancements:
- Add universe hash to build locks enabling parallel builds of same
package in different universes
- Add Tool stage for tracking odoc/odoc_driver tool builds
- Store layer_name and temp_log_path in lock files for live viewing

Live log viewer:
- New /live/:lock_file route with auto-refresh every 2 seconds
- Reads temp_log_path from lock file for in-progress builds
- Falls back to layer log for completed builds
- Auto-scroll toggle for monitoring build progress

Race condition fixes:
- Handle EEXIST on symlink creation in parallel worker scenarios
- Remove target before cp to prevent "File exists" errors
- Wrap universes.json atomic swap in try/catch

Source documentation support:
- Scan .ml and .mli files in addition to .cmt/.cmti for odoc_driver
- Enables source code documentation in generated docs

Other improvements:
- Add layer hash to doc log symlinks for uniqueness
- Ensure package symlinks created even for pre-existing layers
- Dashboard shows clickable locks for build/doc/tool stages

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Replace single symlink per package with directory structure:
packages/{pkg.version}/
build-{hash} -> ../../build-{hash} # all builds
doc-{hash} -> ../../doc-{hash} # all doc layers
blessed-build -> ../../build-{hash} # canonical if blessed
blessed-docs -> ../../doc-{hash} # canonical if blessed

This handles packages built multiple times with different deps:
- All builds are tracked via symlinks
- blessed-* symlinks mark the canonical build for docs
- Web dashboard checks blessed-build first, falls back to listing

Changes:
- bin/util.ml: New ensure_package_layer_symlink and
ensure_package_blessed_symlink functions
- bin/main.ml: Create doc layer symlinks and blessed symlinks
when doc generation succeeds
- web/data/layer_data.ml: Updated to use new directory structure,
checks blessed-build first

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

The package detail page now determines build status from multiple sources:

1. Layer info (cache symlinks) - has exit_status and build timestamp
2. Run summary - check if package is in failures list
3. Log existence - if logs exist, package was processed

Status values:
- success: layer exit_status=0, or in run but not in failures
- failed: layer exit_status!=0, or in run failures list
- in_progress: run has no summary.json yet and build log exists
- pending: run in progress but no logs yet for this package
- unknown: no layer info and no logs found

Also shows log links only when logs actually exist, with clear
messaging when they don't.

Fixes issue where async.v0.17.0 showed "skipped" despite having logs.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Package authors need to see dependencies, reverse dependencies, and
log links even when data isn't available yet. Now:

- Build & Logs section always visible with status badge and log links
- Dependencies section always shown (with count and "not available" msg)
- Reverse Dependencies section always shown (with count)
- Log links shown when runs exist, otherwise clear "no logs yet" message

Previously these sections were hidden when data was missing, making
the page look incomplete and not discoverable.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Complete visual overhaul inspired by vintage scientific instrumentation
and CRT terminal displays:

- Typography: IBM Plex Mono for data, Space Grotesk for headings
- Color: Phosphor green (#00ff9d) primary with amber/red indicators
- Effects: Subtle CRT scan-lines, pulsing LED indicators, glow shadows
- Motion: Fade-in animations with staggered delays
- Layout: Instrument panel cards with inset data displays
- Details: Vignette overlay, hover glow effects, '>' list markers

The aesthetic matches the technical/analytical nature of a package
health checker while being visually distinctive and memorable.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Add Dream.static handler for /docs/** route to serve generated odoc HTML.
Update docs links to point directly to /doc/index.html since Dream.static
doesn't serve directory indexes.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Add comprehensive package information that package authors need:

- Build status with success/failure badge
- Build timestamp showing when package was last built
- Direct links to build and doc logs for the latest run
- Dependencies list with clickable links to each dependency
- Reverse dependencies showing packages that depend on this one
- New /packages/:name/:version/logs route for combined log view

Implementation details:
- Add symlink creation in bin/main.ml when building layers
Creates packages/{pkg_str} -> ../build-{hash} for O(1) lookup
- Add layer_data module to read layer.json via symlinks
- Add parse_package_str helper for proper name.version parsing
(handles versions like 3.21.0 and v0.17.0 correctly)

This addresses the usability issues identified in package author review:
- Package authors can now see if their package is building
- Easy access to logs when debugging build failures
- Clear dependency information in both directions

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

The previous percentage calculations were comparing apples to oranges:
- build_success (91 layers) / targets_requested (3) = 3033%
- doc_success (91 layers) / solutions_found (3) = 3033%

The confusion is that build_success/doc_success count ALL layers including
transitive dependencies, while targets_requested/solutions_found count
only the top-level packages requested.

Changed to show raw counts which are more meaningful:
- "Targets Solved: 3/3" (requested packages that solved)
- "Doc Layers OK: 91/91" (total doc layers built successfully)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

run_e2e.sh tests the full pipeline:
1. Solve a package (default: seq.0.3.1)
2. Build all dependencies
3. Generate docs with --with-doc
4. Verify outputs (summary.json, HTML, layers)

Usage:
./tests/integration/run_e2e.sh [--keep] [--package PKG]

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Run all 31 tests with: dune runtest

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- Use solutions_found/targets_requested for Solve Rate (not build_success)
- Use doc_success/solutions_found for Doc Rate
- Rename labels to be clearer: 'Solve Rate' and 'Doc Rate'

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Add package browsing functionality with:
- List page showing all packages with searchable table
- Detail page showing package documentation link and version history

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Add runs.ml view with three pages:
- list_page: Shows all runs with summary stats (builds, failures, docs)
- detail_page: Individual run with full metrics and failure table
- log_page: Build and doc log viewer with HTML escaping

Wire up routes in main.ml:
- /runs - Run history list
- /runs/:run_id - Run detail page
- /runs/:run_id/build/:package - Build log viewer
- /runs/:run_id/docs/:package - Doc log viewer

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Display overview stats including package count, build/doc success rates,
and latest run details with failure summary when available.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Add package_data module for reading package information from the html
directory. Includes functions for listing packages, versions, and
checking if docs exist for specific packages.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Add Day10_web_data.Run_data module for reading run information from day10's
log directory. Includes functions for listing runs, reading summaries,
and accessing build/doc logs.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Add command-line options for day10-web:
- --cache-dir (required): Path to day10's cache directory
- --html-dir (required): Path to generated documentation directory
- --port/-p (default: 8080): HTTP port to listen on
- --host (default: 127.0.0.1): Host address to bind to
- --platform (default: debian-12-x86_64): Platform subdirectory in cache

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Add day10-web package to the project with:
- New package stanza in dune-project with Dream dependency
- Minimal web server with Dream serving a basic HTML page
- Add explicit (package day10) to bin/dune to resolve multi-package ambiguity

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Detailed step-by-step plan for building the web frontend:
- 9 tasks with TDD approach
- Project setup with Dream
- Data layer (run_data, package_data) with 9 unit tests
- HTML layout and view modules
- Dashboard, Packages, and Runs pages
- Admin guide updates

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Design for day10-web, a status dashboard for package maintainers
and operators. Key decisions:

- Separate service reading day10's output directories
- OCaml with Dream web framework
- Server-rendered HTML with minimal JS
- Pages: Dashboard, Package list/detail, Run history/detail
- Dependency graph exploration (both directions)
- Public read-only, no authentication
- No database (filesystem as source of truth)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Covers:
- Prerequisites and installation
- Directory structure
- Basic and batch usage
- Production setup (systemd, cron, webhooks, nginx)
- Monitoring (logs, summary.json, disk usage)
- Maintenance (cache, GC, epoch transitions)
- Troubleshooting common issues
- Architecture overview

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>