pierrelf.com / we
web engine - experimental web browser
fork atom
we / PLAN.md
at main 7.8 kB view raw view code
pierrelf.com Initialize workspace: 15-crate browser engine scaffold
e25a0141

we — Development Plan#

A from-scratch web browser engine in pure Rust. macOS ARM only. Zero external crate dependencies.

Phase 1: Platform Layer#

Goal: Open a window on macOS, draw pixels, handle input events.

  • Objective-C runtime FFI (objc_msgSend, objc_getClass, sel_registerName, class_addMethod, objc_allocateClassPair/registerClassPair)
  • Rust wrappers: msg_send! macro, Class, Sel, Id types
  • CoreFoundation minimal bindings (CFString, CFRetain/CFRelease)
  • AppKit FFI: NSApplication, NSWindow, NSView, NSAutoreleasePool
  • CoreGraphics rendering surface: CGContext, CGBitmapContext, CGColorSpace
  • Custom NSView subclass overriding drawRect:
  • Basic event handling: window close, resize, key/mouse events
  • cargo run -p browser opens a blank window with a colored rectangle

Phase 2: Pure Rust Font Engine#

Goal: Parse system fonts and rasterize glyphs without CoreText.

  • OTF/TTF parser: read cmap, glyf, loca, head, hhea, hmtx, maxp, OS/2, name tables
  • Glyph outline extraction (TrueType quadratic beziers)
  • Rasterizer: scanline fill of glyph outlines to grayscale bitmaps
  • Basic shaping: horizontal advance, kern table lookups
  • Glyph cache (HashMap of rasterized bitmaps)
  • Load fonts from /System/Library/Fonts/ or /Library/Fonts/

Phase 3: HTML Parsing + Basic DOM + Block Layout + Software Rendering#

Goal: Parse HTML, build a DOM, do basic block layout, render text to the window.

  • HTML5 tokenizer (per spec state machine)
  • Tree builder (subset: <html>, <head>, <body>, <p>, <h1>–<h6>, <div>, <span>, <a>, <br>, <pre>, <title>, text nodes)
  • DOM tree: Document, Element, Text, Comment node types
  • Basic block layout: stack blocks vertically, wrap text into lines
  • Software renderer: paint text glyphs and block backgrounds into CG bitmap
  • Display painted bitmap in the AppKit window

Test milestone: Pass html5lib tokenizer tests.

Phase 4: CSS Parsing + Selector Matching + Cascade + Inline/Text Layout#

Goal: Parse CSS, match selectors, resolve styles, lay out inline content.

  • CSS tokenizer (per spec)
  • CSS parser: selectors, declarations, @-rules (at least @media, @import)
  • Selector matching: type, class, ID, attribute, combinators (descendant, child, sibling)
  • Cascade: origin, specificity, source order
  • Computed style resolution
  • Inline layout: inline boxes, line boxes, text wrapping, vertical alignment
  • Handle <style> elements and style attributes

Test milestone: Pass Acid1.

Phase 5: Pure Rust Crypto#

Goal: Implement the cryptographic primitives needed for TLS 1.3.

  • SHA-256, SHA-384, SHA-512 (FIPS 180-4)
  • HMAC (RFC 2104)
  • HKDF (RFC 5869)
  • AES-128-GCM, AES-256-GCM (NIST SP 800-38D)
  • ChaCha20-Poly1305 (RFC 8439)
  • X25519 key exchange (RFC 7748)
  • RSA PKCS#1 v1.5 signature verification (RFC 8017)
  • ECDSA signature verification (P-256, P-384) (FIPS 186-4)
  • ASN.1 DER parser
  • X.509 certificate parsing and chain validation

Phase 6: Pure Rust TLS 1.3 + TCP + DNS + HTTP/1.1 + URL Parser#

Goal: Fetch resources over HTTPS.

  • WHATWG URL parser (full spec compliance)
  • DNS stub resolver (UDP, A/AAAA records, system resolver config)
  • TCP socket wrapper (std::net)
  • TLS 1.3 client (RFC 8446): handshake, record layer, key schedule, certificate verification
  • HTTP/1.1 client: request/response parsing, chunked transfer encoding, keep-alive
  • Connection pooling
  • Content-Length and Transfer-Encoding handling

Phase 7: Image Decoders#

Goal: Decode common image formats in pure Rust.

  • DEFLATE decompressor (RFC 1951)
  • zlib wrapper (RFC 1950)
  • PNG decoder (RFC 2083): IHDR, IDAT, PLTE, tRNS, interlacing
  • JPEG decoder: baseline DCT, Huffman, JFIF/EXIF markers
  • GIF decoder: LZW, frames, transparency
  • Pixel format conversion (RGBA8)

Phase 8: Resource Loading + Character Encoding + Real Page Loading#

Goal: Load and render real web pages.

  • WHATWG Encoding Standard: UTF-8, UTF-16, ISO-8859-1, Windows-1252, and other legacy encodings
  • Encoding sniffing (BOM, HTTP header, meta tag, prescan)
  • Resource loader: fetch HTML, CSS, images via net crate
  • <link rel="stylesheet"> and <img> loading
  • Data URLs
  • about:blank

Phase 9: Advanced Layout#

Goal: Handle modern CSS layout modes.

  • Positioned layout: relative, absolute, fixed, sticky
  • Flexbox (CSS Flexible Box Layout Level 1)
  • Floats and clear
  • Overflow and scrolling (scroll bars, overflow: auto/scroll/hidden)
  • display: none, visibility: hidden
  • box-sizing, margin collapsing
  • Viewport units, percentage resolution

Test milestone: Pass Acid2.

Phase 10: JavaScript Engine#

Goal: Execute JavaScript with a JIT-ready architecture.

  • Lexer/tokenizer (ECMAScript 2024)
  • Parser → AST
  • Bytecode compiler: register-based bytecode format designed for JIT
  • Register-based VM: instruction dispatch, operand stack
  • Garbage collector: tri-color mark-and-sweep
  • Core built-ins: Object, Array, String, Number, Boolean, Function, Math, Date, RegExp, JSON, Error, Map, Set, Promise, Symbol
  • typeof, instanceof, prototype chain, closures, this binding
  • var/let/const scoping
  • Iterators, generators, async/await

Test milestone: Pass Test262 core language subset.

Phase 11: DOM-JS Bindings + Events + Timers + Fetch API#

Goal: Connect JavaScript to the DOM and web APIs.

  • DOM bindings: document.getElementById, querySelector, createElement, appendChild, textContent, innerHTML
  • Event system: addEventListener, removeEventListener, dispatchEvent, bubbling/capturing
  • setTimeout, setInterval, requestAnimationFrame
  • Fetch API (backed by net crate)
  • Console API (console.log, console.error, console.warn)
  • <script> element loading and execution (defer, async)

Test milestone: Begin WPT pass rate tracking.

Phase 12: Metal GPU Compositor#

Goal: GPU-accelerated rendering.

  • Metal device/command queue setup via platform crate
  • Texture atlas for glyph cache
  • Display list → Metal render passes
  • Compositing layers (for transforms, opacity, will-change)
  • Double-buffered presentation
  • Fallback to software rendering when Metal unavailable

Phase 13: Advanced CSS#

Goal: Modern CSS features.

  • CSS Grid Layout (Level 1)
  • CSS Animations (@keyframes, animation-*)
  • CSS Transitions (transition-*)
  • CSS Custom Properties (--var, var())
  • @font-face (load and parse web fonts via text + net)
  • calc(), min(), max(), clamp()
  • Media queries (screen width, prefers-color-scheme)
  • SVG basics: <svg>, <rect>, <circle>, <path>, <text>

Phase 14: Security + Storage#

Goal: Web security model and client-side storage.

  • Same-Origin Policy enforcement
  • CORS (preflight, simple requests, credentialed requests)
  • Content Security Policy (CSP Level 2)
  • Cookie jar (Set-Cookie parsing, Cookie header, SameSite, Secure, HttpOnly)
  • localStorage / sessionStorage
  • IndexedDB (basic subset)
  • <iframe> isolation and postMessage
  • Referrer Policy

Phase 15: Performance#

Goal: Production-grade performance.

  • Baseline JIT compiler: bytecode → AArch64 machine code
  • Inline caches for property access
  • Incremental layout (dirty bit propagation)
  • HTTP/2 (HPACK, multiplexed streams, flow control)
  • Speculative HTML parsing (off-main-thread tokenization)
  • Style sharing cache
  • Layer tree and damage tracking for minimal repaints
  • Memory profiling and optimization pass

Test Suites#

Suite Crate(s) Phase
html5lib-tests (tokenizer) html 3
Acid1 css, style, layout, render 4
Acid2 layout, render 9
Test262 (core language) js 10
WPT (progressive) all 11+