commits
Sync opam package metadata including x-maintenance-intent
and external dependency specifications.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add immich_auth library with session management (JWT + API key)
- Add CLI commands: auth, server, albums, faces
- Support .well-known/immich endpoint for API URL discovery
- Support multiple profiles for managing multiple servers
Connection pool and H2 improvements:
- Use fork_daemon for connection pool fibers to allow clean shutdown
- Use fork_daemon for H2 reader fiber to prevent switch blocking
- Handle Cancelled exceptions properly during cleanup
- Add await_cancel() for proper fiber lifecycle
OpenAPI code generator fix:
- Check nullable flag on all schema types, not just the fallback case
- Fields with nullable: true are now correctly treated as optional
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Replace unsafe Obj.magic casts with proper type-safe alternatives:
- conpool: Make protocol parameter required, add create_basic for simple
pools. The previous optional protocol with Obj.magic default was
fundamentally unsound as OCaml cannot have optional parameters that
change return types.
- publicsuffix: Add explicit id field to trie_node instead of using
Obj.magic to cast nodes to int for hashtable keys.
- yamlt: Add init_unknown_builder helper that properly handles GADT
refinement, returning () for Unknown_skip/Unknown_error cases where
builder=unit.
- jmap_brr: Use Jsont_brr.encode/decode Jsont.json instead of unsafe
casts between Jv.t and Jsont.json.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add explicit unix library dependency to dune files (html5rw, tomlt)
- Fix odoc heading levels ({0 -> {1) in imap subject.mli and thread.mli
- Fix code block indentation in subject.mli and h2_stream.mli
- Change unresolved module references to plain text (Bytesrw_unix,
Bytesrw_eio, Webfinger.Jrd.t)
- Fix @raise tags to use Error instead of Error.t
- Escape @mention/@mentions text in poe docs to avoid unknown tag warnings
- Remove unreachable flag_perm rule and redundant list_mailbox production
from IMAP grammar
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Conpool changes:
- Add connection-lifetime switch passed to protocol init_state, enabling
long-running fibers (e.g., HTTP/2 background reader)
- Add on_acquire/on_release protocol hooks for lazy fiber initialization
- Enforce max_idle_time via pc_last_used tracking
- Enforce max_connection_uses via pc_use_count tracking
- Track idle count (connections with no active users)
- Track error count (protocol failures vs normal lifecycle closes)
- Distinguish Unhealthy_error from Unhealthy_lifecycle in health checks
HTTP/2 changes:
- Enable true multiplexing: access_mode now returns Shared
- Start background reader fiber on first acquire (lazy init)
- Add on_goaway callback to start_reader for GOAWAY notifications
- Use concurrent request path instead of synchronous
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The previous implementation spawned a background reader fiber that
blocked on read_frame after the response completed, preventing the
switch from exiting. This caused requests to hang indefinitely.
Changes:
- Add request_sync for synchronous single-request operation
- one_request now uses request_sync without background fibers
- H2_conpool_handler uses request_sync to avoid fiber lifecycle issues
- Connection pool uses Exclusive mode (no multiplexing for now)
- Keep concurrent request infrastructure for future multiplexing support
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
When Conpool reuses a TCP connection for a second HTTP/2 request, the
code was attempting to send the HTTP/2 connection preface again, which
is invalid - you can only send the preface once per TCP connection.
This caused requests to hang.
The fix explicitly closes the flow after HTTP/2 request completion,
preventing Conpool from returning it to the idle pool. This is less
efficient than proper HTTP/2 multiplexing but ensures correctness.
Proper HTTP/2 connection reuse would require architectural changes to
tie H2_client state to TCP connections at the Conpool level.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The H2_adapter was caching HTTP/2 client state (including flow control
windows) independently from Conpool's TCP connection management. When
Conpool provided a new TCP connection, the cached H2_client would have
stale flow control windows from the previous connection. If the server
sent WINDOW_UPDATE frames, the accumulated window could exceed 2^31-1,
causing "Connection window overflow" errors.
The fix removes the caching layer from H2_adapter entirely. Each request
now creates a fresh H2_client and performs a new handshake. For proper
HTTP/2 multiplexing with connection reuse, the connection management
needs to be integrated at the Conpool level rather than in H2_adapter.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
HTTP/2 requires all header names to be lowercase (RFC 9113). Previously,
the library stored headers with canonical HTTP/1.x capitalization (e.g.,
"Accept-Encoding"), causing HTTP/2 requests to fail with "Header name
contains uppercase" errors.
Since HTTP/1.x headers are case-insensitive (RFC 9110), storing them in
lowercase is equally valid for both protocols. This change:
- Updates Headers.add and Headers.set to store lowercase names
- Removes uppercase validation from validate_h2_user_headers
- Removes redundant headers_to_h2/headers_of_h2 wrappers from h2_adapter
(now just uses Headers.to_list/of_list directly)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Retry-After fix (RFC 9110 Section 10.2.3):
- Implement actual HTTP-date parsing in retry_after_to_seconds using
Http_date.parse instead of returning None with a TODO comment
- Compute time difference from provided 'now' timestamp
- Return 0 for past dates (clamped to non-negative)
- Add 3 new tests for date-to-seconds conversion
HTTP/2 Connection Pooling Plan:
- Document the architectural conflict between Conpool (1 conn = 1 req)
and HTTP/2 multiplexing (1 conn = N streams)
- Propose H2_connection_pool module with proper stream slot management
- Define 5-phase implementation plan from low-risk fixes to optimizations
- Include data structures, API changes, and testing strategy
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add validate_h2_user_headers function to validate user-provided headers
for HTTP/2 compliance before pseudo-headers are added:
- Rejects pseudo-headers in user input (they should not provide them)
- Validates no uppercase in header names (RFC 9113 Section 8.2)
- Validates no connection-specific headers (RFC 9113 Section 8.2.2)
- Validates TE header only contains "trailers" if present
Call this validation in h2_adapter.ml for both request() and one_request()
functions to ensure HTTP/2 header constraints are enforced.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Same-origin check (RFC 6454):
- Include port in origin comparison, not just scheme and host
- http://example.com:80 and http://example.com:8080 are now different origins
Digest authentication (RFC 7616):
- Reject unknown algorithms instead of silent MD5 fallback
- Implement -sess algorithm variants (MD5-sess, SHA-256-sess)
with proper session key derivation: HA1 = hash(hash(u:r:p):nonce:cnonce)
HTTP methods (RFC 9110):
- Add request_body_semantics function with Body_required/Optional/Forbidden
- DELETE, OPTIONS, GET now correctly have Body_optional semantics
- Deprecate has_request_body in favor of the more accurate new function
Status codes:
- 501 Not Implemented and 505 HTTP Version Not Supported are no longer
marked as retryable (they indicate permanent conditions)
HTTP/2 (RFC 9113):
- Add SETTINGS_NO_RFC7540_PRIORITIES (0x09) to disable deprecated priorities
- Validate :protocol pseudo-header requires CONNECT method (RFC 8441)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
HTTP Message Signatures now use an explicit Eio clock for time validation
instead of Ptime_clock.now(), making the code testable and consistent with
Eio's capability-passing design.
Time validations now performed:
- Signatures with `expires` in the past are rejected
- Signatures with `created` in the future (beyond 60s clock skew) are rejected
- If `max_age` is specified and `created` is older, signature is rejected
API changes:
- sign/sign_with_digest now require ~clock parameter
- verify/verify_all now require ~clock parameter
- Auth.apply_signature now requires ~clock parameter
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
HTTP/2 Implementation:
- Add h2_frame module for RFC 7540 frame parsing/serialization
- Add h2_hpack module for RFC 7541 header compression (HPACK)
- Add h2_stream for HTTP/2 stream state machine
- Add h2_connection for multiplexed connection management
- Add h2_client high-level API matching HTTP/1.1 interface
- Add h2_adapter for protocol version abstraction
URI Library Migration:
- Replace custom Huri.t with Uri.t from uri opam package
- Keep Huri.write for efficient Buf_write serialization
- Remove Uri module shadowing from requests and apubt libraries
- Use Uri.* functions directly throughout codebase
Requests Library Reorganization:
- core/: fundamental types (body, headers, status, method, error)
- features/: optional functionality (auth, cache, retry, signature)
- h1/: HTTP/1.1 client implementation
- h2/: HTTP/2 client implementation
- parsing/: header and structured field parsers
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
ocaml-requests: Add automatic suppression of tls.tracing source after
first TLS connection. The source is created lazily by ocaml-tls, so
early logging setup misses it. Now suppressed to Warning level after
first connection unless explicitly set to Debug.
ocaml-zulip: Log full URL with query params in debug output instead of
just the path, making it easier to debug API issues.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The .mli file declared `module Uri = Uri` but the implementation
was missing it, causing a build error.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
TODO is to improve the SF implementation in the future
- Fix ambiguous {!decode} and {!connection} references by using val- prefix
- Add section IDs to header_parsing.mli and fix cross-references to sections
- Escape curly braces in documentation examples to avoid bad markup errors
- Fix unpaired brace in URL reference in headers.mli
- Indent verbatim block content in mqtte_cmd.mli
Unresolved references to external libraries (Jsont, Tomlt_bytesrw, etc.)
are intentionally kept as cross-references for future resolution.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Implement three new header parsing modules in header_parsing.ml:
Cache-Status (RFC 9211):
- Parse cache status entries with hit/fwd/stored/ttl parameters
- Support forward reasons: uri-miss, vary-miss, stale, bypass, etc.
- Helper functions for checking hits and getting forward reasons
Content-Digest / Repr-Digest (RFC 9530):
- Parse and generate digest headers with :base64: format
- Compute SHA-256 and SHA-512 digests using digestif
- Validate content against provided digests
- Get strongest available digest (prefer SHA-512)
Strict-Transport-Security (RFC 6797):
- Parse HSTS directives (max-age, includeSubDomains, preload)
- Case-insensitive directive matching
- Preset configurations for common use cases
- Helper to check if HSTS is effectively enabled
Add 20 new tests covering all three header types, bringing the
header parsing test suite to 44 tests total.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add printf-style format string variants of error constructors to reduce
boilerplate when error messages need interpolation:
- invalid_requestf, invalid_redirectf, invalid_urlf
- proxy_errorf, tls_handshake_failedf, tcp_connect_failedf
Also add non-format convenience constructors:
- tls_handshake_failed, tcp_connect_failed
Update callers in http_read.ml, proxy_tunnel.ml, retry.ml, and
redirect.ml to use the new format functions, reducing verbosity from:
raise (Error.err (Error.Invalid_request {
reason = Printf.sprintf "Invalid: %s" x
}))
To:
raise (Error.invalid_requestf "Invalid: %s" x)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add ~45 new standard headers to header_name.ml organized by RFC/category
(CORS, security, WebSocket, authentication, digest headers)
- Add header category lists and predicates for header classification
- Create header_parsing.ml/mli with RFC 9110 header value parsing:
- Content-Range parsing for partial content (206 responses)
- If-Range parsing (ETag vs Last-Modified detection)
- Allow header method list parsing
- Authentication-Info parsing for Digest auth nextnonce
- Retry-After and Accept-Ranges parsing
- Create websocket.ml/mli with RFC 6455 handshake support:
- Sec-WebSocket-Key generation (16 random bytes, base64)
- Sec-WebSocket-Accept computation/validation (SHA-1 + GUID)
- Protocol and extension negotiation
- Upgrade headers helper and response validation
- Add IANA HTTP fields registry CSV to specs folder for reference
- Add comprehensive test suites (50 tests total):
- test_header_parsing.ml: 24 tests
- test_websocket.ml: 26 tests
- Export new modules from requests.ml/mli
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Remove rarely-used functions and duplicates from the public interface
while preserving all functionality needed for HTTP clients/servers:
- Remove pct_encoder type and constructor (advanced encoding customization)
- Remove pp_hum (alias for pp), with_uri (use individual setters)
- Remove singleton variants: with_query', add_query_params', get_query_param'
- Remove user/password/with_password (use userinfo/with_userinfo)
- Remove host_with_default, verbatim_query, query_of_encoded, canonicalize
Internal implementation uses wrapper functions to hide optional
pct_encoder parameters from the simplified public interface.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Host header improvements:
- Add Host header validation against URI authority (warns on mismatch)
- Add CONNECT request support with authority-form (host:port) per RFC 9112 Section 3.2.3
TE header support:
- Add Headers.te and Headers.te_trailers functions per RFC 9110 Section 10.1.4
Method semantics enforcement:
- Add strict_method_semantics option to Retry.config
- When enabled, raises error on non-idempotent retry attempts
- Enhanced debug logging for method property violations
Update SPEC-TODO.md:
- Mark Host validation, TE header, trailer headers as complete
- Mark Expect 100-continue timeout, method properties as complete
- All P2/P3 items now complete except URI normalization and IRI support
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Mark URI library inlining as complete (already done in lib/uri.ml)
- Update compliance summary: RFC 3986 now 95%+
- Mark implementation phases 1-4 as complete
- Merge TODO.md items into SPEC-TODO.md Section 8 (Feature Roadmap)
- Remove redundant TODO.md file
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add negative Content-Length validation per RFC 9110 Section 8.6
- Add Transfer-Encoding validation for bodiless responses per RFC 9112 Section 6.1
- Logs warning when TE present in HEAD, 1xx, 204, 304 responses
- New validate_no_transfer_encoding function exposed in http_read.mli
- Add optional method_ parameter to response_stream for HEAD detection
- Update SPEC-TODO.md to mark completed P0, P1, and P2 items
- Update compliance summary percentages to reflect improvements
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
When users provide a URL like "anil.recoil.org" without a scheme,
normalize it to "http://anil.recoil.org" to avoid invalid redirect
URLs being formed (e.g., "///anil.recoil.org").
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Introduces a new Header_name module that provides type-safe header name
handling with polymorphic variants for common HTTP headers. This improves
type safety and reduces string-based errors when working with headers.
Key changes:
- Add lib/header_name.ml and lib/header_name.mli with typed header names
- Refactor Headers module to use Header_name.t for core operations
- Update all modules to use typed header names where applicable
- Maintain string-based operations for wire format compatibility
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The path_of_encoded function was incorrectly ordering "/" separators
in the accumulator. When building the path list in reverse and then
reversing it, "/" separators ended up at the wrong positions.
For example, "/status/200" was being parsed as ["/"; "/"; "status"; "200"]
instead of the correct ["/"; "status"; "/"; "200"]. This caused URLs
like http://localhost:8088/status/200 to be serialized incorrectly as
http://localhost:8088//status200, resulting in 404 errors.
The fix changes the order of prepending to the accumulator so that
"/" appears AFTER the segment when the list is reversed.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Security (P0):
- Add bare CR validation to prevent HTTP request smuggling (RFC 9112 Section 2.2)
- Add chunk size overflow protection (max 16 hex digits)
URI Library (RFC 3986):
- Inline URI module with Eio.Buf_read parsers replacing Angstrom
- Add Pct module for percent encoding/decoding
- Add Path module with dot segment removal per RFC 3986 Section 5.2.4
- Add Query module for query string parsing
- Add Absolute_http submodule for HTTP-specific URI handling
HTTP Caching (RFC 9111):
- Add age calculation per RFC 9111 Section 4.2.3
- Add heuristic freshness computation per RFC 9111 Section 4.2.2
- Add in-memory cache module with thread-safe operations
- Support Vary header matching and validation headers
Authentication (RFC 7616, RFC 6750):
- Add auth-int qop support with body hashing for Digest auth
- Add userhash field to digest_challenge per RFC 7616
- Add Bearer form authentication per RFC 6750 Section 2.2
- Add digest_is_stale for stale nonce handling
Transfer-Encoding (RFC 9112 Section 6.1):
- Add multi-encoding validation (chunked must be final)
- Parse comma-separated encoding list
Connection Headers (RFC 9110 Section 7.6.1):
- Add hop-by-hop header parsing and removal
- Add connection_close and connection_keep_alive helpers
Trailer Headers (RFC 9112 Section 7.1.2):
- Add proper trailer parsing instead of skipping
- Filter forbidden trailer headers per RFC 9110 Section 6.5.1
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
High priority:
- 303 redirect: change POST/PUT/DELETE/PATCH to GET (RFC 9110 Section 15.4.4)
- obs-fold header handling: merge continuation lines (RFC 9112 Section 5.2)
- Basic auth: validate username for colons/control chars (RFC 7617 Section 2)
Medium priority:
- Close-delimited body: read until EOF when no length (RFC 9112 Section 6.3)
- Retry-After: use Http_date.parse for IMF-fixdate (RFC 9110 Section 10.2.3)
- 407 proxy auth: auto-retry with Proxy-Authorization (RFC 7235 Section 3.2)
- 417 Expectation Failed: retry without Expect header (RFC 9110 Section 10.1.1)
Low priority:
- Asterisk-form OPTIONS: support OPTIONS * requests (RFC 9112 Section 3.2.4)
- Accept-Language: add header builder function (RFC 9110 Section 12.5.4)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Replace two parameters (expect_100_continue:bool, expect_100_continue_threshold:int64)
with a single polymorphic variant parameter:
type config = [
| `Disabled (* Never use 100-continue *)
| `Always (* Always use for requests with bodies *)
| `Threshold of int64 (* Use for bodies >= n bytes *)
]
Default: `Threshold 1_048_576L (1MB) - same behavior as before.
This provides a cleaner API while maintaining full flexibility.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Body.jsonv: encode typed values to JSON using Jsont.t codecs
- Response.jsonv: decode JSON responses to typed values using Jsont.t codecs
These functions provide type-safe JSON serialization/deserialization,
complementing the existing untyped Jsont.json-based functions.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Fix SHA algorithm support in Digest auth: properly support MD5, SHA-256,
SHA-512; reject SHA-512-256 with clear error (requires special IVs)
- Add nonce count tracking for Digest auth replay protection (RFC 7616)
- Fix Content-Length parsing to handle malformed values safely
- Use is_chunked_encoding helper consistently in http_read.ml
- Extract write_body_to_flow helper to reduce duplication in http_client.ml
- Add SOCKS5 proxy validation (not yet implemented, raises clear error)
- Add error convenience constructors for cleaner error raising
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Create redirect.ml for cross-origin detection and sensitive header stripping
- Create tls_config.ml for TLS client configuration creation
- Factor out decompress_with helper in http_client.ml (84->45 lines)
- Extract host_matches_pattern in proxy.ml for NO_PROXY pattern matching
- Simplify timeout.ml pretty-printer using Option.map and List.filter_map
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add immich_auth library with session management (JWT + API key)
- Add CLI commands: auth, server, albums, faces
- Support .well-known/immich endpoint for API URL discovery
- Support multiple profiles for managing multiple servers
Connection pool and H2 improvements:
- Use fork_daemon for connection pool fibers to allow clean shutdown
- Use fork_daemon for H2 reader fiber to prevent switch blocking
- Handle Cancelled exceptions properly during cleanup
- Add await_cancel() for proper fiber lifecycle
OpenAPI code generator fix:
- Check nullable flag on all schema types, not just the fallback case
- Fields with nullable: true are now correctly treated as optional
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Replace unsafe Obj.magic casts with proper type-safe alternatives:
- conpool: Make protocol parameter required, add create_basic for simple
pools. The previous optional protocol with Obj.magic default was
fundamentally unsound as OCaml cannot have optional parameters that
change return types.
- publicsuffix: Add explicit id field to trie_node instead of using
Obj.magic to cast nodes to int for hashtable keys.
- yamlt: Add init_unknown_builder helper that properly handles GADT
refinement, returning () for Unknown_skip/Unknown_error cases where
builder=unit.
- jmap_brr: Use Jsont_brr.encode/decode Jsont.json instead of unsafe
casts between Jv.t and Jsont.json.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add explicit unix library dependency to dune files (html5rw, tomlt)
- Fix odoc heading levels ({0 -> {1) in imap subject.mli and thread.mli
- Fix code block indentation in subject.mli and h2_stream.mli
- Change unresolved module references to plain text (Bytesrw_unix,
Bytesrw_eio, Webfinger.Jrd.t)
- Fix @raise tags to use Error instead of Error.t
- Escape @mention/@mentions text in poe docs to avoid unknown tag warnings
- Remove unreachable flag_perm rule and redundant list_mailbox production
from IMAP grammar
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Conpool changes:
- Add connection-lifetime switch passed to protocol init_state, enabling
long-running fibers (e.g., HTTP/2 background reader)
- Add on_acquire/on_release protocol hooks for lazy fiber initialization
- Enforce max_idle_time via pc_last_used tracking
- Enforce max_connection_uses via pc_use_count tracking
- Track idle count (connections with no active users)
- Track error count (protocol failures vs normal lifecycle closes)
- Distinguish Unhealthy_error from Unhealthy_lifecycle in health checks
HTTP/2 changes:
- Enable true multiplexing: access_mode now returns Shared
- Start background reader fiber on first acquire (lazy init)
- Add on_goaway callback to start_reader for GOAWAY notifications
- Use concurrent request path instead of synchronous
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The previous implementation spawned a background reader fiber that
blocked on read_frame after the response completed, preventing the
switch from exiting. This caused requests to hang indefinitely.
Changes:
- Add request_sync for synchronous single-request operation
- one_request now uses request_sync without background fibers
- H2_conpool_handler uses request_sync to avoid fiber lifecycle issues
- Connection pool uses Exclusive mode (no multiplexing for now)
- Keep concurrent request infrastructure for future multiplexing support
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
When Conpool reuses a TCP connection for a second HTTP/2 request, the
code was attempting to send the HTTP/2 connection preface again, which
is invalid - you can only send the preface once per TCP connection.
This caused requests to hang.
The fix explicitly closes the flow after HTTP/2 request completion,
preventing Conpool from returning it to the idle pool. This is less
efficient than proper HTTP/2 multiplexing but ensures correctness.
Proper HTTP/2 connection reuse would require architectural changes to
tie H2_client state to TCP connections at the Conpool level.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The H2_adapter was caching HTTP/2 client state (including flow control
windows) independently from Conpool's TCP connection management. When
Conpool provided a new TCP connection, the cached H2_client would have
stale flow control windows from the previous connection. If the server
sent WINDOW_UPDATE frames, the accumulated window could exceed 2^31-1,
causing "Connection window overflow" errors.
The fix removes the caching layer from H2_adapter entirely. Each request
now creates a fresh H2_client and performs a new handshake. For proper
HTTP/2 multiplexing with connection reuse, the connection management
needs to be integrated at the Conpool level rather than in H2_adapter.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
HTTP/2 requires all header names to be lowercase (RFC 9113). Previously,
the library stored headers with canonical HTTP/1.x capitalization (e.g.,
"Accept-Encoding"), causing HTTP/2 requests to fail with "Header name
contains uppercase" errors.
Since HTTP/1.x headers are case-insensitive (RFC 9110), storing them in
lowercase is equally valid for both protocols. This change:
- Updates Headers.add and Headers.set to store lowercase names
- Removes uppercase validation from validate_h2_user_headers
- Removes redundant headers_to_h2/headers_of_h2 wrappers from h2_adapter
(now just uses Headers.to_list/of_list directly)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Retry-After fix (RFC 9110 Section 10.2.3):
- Implement actual HTTP-date parsing in retry_after_to_seconds using
Http_date.parse instead of returning None with a TODO comment
- Compute time difference from provided 'now' timestamp
- Return 0 for past dates (clamped to non-negative)
- Add 3 new tests for date-to-seconds conversion
HTTP/2 Connection Pooling Plan:
- Document the architectural conflict between Conpool (1 conn = 1 req)
and HTTP/2 multiplexing (1 conn = N streams)
- Propose H2_connection_pool module with proper stream slot management
- Define 5-phase implementation plan from low-risk fixes to optimizations
- Include data structures, API changes, and testing strategy
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add validate_h2_user_headers function to validate user-provided headers
for HTTP/2 compliance before pseudo-headers are added:
- Rejects pseudo-headers in user input (they should not provide them)
- Validates no uppercase in header names (RFC 9113 Section 8.2)
- Validates no connection-specific headers (RFC 9113 Section 8.2.2)
- Validates TE header only contains "trailers" if present
Call this validation in h2_adapter.ml for both request() and one_request()
functions to ensure HTTP/2 header constraints are enforced.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Same-origin check (RFC 6454):
- Include port in origin comparison, not just scheme and host
- http://example.com:80 and http://example.com:8080 are now different origins
Digest authentication (RFC 7616):
- Reject unknown algorithms instead of silent MD5 fallback
- Implement -sess algorithm variants (MD5-sess, SHA-256-sess)
with proper session key derivation: HA1 = hash(hash(u:r:p):nonce:cnonce)
HTTP methods (RFC 9110):
- Add request_body_semantics function with Body_required/Optional/Forbidden
- DELETE, OPTIONS, GET now correctly have Body_optional semantics
- Deprecate has_request_body in favor of the more accurate new function
Status codes:
- 501 Not Implemented and 505 HTTP Version Not Supported are no longer
marked as retryable (they indicate permanent conditions)
HTTP/2 (RFC 9113):
- Add SETTINGS_NO_RFC7540_PRIORITIES (0x09) to disable deprecated priorities
- Validate :protocol pseudo-header requires CONNECT method (RFC 8441)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
HTTP Message Signatures now use an explicit Eio clock for time validation
instead of Ptime_clock.now(), making the code testable and consistent with
Eio's capability-passing design.
Time validations now performed:
- Signatures with `expires` in the past are rejected
- Signatures with `created` in the future (beyond 60s clock skew) are rejected
- If `max_age` is specified and `created` is older, signature is rejected
API changes:
- sign/sign_with_digest now require ~clock parameter
- verify/verify_all now require ~clock parameter
- Auth.apply_signature now requires ~clock parameter
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
HTTP/2 Implementation:
- Add h2_frame module for RFC 7540 frame parsing/serialization
- Add h2_hpack module for RFC 7541 header compression (HPACK)
- Add h2_stream for HTTP/2 stream state machine
- Add h2_connection for multiplexed connection management
- Add h2_client high-level API matching HTTP/1.1 interface
- Add h2_adapter for protocol version abstraction
URI Library Migration:
- Replace custom Huri.t with Uri.t from uri opam package
- Keep Huri.write for efficient Buf_write serialization
- Remove Uri module shadowing from requests and apubt libraries
- Use Uri.* functions directly throughout codebase
Requests Library Reorganization:
- core/: fundamental types (body, headers, status, method, error)
- features/: optional functionality (auth, cache, retry, signature)
- h1/: HTTP/1.1 client implementation
- h2/: HTTP/2 client implementation
- parsing/: header and structured field parsers
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
ocaml-requests: Add automatic suppression of tls.tracing source after
first TLS connection. The source is created lazily by ocaml-tls, so
early logging setup misses it. Now suppressed to Warning level after
first connection unless explicitly set to Debug.
ocaml-zulip: Log full URL with query params in debug output instead of
just the path, making it easier to debug API issues.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Fix ambiguous {!decode} and {!connection} references by using val- prefix
- Add section IDs to header_parsing.mli and fix cross-references to sections
- Escape curly braces in documentation examples to avoid bad markup errors
- Fix unpaired brace in URL reference in headers.mli
- Indent verbatim block content in mqtte_cmd.mli
Unresolved references to external libraries (Jsont, Tomlt_bytesrw, etc.)
are intentionally kept as cross-references for future resolution.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Implement three new header parsing modules in header_parsing.ml:
Cache-Status (RFC 9211):
- Parse cache status entries with hit/fwd/stored/ttl parameters
- Support forward reasons: uri-miss, vary-miss, stale, bypass, etc.
- Helper functions for checking hits and getting forward reasons
Content-Digest / Repr-Digest (RFC 9530):
- Parse and generate digest headers with :base64: format
- Compute SHA-256 and SHA-512 digests using digestif
- Validate content against provided digests
- Get strongest available digest (prefer SHA-512)
Strict-Transport-Security (RFC 6797):
- Parse HSTS directives (max-age, includeSubDomains, preload)
- Case-insensitive directive matching
- Preset configurations for common use cases
- Helper to check if HSTS is effectively enabled
Add 20 new tests covering all three header types, bringing the
header parsing test suite to 44 tests total.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add printf-style format string variants of error constructors to reduce
boilerplate when error messages need interpolation:
- invalid_requestf, invalid_redirectf, invalid_urlf
- proxy_errorf, tls_handshake_failedf, tcp_connect_failedf
Also add non-format convenience constructors:
- tls_handshake_failed, tcp_connect_failed
Update callers in http_read.ml, proxy_tunnel.ml, retry.ml, and
redirect.ml to use the new format functions, reducing verbosity from:
raise (Error.err (Error.Invalid_request {
reason = Printf.sprintf "Invalid: %s" x
}))
To:
raise (Error.invalid_requestf "Invalid: %s" x)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add ~45 new standard headers to header_name.ml organized by RFC/category
(CORS, security, WebSocket, authentication, digest headers)
- Add header category lists and predicates for header classification
- Create header_parsing.ml/mli with RFC 9110 header value parsing:
- Content-Range parsing for partial content (206 responses)
- If-Range parsing (ETag vs Last-Modified detection)
- Allow header method list parsing
- Authentication-Info parsing for Digest auth nextnonce
- Retry-After and Accept-Ranges parsing
- Create websocket.ml/mli with RFC 6455 handshake support:
- Sec-WebSocket-Key generation (16 random bytes, base64)
- Sec-WebSocket-Accept computation/validation (SHA-1 + GUID)
- Protocol and extension negotiation
- Upgrade headers helper and response validation
- Add IANA HTTP fields registry CSV to specs folder for reference
- Add comprehensive test suites (50 tests total):
- test_header_parsing.ml: 24 tests
- test_websocket.ml: 26 tests
- Export new modules from requests.ml/mli
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Remove rarely-used functions and duplicates from the public interface
while preserving all functionality needed for HTTP clients/servers:
- Remove pct_encoder type and constructor (advanced encoding customization)
- Remove pp_hum (alias for pp), with_uri (use individual setters)
- Remove singleton variants: with_query', add_query_params', get_query_param'
- Remove user/password/with_password (use userinfo/with_userinfo)
- Remove host_with_default, verbatim_query, query_of_encoded, canonicalize
Internal implementation uses wrapper functions to hide optional
pct_encoder parameters from the simplified public interface.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Host header improvements:
- Add Host header validation against URI authority (warns on mismatch)
- Add CONNECT request support with authority-form (host:port) per RFC 9112 Section 3.2.3
TE header support:
- Add Headers.te and Headers.te_trailers functions per RFC 9110 Section 10.1.4
Method semantics enforcement:
- Add strict_method_semantics option to Retry.config
- When enabled, raises error on non-idempotent retry attempts
- Enhanced debug logging for method property violations
Update SPEC-TODO.md:
- Mark Host validation, TE header, trailer headers as complete
- Mark Expect 100-continue timeout, method properties as complete
- All P2/P3 items now complete except URI normalization and IRI support
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Mark URI library inlining as complete (already done in lib/uri.ml)
- Update compliance summary: RFC 3986 now 95%+
- Mark implementation phases 1-4 as complete
- Merge TODO.md items into SPEC-TODO.md Section 8 (Feature Roadmap)
- Remove redundant TODO.md file
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add negative Content-Length validation per RFC 9110 Section 8.6
- Add Transfer-Encoding validation for bodiless responses per RFC 9112 Section 6.1
- Logs warning when TE present in HEAD, 1xx, 204, 304 responses
- New validate_no_transfer_encoding function exposed in http_read.mli
- Add optional method_ parameter to response_stream for HEAD detection
- Update SPEC-TODO.md to mark completed P0, P1, and P2 items
- Update compliance summary percentages to reflect improvements
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Introduces a new Header_name module that provides type-safe header name
handling with polymorphic variants for common HTTP headers. This improves
type safety and reduces string-based errors when working with headers.
Key changes:
- Add lib/header_name.ml and lib/header_name.mli with typed header names
- Refactor Headers module to use Header_name.t for core operations
- Update all modules to use typed header names where applicable
- Maintain string-based operations for wire format compatibility
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The path_of_encoded function was incorrectly ordering "/" separators
in the accumulator. When building the path list in reverse and then
reversing it, "/" separators ended up at the wrong positions.
For example, "/status/200" was being parsed as ["/"; "/"; "status"; "200"]
instead of the correct ["/"; "status"; "/"; "200"]. This caused URLs
like http://localhost:8088/status/200 to be serialized incorrectly as
http://localhost:8088//status200, resulting in 404 errors.
The fix changes the order of prepending to the accumulator so that
"/" appears AFTER the segment when the list is reversed.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Security (P0):
- Add bare CR validation to prevent HTTP request smuggling (RFC 9112 Section 2.2)
- Add chunk size overflow protection (max 16 hex digits)
URI Library (RFC 3986):
- Inline URI module with Eio.Buf_read parsers replacing Angstrom
- Add Pct module for percent encoding/decoding
- Add Path module with dot segment removal per RFC 3986 Section 5.2.4
- Add Query module for query string parsing
- Add Absolute_http submodule for HTTP-specific URI handling
HTTP Caching (RFC 9111):
- Add age calculation per RFC 9111 Section 4.2.3
- Add heuristic freshness computation per RFC 9111 Section 4.2.2
- Add in-memory cache module with thread-safe operations
- Support Vary header matching and validation headers
Authentication (RFC 7616, RFC 6750):
- Add auth-int qop support with body hashing for Digest auth
- Add userhash field to digest_challenge per RFC 7616
- Add Bearer form authentication per RFC 6750 Section 2.2
- Add digest_is_stale for stale nonce handling
Transfer-Encoding (RFC 9112 Section 6.1):
- Add multi-encoding validation (chunked must be final)
- Parse comma-separated encoding list
Connection Headers (RFC 9110 Section 7.6.1):
- Add hop-by-hop header parsing and removal
- Add connection_close and connection_keep_alive helpers
Trailer Headers (RFC 9112 Section 7.1.2):
- Add proper trailer parsing instead of skipping
- Filter forbidden trailer headers per RFC 9110 Section 6.5.1
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
High priority:
- 303 redirect: change POST/PUT/DELETE/PATCH to GET (RFC 9110 Section 15.4.4)
- obs-fold header handling: merge continuation lines (RFC 9112 Section 5.2)
- Basic auth: validate username for colons/control chars (RFC 7617 Section 2)
Medium priority:
- Close-delimited body: read until EOF when no length (RFC 9112 Section 6.3)
- Retry-After: use Http_date.parse for IMF-fixdate (RFC 9110 Section 10.2.3)
- 407 proxy auth: auto-retry with Proxy-Authorization (RFC 7235 Section 3.2)
- 417 Expectation Failed: retry without Expect header (RFC 9110 Section 10.1.1)
Low priority:
- Asterisk-form OPTIONS: support OPTIONS * requests (RFC 9112 Section 3.2.4)
- Accept-Language: add header builder function (RFC 9110 Section 12.5.4)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Replace two parameters (expect_100_continue:bool, expect_100_continue_threshold:int64)
with a single polymorphic variant parameter:
type config = [
| `Disabled (* Never use 100-continue *)
| `Always (* Always use for requests with bodies *)
| `Threshold of int64 (* Use for bodies >= n bytes *)
]
Default: `Threshold 1_048_576L (1MB) - same behavior as before.
This provides a cleaner API while maintaining full flexibility.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Body.jsonv: encode typed values to JSON using Jsont.t codecs
- Response.jsonv: decode JSON responses to typed values using Jsont.t codecs
These functions provide type-safe JSON serialization/deserialization,
complementing the existing untyped Jsont.json-based functions.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Fix SHA algorithm support in Digest auth: properly support MD5, SHA-256,
SHA-512; reject SHA-512-256 with clear error (requires special IVs)
- Add nonce count tracking for Digest auth replay protection (RFC 7616)
- Fix Content-Length parsing to handle malformed values safely
- Use is_chunked_encoding helper consistently in http_read.ml
- Extract write_body_to_flow helper to reduce duplication in http_client.ml
- Add SOCKS5 proxy validation (not yet implemented, raises clear error)
- Add error convenience constructors for cleaner error raising
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Create redirect.ml for cross-origin detection and sensitive header stripping
- Create tls_config.ml for TLS client configuration creation
- Factor out decompress_with helper in http_client.ml (84->45 lines)
- Extract host_matches_pattern in proxy.ml for NO_PROXY pattern matching
- Simplify timeout.ml pretty-printer using Option.map and List.filter_map
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>