anil.recoil.org
A batteries included HTTP/1.1 client in OCaml
1{
2 "recommendations": [
3 {
4 "source_repo": "cpp-httplib",
5 "source_language": "C++",
6 "criticality": "medium",
7 "change_type": "feature",
8 "title": "Add progress callbacks for upload and download operations",
9 "description": "cpp-httplib provides progress callbacks that report bytes transferred during upload and download. This is useful for implementing progress bars, monitoring large transfers, and providing user feedback. The OCaml library should add similar functionality through optional callback parameters.",
10 "affected_files": [
11 "lib/one.ml",
12 "lib/one.mli",
13 "lib/requests.ml",
14 "lib/requests.mli",
15 "lib/body.ml"
16 ],
17 "rationale": "Progress monitoring is essential for user-facing applications that handle large file uploads/downloads. cpp-httplib implements this via UploadProgress and DownloadProgress callbacks that receive (current_bytes, total_bytes) and can cancel operations by returning false. The OCaml library currently streams data but doesn't provide progress hooks, making it difficult to build responsive UIs."
18 },
19 {
20 "source_repo": "cpp-httplib",
21 "source_language": "C++",
22 "criticality": "low",
23 "change_type": "feature",
24 "title": "Support for multipart form data streaming with content receivers",
25 "description": "cpp-httplib allows streaming processing of multipart form data fields as they arrive, rather than buffering everything in memory. This is critical for handling large file uploads efficiently. The OCaml library should add similar streaming support for multipart data.",
26 "affected_files": [
27 "lib/body.ml",
28 "lib/body.mli",
29 "lib/http_read.ml"
30 ],
31 "rationale": "cpp-httplib's HandlerWithContentReader allows processing multipart form data incrementally using callbacks for each field and its content. This prevents memory exhaustion when receiving large files. The OCaml library currently has Body.multipart for sending but lacks efficient streaming reception."
32 },
33 {
34 "source_repo": "cpp-httplib",
35 "source_language": "C++",
36 "criticality": "medium",
37 "change_type": "feature",
38 "title": "Add support for HTTP Range requests (byte ranges)",
39 "description": "cpp-httplib provides make_range_header() helper and automatic Range header handling for partial content requests. This enables resumable downloads and seeking in large files. The OCaml library should add Range request support with helpers for common range patterns.",
40 "affected_files": [
41 "lib/headers.ml",
42 "lib/headers.mli",
43 "lib/response.ml",
44 "lib/response.mli"
45 ],
46 "rationale": "Range requests (RFC 7233) are essential for resumable downloads, video streaming, and large file access. cpp-httplib supports multi-range requests like 'bytes=1-10,20-30' via make_range_header. The OCaml library currently has no built-in Range support, forcing users to manually construct headers and parse 206 Partial Content responses."
47 },
48 {
49 "source_repo": "cpp-httplib",
50 "source_language": "C++",
51 "criticality": "high",
52 "change_type": "enhancement",
53 "title": "Add dedicated error types for socket/file descriptor exhaustion",
54 "description": "cpp-httplib has specific error codes for ResourceExhaustion and ExceedMaxSocketDescriptorCount. These help diagnose resource limit issues vs network failures. The OCaml library should add similar granular resource error types.",
55 "affected_files": [
56 "lib/error.ml",
57 "lib/error.mli"
58 ],
59 "rationale": "cpp-httplib distinguishes between ResourceExhaustion (general resource issues) and ExceedMaxSocketDescriptorCount (too many file descriptors). The OCaml Error module has granular connection errors but no specific resource exhaustion types. This makes it harder to diagnose and handle resource limit errors differently from network failures."
60 },
61 {
62 "source_repo": "cpp-httplib",
63 "source_language": "C++",
64 "criticality": "medium",
65 "change_type": "enhancement",
66 "title": "Implement separate read and write timeout tracking",
67 "description": "cpp-httplib tracks read and write timeouts independently with microsecond precision, allowing fine-grained control over slow reads vs slow writes. The OCaml library should split timeout configuration to distinguish read/write operations.",
68 "affected_files": [
69 "lib/timeout.ml",
70 "lib/timeout.mli",
71 "lib/http_client.ml"
72 ],
73 "rationale": "cpp-httplib provides separate CLIENT_READ_TIMEOUT and CLIENT_WRITE_TIMEOUT defaults (300s and 5s respectively), recognizing that reads (waiting for server) often need longer timeouts than writes. The OCaml Timeout module already has connect/read/total fields but could benefit from separating write timeout, especially for large uploads."
74 },
75 {
76 "source_repo": "cpp-httplib",
77 "source_language": "C++",
78 "criticality": "low",
79 "change_type": "feature",
80 "title": "Add support for Server-Sent Events (SSE) streaming",
81 "description": "cpp-httplib has examples demonstrating SSE support for real-time server-to-client event streaming. The OCaml library should add helpers for SSE format parsing and event stream handling.",
82 "affected_files": [
83 "lib/response.ml",
84 "lib/response.mli"
85 ],
86 "rationale": "Server-Sent Events (text/event-stream) is a standard for server push over HTTP. cpp-httplib's examples show SSE server and client implementations. The OCaml library could add SSE parsing helpers to make it easier to consume event streams from APIs like OpenAI, Anthropic, etc."
87 },
88 {
89 "source_repo": "cpp-httplib",
90 "source_language": "C++",
91 "criticality": "medium",
92 "change_type": "enhancement",
93 "title": "Add configurable limits for URI length and header count",
94 "description": "cpp-httplib enforces REQUEST_URI_MAX_LENGTH (8192) and HEADER_MAX_COUNT (100) to prevent attacks. The OCaml library should make these limits configurable rather than using only max_header_size.",
95 "affected_files": [
96 "lib/response_limits.ml",
97 "lib/response_limits.mli",
98 "lib/http_read.ml"
99 ],
100 "rationale": "cpp-httplib defines separate limits for URI length (8192), header count (100), and header size (8192) as defense against malicious requests. The OCaml Response_limits module has max_header_size and max_header_count but doesn't enforce URI length limits. Adding URI length validation prevents memory exhaustion from pathologically long URLs."
101 },
102 {
103 "source_repo": "cpp-httplib",
104 "source_language": "C++",
105 "criticality": "low",
106 "change_type": "feature",
107 "title": "Add automatic path encoding for URL special characters",
108 "description": "cpp-httplib provides set_path_encode() to automatically encode special characters in URL paths (spaces, plus signs, etc.). The OCaml library should add similar automatic encoding with a toggle.",
109 "affected_files": [
110 "lib/requests.ml",
111 "lib/requests.mli"
112 ],
113 "rationale": "cpp-httplib's set_path_encode(true) automatically encodes spaces and special characters in URL paths, with set_path_encode(false) for pre-encoded paths. This prevents common bugs from unencoded URLs. The OCaml library relies on Uri module encoding but doesn't provide a convenience toggle for automatic path encoding."
114 },
115 {
116 "source_repo": "cpp-httplib",
117 "source_language": "C++",
118 "criticality": "high",
119 "change_type": "security",
120 "title": "Add support for trusted proxy headers (X-Forwarded-For validation)",
121 "description": "cpp-httplib provides set_trusted_proxies() to validate X-Forwarded-For headers and prevent IP spoofing. The OCaml library should add proxy trust configuration for secure header handling.",
122 "affected_files": [
123 "lib/requests.ml",
124 "lib/requests.mli",
125 "lib/headers.ml"
126 ],
127 "rationale": "cpp-httplib's set_trusted_proxies() prevents IP spoofing by validating which proxies can set X-Forwarded-For/X-Real-IP headers. Without this, malicious clients can forge their IP address. The OCaml library currently has no proxy trust mechanism, making it unsafe to rely on forwarded headers for security decisions."
128 },
129 {
130 "source_repo": "cpp-httplib",
131 "source_language": "C++",
132 "criticality": "medium",
133 "change_type": "enhancement",
134 "title": "Add structured logging with separate access and error loggers",
135 "description": "cpp-httplib separates set_logger() for successful requests from set_error_logger() for failures, following nginx/Apache patterns. The OCaml library should provide separate logging callbacks for success vs error paths.",
136 "affected_files": [
137 "lib/requests.ml",
138 "lib/requests.mli",
139 "lib/one.ml"
140 ],
141 "rationale": "cpp-httplib provides three logger types: access logger (successful requests), pre-compression logger (before compression), and error logger (failures with different signature taking Error+Request). This separation allows different log destinations and formats. The OCaml library uses Logs module but doesn't provide structured callback hooks for request/response logging."
142 },
143 {
144 "source_repo": "cpp-httplib",
145 "source_language": "C++",
146 "criticality": "low",
147 "change_type": "feature",
148 "title": "Add support for connection state polling during long requests",
149 "description": "cpp-httplib provides Request.is_connection_closed() to check if client disconnected during long-running operations. The OCaml library should add connection state checking for long operations.",
150 "affected_files": [
151 "lib/http_client.ml",
152 "lib/requests.ml"
153 ],
154 "rationale": "cpp-httplib allows checking req.is_connection_closed() in handler loops to detect client disconnection and abort expensive operations early. This prevents wasted work when clients disconnect. The OCaml library could expose connection state through the Eio flow abstraction."
155 },
156 {
157 "source_repo": "cpp-httplib",
158 "source_language": "C++",
159 "criticality": "medium",
160 "change_type": "enhancement",
161 "title": "Add configurable idle interval for request processing",
162 "description": "cpp-httplib provides set_idle_interval() to control polling interval during request processing. The OCaml library should add similar tuning for CPU usage vs responsiveness trade-off.",
163 "affected_files": [
164 "lib/requests.ml",
165 "lib/requests.mli"
166 ],
167 "rationale": "cpp-httplib's IDLE_INTERVAL_USECOND (1000μs on Windows, 0 on Unix) controls polling frequency during I/O operations, balancing CPU usage and responsiveness. The OCaml library relies on Eio's scheduler but doesn't expose idle interval tuning, which could be useful for embedded or high-frequency scenarios."
168 },
169 {
170 "source_repo": "cpp-httplib",
171 "source_language": "C++",
172 "criticality": "low",
173 "change_type": "enhancement",
174 "title": "Add default User-Agent header with library version",
175 "description": "Many HTTP libraries set a default User-Agent identifying the library and version. The OCaml library should add a default User-Agent like 'ocaml-requests/VERSION' that can be overridden.",
176 "affected_files": [
177 "lib/requests.ml",
178 "lib/headers.ml"
179 ],
180 "rationale": "cpp-httplib doesn't enforce User-Agent but most HTTP clients provide defaults (e.g., 'python-requests/2.31.0'). This helps server operators identify clients and debug issues. The OCaml library currently requires users to set User-Agent manually. Adding a default like 'ocaml-requests/0.1.0' with easy override would follow HTTP best practices."
181 },
182 {
183 "source_repo": "cpp-httplib",
184 "source_language": "C++",
185 "criticality": "medium",
186 "change_type": "feature",
187 "title": "Add support for custom socket options via callback",
188 "description": "cpp-httplib provides set_socket_options() callback for custom socket configuration (SO_SNDBUF, SO_RCVBUF, etc.). The OCaml library should expose similar low-level socket tuning.",
189 "affected_files": [
190 "lib/requests.ml",
191 "lib/requests.mli"
192 ],
193 "rationale": "cpp-httplib's set_socket_options(SocketOptions) allows configuring TCP_NODELAY, buffer sizes, keepalive, etc. This is critical for performance tuning and embedded systems. The OCaml library currently doesn't expose socket-level configuration, relying on Eio defaults. Adding optional socket options callback would enable performance tuning."
194 },
195 {
196 "source_repo": "cpp-httplib",
197 "source_language": "C++",
198 "criticality": "high",
199 "change_type": "security",
200 "title": "Add validation for form-urlencoded payload size limits",
201 "description": "cpp-httplib enforces FORM_URL_ENCODED_PAYLOAD_MAX_LENGTH (8192) separate from general payload limits to prevent hash collision attacks. The OCaml library should add similar form-specific limits.",
202 "affected_files": [
203 "lib/response_limits.ml",
204 "lib/body.ml",
205 "lib/http_read.ml"
206 ],
207 "rationale": "cpp-httplib limits form-urlencoded payloads to 8KB by default, recognizing that parsing form data into hash tables is vulnerable to collision attacks. The OCaml Response_limits module has max_response_body_size but no separate form data limit. This leaves the library vulnerable to hash collision DoS on form parsing."
208 },
209 {
210 "source_repo": "cpp-httplib",
211 "source_language": "C++",
212 "criticality": "medium",
213 "change_type": "feature",
214 "title": "Add support for Unix domain socket connections",
215 "description": "cpp-httplib supports Unix domain sockets via set_address_family(AF_UNIX) for IPC. The OCaml library should add Unix socket support for local service communication.",
216 "affected_files": [
217 "lib/one.ml",
218 "lib/requests.ml",
219 "lib/http_client.ml"
220 ],
221 "rationale": "cpp-httplib supports Unix domain sockets on Linux/macOS via set_address_family(AF_UNIX) and abstract sockets. This is useful for Docker, systemd, and local IPC. The OCaml library could leverage Eio.Net's Unix socket support to enable HTTP over Unix sockets for container/local scenarios."
222 },
223 {
224 "source_repo": "cpp-httplib",
225 "source_language": "C++",
226 "criticality": "low",
227 "change_type": "enhancement",
228 "title": "Add support for pre-compression logging hooks",
229 "description": "cpp-httplib provides set_pre_compression_logger() to log request/response before compression is applied. This helps debug compression issues. The OCaml library should add similar pre/post compression hooks.",
230 "affected_files": [
231 "lib/requests.ml",
232 "lib/response.ml"
233 ],
234 "rationale": "cpp-httplib's set_pre_compression_logger() captures data before compression, showing original content and allowing inspection of compression decisions. This is valuable for debugging compression issues (e.g., when compressed response is malformed). The OCaml library has auto_decompress but no hooks for debugging compression."
235 },
236 {
237 "source_repo": "cpp-httplib",
238 "source_language": "C++",
239 "criticality": "low",
240 "change_type": "feature",
241 "title": "Add convenience helpers for URI encoding/decoding",
242 "description": "cpp-httplib provides encode_uri(), decode_uri(), encode_uri_component(), and decode_uri_component() as utility functions. The OCaml library should expose similar helpers.",
243 "affected_files": [
244 "lib/requests.ml",
245 "lib/requests.mli"
246 ],
247 "rationale": "cpp-httplib provides httplib::encode_uri() for full URLs (preserves :/?&=) and httplib::encode_uri_component() for query params (encodes all reserved chars). While OCaml has Uri module, exposing convenience functions like Requests.encode_query_param would improve ergonomics and reduce Uri module dependency."
248 },
249 {
250 "source_repo": "cpp-httplib",
251 "source_language": "C++",
252 "criticality": "medium",
253 "change_type": "enhancement",
254 "title": "Add maximum redirect count enforcement at protocol level",
255 "description": "cpp-httplib enforces REDIRECT_MAX_COUNT at the protocol level (default 20). The OCaml library has redirect limits but should ensure they're checked before network I/O to prevent redirect-based DoS.",
256 "affected_files": [
257 "lib/requests.ml",
258 "lib/http_client.ml"
259 ],
260 "rationale": "cpp-httplib has built-in REDIRECT_MAX_COUNT constant and ExceedRedirectCount error. While the OCaml library has max_redirects parameter and Too_many_redirects error, ensuring the check happens early (before attempting connection) prevents redirect-based resource exhaustion. The implementation should short-circuit at max_redirects without network overhead."
261 },
262 {
263 "source_repo": "cpp-httplib",
264 "source_language": "C++",
265 "criticality": "low",
266 "change_type": "feature",
267 "title": "Add support for custom HTTP version specification",
268 "description": "cpp-httplib allows specifying HTTP version (1.0, 1.1) per request. The OCaml library should add version control for compatibility with legacy servers.",
269 "affected_files": [
270 "lib/requests.ml",
271 "lib/http_client.ml",
272 "lib/http_write.ml"
273 ],
274 "rationale": "cpp-httplib allows forcing HTTP/1.0 vs HTTP/1.1 for compatibility. While HTTP/1.1 is standard, some legacy servers/proxies only support 1.0, or require it to avoid chunked encoding. The OCaml library currently hardcodes HTTP/1.1. Adding version control would improve compatibility."
275 }
276 ]
277}