about things

update zig 0.15 notes from firehose implementation

- new: binary encoding patterns (CBOR, CAR, varints, arena per message)
- add: SHA-256 hashing for content addressing
- add: @constCast for inline struct literal slices
- add: std.math.cast for safe integer narrowing at system boundaries
- add: comptime-parameterized adapter pattern for callback bridging
- update: README to reflect zat's expanded scope (CBOR, firehose)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

+469 -2
+1
languages/ziglang/0.15/README.md
··· 14 14 ## notes 15 15 16 16 - [arraylist](./arraylist.md) - ownership patterns (toOwnedSlice vs deinit) 17 + - [binary](./binary.md) - encoding/decoding wire formats (CBOR, CAR, varints, arenas) 17 18 - [io](./io.md) - explicit buffers, http client/server, tls quirk 18 19 - [build](./build.md) - createModule + imports, hash trick 19 20 - [comptime](./comptime.md) - type generation, tuple synthesis, string validation
+159
languages/ziglang/0.15/binary.md
··· 1 + # binary encoding 2 + 3 + patterns for encoding/decoding binary wire formats (CBOR, CAR, protocol frames). distinct from JSON - you're working with raw bytes and need to handle endianness, varints, and content addressing. 4 + 5 + ## anytype writer for encoders 6 + 7 + the core pattern: an encoder function that accepts any writer via `anytype`. this lets the same encoder write to fixed buffers, ArrayLists, or any other writer: 8 + 9 + ```zig 10 + pub fn encode(allocator: Allocator, writer: anytype, value: Value) !void { 11 + switch (value) { 12 + .unsigned => |v| try writeArgument(writer, 0, v), 13 + .text => |t| { 14 + try writeArgument(writer, 3, t.len); 15 + try writer.writeAll(t); 16 + }, 17 + .map => |entries| { 18 + // sort keys (DAG-CBOR determinism), needs allocator 19 + const sorted = try allocator.dupe(MapEntry, entries); 20 + defer allocator.free(sorted); 21 + std.mem.sort(MapEntry, sorted, {}, keyLessThan); 22 + // ... 23 + }, 24 + // ... 25 + } 26 + } 27 + ``` 28 + 29 + the allocator parameter is separate from the writer - needed for temporary allocations during encoding (sorting map keys, building intermediate buffers), not for the output itself. 30 + 31 + usage with different writers: 32 + 33 + ```zig 34 + // fixed buffer (no allocation for output) 35 + var buf: [1024]u8 = undefined; 36 + var stream = std.io.fixedBufferStream(&buf); 37 + try encode(alloc, stream.writer(), value); 38 + const result = stream.getWritten(); 39 + 40 + // growable buffer 41 + var list: std.ArrayList(u8) = .{}; 42 + defer list.deinit(alloc); 43 + try encode(alloc, list.writer(alloc), value); 44 + ``` 45 + 46 + see: [zat/cbor.zig](https://tangled.sh/@zzstoatzz.io/zat/tree/main/src/internal/cbor.zig) 47 + 48 + ## encodeAlloc convenience 49 + 50 + wrap the growable-buffer pattern into a helper: 51 + 52 + ```zig 53 + pub fn encodeAlloc(allocator: Allocator, value: Value) ![]u8 { 54 + var list: std.ArrayList(u8) = .{}; 55 + errdefer list.deinit(allocator); 56 + try encode(allocator, list.writer(allocator), value); 57 + return try list.toOwnedSlice(allocator); 58 + } 59 + ``` 60 + 61 + caller owns the returned slice. `errdefer` ensures cleanup if encoding fails partway through. 62 + 63 + ## big-endian integers without writeInt 64 + 65 + when writing fixed-width big-endian integers to an `anytype` writer, build the bytes manually rather than depending on `writeInt` (which may not be available on all writer types): 66 + 67 + ```zig 68 + fn writeArgument(writer: anytype, major: u3, val: u64) !void { 69 + const prefix: u8 = @as(u8, major) << 5; 70 + if (val <= 0xffff) { 71 + try writer.writeByte(prefix | 25); 72 + const v: u16 = @intCast(val); 73 + try writer.writeAll(&[2]u8{ @truncate(v >> 8), @truncate(v) }); 74 + } 75 + // ... 76 + } 77 + ``` 78 + 79 + `@truncate` on shifted values is the idiomatic way to extract individual bytes. 80 + 81 + ## unsigned varint (LEB128) 82 + 83 + used by CID, CAR, and other IPLD formats for variable-length integers: 84 + 85 + ```zig 86 + // write 87 + pub fn writeUvarint(writer: anytype, val: u64) !void { 88 + var v = val; 89 + while (v >= 0x80) { 90 + try writer.writeByte(@as(u8, @truncate(v)) | 0x80); 91 + v >>= 7; 92 + } 93 + try writer.writeByte(@as(u8, @truncate(v))); 94 + } 95 + 96 + // read 97 + fn readUvarint(data: []const u8, pos: *usize) ?u64 { 98 + var result: u64 = 0; 99 + var shift: u6 = 0; 100 + while (pos.* < data.len) { 101 + const byte = data[pos.*]; 102 + pos.* += 1; 103 + result |= @as(u64, byte & 0x7f) << shift; 104 + if (byte & 0x80 == 0) return result; 105 + shift +|= 7; 106 + if (shift >= 64) return null; 107 + } 108 + return null; 109 + } 110 + ``` 111 + 112 + note `+|=` (saturating add) prevents overflow on the shift counter. 113 + 114 + ## arena per message 115 + 116 + for streaming protocols, create an arena per incoming message. all decoding allocations go into it, then free everything at once: 117 + 118 + ```zig 119 + pub fn serverMessage(self: *Self, data: []const u8) !void { 120 + var arena = std.heap.ArenaAllocator.init(self.allocator); 121 + defer arena.deinit(); 122 + 123 + const event = decodeFrame(arena.allocator(), data) catch |err| { 124 + log.debug("decode error: {s}", .{@errorName(err)}); 125 + return; 126 + }; 127 + 128 + self.handler.onEvent(event); 129 + // arena freed here — all decoded data is gone 130 + } 131 + ``` 132 + 133 + this means the handler's `onEvent` must not hold references to event data past the call. if it needs to, it must copy into its own allocator. 134 + 135 + see: [zat/firehose.zig](https://tangled.sh/@zzstoatzz.io/zat/tree/main/src/internal/firehose.zig), [zat/jetstream.zig](https://tangled.sh/@zzstoatzz.io/zat/tree/main/src/internal/jetstream.zig) 136 + 137 + ## deterministic encoding 138 + 139 + DAG-CBOR requires deterministic output (same value → same bytes). the main rules: 140 + 141 + - **shortest integer encoding**: 0-23 inline, 24-255 in 1 byte, etc. 142 + - **map keys sorted**: by byte length first, then lexicographically 143 + - **no floats, no indefinite lengths** 144 + 145 + sorting map keys during encoding: 146 + 147 + ```zig 148 + fn dagCborKeyLessThan(_: void, a: MapEntry, b: MapEntry) bool { 149 + if (a.key.len != b.key.len) return a.key.len < b.key.len; 150 + return std.mem.order(u8, a.key, b.key) == .lt; 151 + } 152 + 153 + // in encoder: 154 + const sorted = try allocator.dupe(MapEntry, entries); 155 + defer allocator.free(sorted); 156 + std.mem.sort(MapEntry, sorted, {}, dagCborKeyLessThan); 157 + ``` 158 + 159 + the dupe + sort pattern avoids mutating the input — the caller's `entries` slice stays unchanged.
+17
languages/ziglang/0.15/crypto.md
··· 42 42 compressed keys start with 0x02 or 0x03. uncompressed start with 0x04. 43 43 44 44 see: [zat/jwt.zig](https://tangled.sh/@zzstoatzz.io/zat/tree/main/src/internal/jwt.zig) 45 + 46 + ## sha-256 hashing 47 + 48 + for content addressing (CIDs, integrity checks). hash data into a fixed-size digest: 49 + 50 + ```zig 51 + const Sha256 = std.crypto.hash.sha2.Sha256; 52 + 53 + var hash: [Sha256.digest_length]u8 = undefined; // 32 bytes 54 + Sha256.hash(data, &hash, .{}); 55 + ``` 56 + 57 + no allocation needed — the digest is a fixed 32-byte array on the stack. the `.{}` is options (empty = defaults). 58 + 59 + use case: creating CIDs (content identifiers) for IPLD/AT Protocol — hash DAG-CBOR encoded data, then wrap the digest in a CID with version + codec + multihash framing. 60 + 61 + see: [zat/cbor.zig Cid.forDagCbor](https://tangled.sh/@zzstoatzz.io/zat/tree/main/src/internal/cbor.zig)
+30
languages/ziglang/0.15/interfaces.md
··· 59 59 60 60 this is the std.io.Writer pattern. more boilerplate, but allows swapping implementations at runtime. 61 61 62 + ## comptime-parameterized adapter 63 + 64 + when you need a struct that bridges between a library's callback interface and a user's handler type, return a type parameterized on the handler: 65 + 66 + ```zig 67 + fn WsHandler(comptime H: type) type { 68 + return struct { 69 + allocator: Allocator, 70 + handler: *H, 71 + client_state: *FirehoseClient, 72 + 73 + const Self = @This(); 74 + 75 + pub fn serverMessage(self: *Self, data: []const u8) !void { 76 + var arena = std.heap.ArenaAllocator.init(self.allocator); 77 + defer arena.deinit(); 78 + 79 + const event = decodeFrame(arena.allocator(), data) catch return; 80 + self.handler.onEvent(event); 81 + } 82 + 83 + pub fn close(_: *Self) void {} 84 + }; 85 + } 86 + ``` 87 + 88 + this adapts the websocket library's `serverMessage` callback to our firehose handler's `onEvent` interface. the comptime parameter `H` is the user's handler type — the compiler generates a specialized struct for each handler type used. no vtables, no allocation, full inlining. 89 + 90 + see: [zat/firehose.zig](https://tangled.sh/@zzstoatzz.io/zat/tree/main/src/internal/firehose.zig) 91 + 62 92 ## practical: module boundary pattern 63 93 64 94 for most apps, just separate interface documentation from implementation:
+23 -1
languages/ziglang/0.15/json.md
··· 36 36 - `beginObject()` / `endObject()` - `{` and `}` 37 37 - `beginArray()` / `endArray()` - `[` and `]` 38 38 - `objectField("key")` - write a key, call `write()` next for the value 39 - - `write(value)` - writes any json-serializable value 39 + - `write(value)` - writes any json-serializable value (handles UTF-8 and escaping correctly) 40 + 41 + ## fixed buffer (no allocation) 42 + 43 + when you have a stack buffer and don't want to allocate: 44 + 45 + ```zig 46 + fn toJson(data: MyData, buf: []u8) []const u8 { 47 + var w: std.Io.Writer = .fixed(buf); 48 + var jw: std.json.Stringify = .{ .writer = &w }; 49 + 50 + jw.beginObject() catch return "{}"; 51 + jw.objectField("name") catch return "{}"; 52 + jw.write(data.name) catch return "{}"; 53 + jw.endObject() catch return "{}"; 54 + 55 + return w.buffered(); // slice of what was written 56 + } 57 + ``` 58 + 59 + key difference: use `std.Io.Writer = .fixed(buf)` and call `w.buffered()` to get the written slice. 60 + 61 + see: [coral/backend/src/entities.zig](https://tangled.sh/@zzstoatzz.io/coral/tree/main/backend/src/entities.zig) 40 62 41 63 ## raw json passthrough 42 64
+32
languages/ziglang/0.15/structs.md
··· 88 88 89 89 see: [logfire-zig/attribute.zig](https://tangled.sh/@zzstoatzz.io/logfire-zig/tree/main/src/attribute.zig) 90 90 91 + ## @constCast for inline struct literal slices 92 + 93 + when you need to pass an inline struct literal as a slice parameter, zig may need `@constCast` because the literal is `const`: 94 + 95 + ```zig 96 + // this builds an array of MapEntry inline and passes it as a slice 97 + try op_values.append(allocator, .{ .map = @constCast(&[_]cbor.Value.MapEntry{ 98 + .{ .key = "action", .value = .{ .text = action_str } }, 99 + .{ .key = "path", .value = .{ .text = path } }, 100 + }) }); 101 + ``` 102 + 103 + without `@constCast`, you get a type mismatch — the literal produces `*const [N]MapEntry` but the field expects `[]const MapEntry` through a mutable pointer. this is safe because the data is embedded in the struct value being appended. 104 + 105 + see: [zat/firehose.zig encodeCommitPayload](https://tangled.sh/@zzstoatzz.io/zat/tree/main/src/internal/firehose.zig) 106 + 107 + ## std.math.cast for safe integer narrowing 108 + 109 + prefer `std.math.cast` over `@intCast` when narrowing integers from untrusted input. `@intCast` panics on overflow, `std.math.cast` returns `null`: 110 + 111 + ```zig 112 + // dangerous — panics on 32-bit if header_len > maxInt(usize) 113 + const len: usize = @intCast(varint_u64); 114 + 115 + // safe — returns an error instead of panicking 116 + const len = std.math.cast(usize, varint_u64) orelse return error.InvalidHeader; 117 + ``` 118 + 119 + use `@intCast` for values you've already bounds-checked or that come from trusted internal code. use `std.math.cast` at system boundaries (parsing wire formats, external input). 120 + 121 + see: [zat/car.zig](https://tangled.sh/@zzstoatzz.io/zat/tree/main/src/internal/car.zig) 122 + 91 123 ## when to use this pattern 92 124 93 125 use internal storage when:
+1
languages/ziglang/0.16/README.md
··· 5 5 ## notes 6 6 7 7 - [io](./io.md) - std.Io interface, async/concurrent, no function coloring 8 + - [migration](./migration.md) - practical API changes, verified patterns
+205
languages/ziglang/0.16/migration.md
··· 1 + # 0.16 migration notes 2 + 3 + verified with `zig-aarch64-macos-0.16.0-dev.2296+fd3657bf8` 4 + 5 + ## high-impact changes 6 + 7 + - **std.net removed** - networking now via `Io.net` 8 + - **std.crypto.random removed** - randomness now via `io.random()` 9 + - **std.Thread.Pool removed** - use `Io.Group` with `Io.Threaded` 10 + - **posix.shutdown removed** - use `std.c.shutdown(fd, how)` 11 + - **posix.getrandom removed** - use `io.random()` 12 + 13 + these require significant refactoring as they move to the Io-based paradigm. 14 + 15 + ## build.zig changes 16 + 17 + `linkLibC()` removed. use module options instead: 18 + 19 + ```zig 20 + // 0.15 21 + const exe = b.addExecutable(.{...}); 22 + exe.linkLibC(); 23 + 24 + // 0.16 25 + const exe = b.addExecutable(.{ 26 + .name = "myapp", 27 + .root_module = b.createModule(.{ 28 + .root_source_file = b.path("src/main.zig"), 29 + .target = target, 30 + .optimize = optimize, 31 + .link_libc = true, // here 32 + }), 33 + }); 34 + ``` 35 + 36 + ## package hashes 37 + 38 + 0.16 calculates hashes differently. compiler will tell you the correct hash. 39 + 40 + ## removed APIs 41 + 42 + ### std.time.timestamp() / milliTimestamp() 43 + 44 + use libc directly: 45 + 46 + ```zig 47 + const c = std.c; 48 + 49 + pub fn timestamp() i64 { 50 + var tv: c.timeval = undefined; 51 + _ = c.gettimeofday(&tv, null); 52 + return tv.sec; 53 + } 54 + 55 + pub fn milliTimestamp() i64 { 56 + var tv: c.timeval = undefined; 57 + _ = c.gettimeofday(&tv, null); 58 + return @as(i64, tv.sec) * 1000 + @divTrunc(@as(i64, tv.usec), 1000); 59 + } 60 + ``` 61 + 62 + ### std.Thread.sleep() 63 + 64 + use libc nanosleep: 65 + 66 + ```zig 67 + pub fn sleep(ns: u64) void { 68 + const secs = ns / std.time.ns_per_s; 69 + const nsecs = ns % std.time.ns_per_s; 70 + var ts: std.c.timespec = .{ 71 + .sec = @intCast(secs), 72 + .nsec = @intCast(nsecs), 73 + }; 74 + _ = std.c.nanosleep(&ts, null); 75 + } 76 + ``` 77 + 78 + ### std.posix.getenv() 79 + 80 + use `std.c.getenv()` with `std.mem.span()`: 81 + 82 + ```zig 83 + // 0.15 84 + const val = std.posix.getenv("FOO") orelse "default"; 85 + 86 + // 0.16 - returns [*:0]const u8, not slice 87 + const val = if (std.c.getenv("FOO")) |p| std.mem.span(p) else "default"; 88 + ``` 89 + 90 + ### @Type builtin 91 + 92 + removed. use specific type builtins: 93 + 94 + ```zig 95 + // 0.15 96 + const Args = @Type(.{ .@"struct" = .{ .is_tuple = true, ... } }); 97 + 98 + // 0.16 99 + const Args = @Tuple(&field_types); 100 + // or @Struct(...) for non-tuples 101 + ``` 102 + 103 + ### posix.shutdown 104 + 105 + use libc directly: 106 + 107 + ```zig 108 + // 0.15 109 + posix.shutdown(fd, .recv) catch {}; 110 + 111 + // 0.16 112 + _ = std.c.shutdown(fd, 0); // SHUT_RD = 0, SHUT_WR = 1, SHUT_RDWR = 2 113 + ``` 114 + 115 + ### std.testing.expectEqual 116 + 117 + argument order changed (minor but affects all tests): 118 + 119 + ```zig 120 + // 0.15 121 + try std.testing.expectEqual(@as(@TypeOf(actual), expected), actual); 122 + 123 + // 0.16 - simpler, expected comes first naturally 124 + try std.testing.expectEqual(expected, actual); 125 + ``` 126 + 127 + ## networking 128 + 129 + `std.net` is gone. use `Io.net`: 130 + 131 + ```zig 132 + const Io = std.Io; 133 + const net = Io.net; 134 + 135 + // connecting to a host 136 + pub fn connect(io: Io, host: []const u8, port: u16) !net.Stream { 137 + const host_name = try net.HostName.init(host); 138 + return host_name.connect(io, port, .{}); 139 + } 140 + ``` 141 + 142 + **note**: `Io.net.Stream` no longer has direct `read`/`writeAll` methods. 143 + you need to create `Stream.Reader` or `Stream.Writer` wrappers: 144 + 145 + ```zig 146 + // reading from stream requires a Reader wrapper 147 + pub fn readFromStream(stream: net.Stream, io: Io, buffer: []u8) !usize { 148 + var reader = net.Stream.Reader.init(stream, io, buffer); 149 + // use reader.interface for reading 150 + return reader.interface.read(buffer); 151 + } 152 + ``` 153 + 154 + `net.Address` → `net.IpAddress`: 155 + ```zig 156 + // 0.15 157 + const addr = std.net.Address.parseIp("127.0.0.1", 8080); 158 + 159 + // 0.16 160 + const addr = Io.net.IpAddress.parse("127.0.0.1", 8080); 161 + ``` 162 + 163 + ## randomness 164 + 165 + `std.crypto.random` removed. use `io.random()`: 166 + 167 + ```zig 168 + // 0.15 169 + var bytes: [16]u8 = undefined; 170 + std.crypto.random.bytes(&bytes); 171 + 172 + // 0.16 - io.random() returns typed value directly 173 + const io = std.Options.debug_io; 174 + const bytes: [16]u8 = io.random(); // returns [16]u8 directly 175 + const seed: u64 = io.random(); // returns u64 directly 176 + ``` 177 + 178 + ## file I/O 179 + 180 + file ops now take `io: Io` parameter. get default io from `std.Options.debug_io`: 181 + 182 + ```zig 183 + const std = @import("std"); 184 + const Io = std.Io; 185 + 186 + pub fn readFile(path: []const u8) ![]u8 { 187 + const io = std.Options.debug_io; 188 + 189 + const file = try Io.Dir.openFileAbsolute(io, path, .{}); 190 + defer file.close(io); 191 + 192 + var buf: [4096]u8 = undefined; 193 + const n = try file.readPositional(io, &.{&buf}, 0); 194 + return buf[0..n]; 195 + } 196 + 197 + pub fn writeFile(path: []const u8, data: []const u8) !void { 198 + const io = std.Options.debug_io; 199 + 200 + const file = try Io.Dir.createFileAbsolute(io, path, .{}); 201 + defer file.close(io); 202 + 203 + try file.writeStreamingAll(io, data); 204 + } 205 + ```
+1 -1
languages/ziglang/README.md
··· 19 19 | [leaflet-search](https://tangled.sh/@zzstoatzz.io/leaflet-search) | fts search backend | 20 20 | [pollz](https://tangled.sh/@zzstoatzz.io/pollz) | bluesky polls (zqlite + transactions) | 21 21 | [zql](https://tangled.sh/@zzstoatzz.io/zql) | comptime sql parsing | 22 - | [zat](https://tangled.sh/@zzstoatzz.io/zat) | atproto primitives (jwt, crypto) | 22 + | [zat](https://tangled.sh/@zzstoatzz.io/zat) | atproto primitives (jwt, crypto, CBOR, firehose) | 23 23 | [logfire-zig](https://tangled.sh/@zzstoatzz.io/logfire-zig) | OTLP observability client | 24 24 | [prefect-zig](https://tangled.sh/@zzstoatzz.io/prefect-zig) | prefect orchestration server | 25 25 | [ghostty](https://github.com/ghostty-org/ghostty) | terminal emulator (build system) |