zat.dev
atproto utils for zig
zat.dev
atproto
sdk
zig
docs: add correctness parity analysis to benchmark devlog
traced indigo's full decode path to verify no correctness work is
skipped. the ~20x gap is implementation cost (reflection, heap alloc,
byte copying), not correctness differences.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
zzstoatzz.io
+21
devlog/002-firehose-and-benchmarks.md
+21
devlog/002-firehose-and-benchmarks.md
···
73
73
74
74
all SDKs: 0 errors. run-to-run variance is ~30-40% — compare ratios within a single run, not across runs. indigo's number is the same in both tables because go-car v1 always verifies.
75
75
76
+
### is the gap real?
77
+
78
+
the ~20x between zat and indigo (both verifying CID hashes) is large enough to be suspicious. we traced indigo's full decode path at the instruction level to check whether indigo does correctness work that zat skips.
79
+
80
+
**what both do per frame:** decode full CBOR payload (all commit fields), parse CAR header and blocks, parse CID structure for each block, SHA-256 hash each block against its CID, decode every block as DAG-CBOR.
81
+
82
+
**what neither does:** DAG-CBOR deterministic encoding validation (indigo's refmt doesn't check this either), signature verification, MST validation.
83
+
84
+
**only asymmetry:** indigo enforces size limits on CBOR maps and a 2MB cap on the blocks field — integer comparisons, effectively free.
85
+
86
+
the gap is entirely implementation cost, not correctness differences. it compounds from:
87
+
88
+
| factor | indigo | zat | approx cost |
89
+
|--------|--------|-----|-------------|
90
+
| per-block CBOR | refmt: token pump → reflection → `reflect.SetMapIndex` per entry | hand-written, direct dispatch | ~3-4x |
91
+
| strings/bytes | Go `string` heap alloc per value | zero-copy slices into input buffer | ~2-3x |
92
+
| memory | per-object GC'd heap; every map, array, int is boxed | arena allocator, 24-byte `Value` union | ~2-3x |
93
+
| CAR reads | `make([]byte, n)` + copy per block; CID parsed twice | reads from input slice; CID parsed once | ~1.5x |
94
+
95
+
indigo's `cbor-gen` (code-generated unmarshal for the commit struct) is fast — the bottleneck is `cbornode.DecodeInto` which uses refmt (unmaintained, reflection-based) for the ~10 per-block DAG-CBOR decodes per frame.
96
+
76
97
### why zat is fast
77
98
78
99
three things compound: