···1+# JSON Pointer Tutorial
2+3+This tutorial introduces JSON Pointer as defined in
4+[RFC 6901](https://www.rfc-editor.org/rfc/rfc6901), and demonstrates
5+the `jsont-pointer` OCaml library through interactive examples.
6+7+## What is JSON Pointer?
8+9+From RFC 6901, Section 1:
10+11+> JSON Pointer defines a string syntax for identifying a specific value
12+> within a JavaScript Object Notation (JSON) document.
13+14+In other words, JSON Pointer is an addressing scheme for locating values
15+inside a JSON structure. Think of it like a filesystem path, but for JSON
16+documents instead of files.
17+18+For example, given this JSON document:
19+20+```json
21+{
22+ "users": [
23+ {"name": "Alice", "age": 30},
24+ {"name": "Bob", "age": 25}
25+ ]
26+}
27+```
28+29+The JSON Pointer `/users/0/name` refers to the string `"Alice"`.
30+31+## Syntax: Reference Tokens
32+33+RFC 6901, Section 3 defines the syntax:
34+35+> A JSON Pointer is a Unicode string containing a sequence of zero or more
36+> reference tokens, each prefixed by a '/' (%x2F) character.
37+38+The grammar is elegantly simple:
39+40+```
41+json-pointer = *( "/" reference-token )
42+reference-token = *( unescaped / escaped )
43+```
44+45+This means:
46+- The empty string `""` is a valid pointer (it refers to the whole document)
47+- Every non-empty pointer starts with `/`
48+- Everything between `/` characters is a "reference token"
49+50+Let's see this in action. We can parse pointers and see their structure:
51+52+```sh
53+$ jsonpp parse ""
54+OK: []
55+```
56+57+The empty pointer has no reference tokens - it points to the root.
58+59+```sh
60+$ jsonpp parse "/foo"
61+OK: [Mem:foo]
62+```
63+64+The pointer `/foo` has one token: `foo`. Since it's not a number, it's
65+interpreted as an object member name (`Mem`).
66+67+```sh
68+$ jsonpp parse "/foo/0"
69+OK: [Mem:foo, Nth:0]
70+```
71+72+Here we have two tokens: `foo` (a member name) and `0` (interpreted as
73+an array index `Nth`).
74+75+```sh
76+$ jsonpp parse "/foo/bar/baz"
77+OK: [Mem:foo, Mem:bar, Mem:baz]
78+```
79+80+Multiple tokens navigate deeper into nested structures.
81+82+### Invalid Syntax
83+84+What happens if a pointer doesn't start with `/`?
85+86+```sh
87+$ jsonpp parse "foo"
88+ERROR: Invalid JSON Pointer: must be empty or start with '/': foo
89+```
90+91+The RFC is strict: non-empty pointers MUST start with `/`.
92+93+## Escaping Special Characters
94+95+RFC 6901, Section 3 explains the escaping rules:
96+97+> Because the characters '~' (%x7E) and '/' (%x2F) have special meanings
98+> in JSON Pointer, '~' needs to be encoded as '~0' and '/' needs to be
99+> encoded as '~1' when these characters appear in a reference token.
100+101+Why these specific characters?
102+- `/` separates tokens, so it must be escaped inside a token
103+- `~` is the escape character itself, so it must also be escaped
104+105+The escape sequences are:
106+- `~0` represents `~` (tilde)
107+- `~1` represents `/` (forward slash)
108+109+Let's see escaping in action:
110+111+```sh
112+$ jsonpp escape "hello"
113+hello
114+```
115+116+No special characters, no escaping needed.
117+118+```sh
119+$ jsonpp escape "a/b"
120+a~1b
121+```
122+123+The `/` becomes `~1`.
124+125+```sh
126+$ jsonpp escape "a~b"
127+a~0b
128+```
129+130+The `~` becomes `~0`.
131+132+```sh
133+$ jsonpp escape "~/"
134+~0~1
135+```
136+137+Both characters are escaped.
138+139+### Unescaping
140+141+And the reverse process:
142+143+```sh
144+$ jsonpp unescape "a~1b"
145+OK: a/b
146+```
147+148+```sh
149+$ jsonpp unescape "a~0b"
150+OK: a~b
151+```
152+153+### The Order Matters!
154+155+RFC 6901, Section 4 is careful to specify the unescaping order:
156+157+> Evaluation of each reference token begins by decoding any escaped
158+> character sequence. This is performed by first transforming any
159+> occurrence of the sequence '~1' to '/', and then transforming any
160+> occurrence of the sequence '~0' to '~'. By performing the substitutions
161+> in this order, an implementation avoids the error of turning '~01' first
162+> into '~1' and then into '/', which would be incorrect (the string '~01'
163+> correctly becomes '~1' after transformation).
164+165+Let's verify this tricky case:
166+167+```sh
168+$ jsonpp unescape "~01"
169+OK: ~1
170+```
171+172+If we unescaped `~0` first, `~01` would become `~1`, which would then become
173+`/`. But that's wrong! The sequence `~01` should become the literal string
174+`~1` (a tilde followed by the digit one).
175+176+Invalid escape sequences are rejected:
177+178+```sh
179+$ jsonpp unescape "~2"
180+ERROR: Invalid JSON Pointer: invalid escape sequence ~2
181+```
182+183+```sh
184+$ jsonpp unescape "hello~"
185+ERROR: Invalid JSON Pointer: incomplete escape sequence at end
186+```
187+188+## Evaluation: Navigating JSON
189+190+Now we come to the heart of JSON Pointer: evaluation. RFC 6901, Section 4
191+describes how a pointer is resolved against a JSON document:
192+193+> Evaluation of a JSON Pointer begins with a reference to the root value
194+> of a JSON document and completes with a reference to some value within
195+> the document. Each reference token in the JSON Pointer is evaluated
196+> sequentially.
197+198+Let's use the example JSON document from RFC 6901, Section 5:
199+200+```sh
201+$ cat rfc6901_example.json
202+{
203+ "foo": ["bar", "baz"],
204+ "": 0,
205+ "a/b": 1,
206+ "c%d": 2,
207+ "e^f": 3,
208+ "g|h": 4,
209+ "i\\j": 5,
210+ "k\"l": 6,
211+ " ": 7,
212+ "m~n": 8
213+}
214+```
215+216+This document is carefully constructed to exercise various edge cases!
217+218+### The Root Pointer
219+220+```sh
221+$ jsonpp eval rfc6901_example.json ""
222+OK: {"foo":["bar","baz"],"":0,"a/b":1,"c%d":2,"e^f":3,"g|h":4,"i\\j":5,"k\"l":6," ":7,"m~n":8}
223+```
224+225+The empty pointer returns the whole document.
226+227+### Object Member Access
228+229+```sh
230+$ jsonpp eval rfc6901_example.json "/foo"
231+OK: ["bar","baz"]
232+```
233+234+`/foo` accesses the member named `foo`, which is an array.
235+236+### Array Index Access
237+238+```sh
239+$ jsonpp eval rfc6901_example.json "/foo/0"
240+OK: "bar"
241+```
242+243+`/foo/0` first goes to `foo`, then accesses index 0 of the array.
244+245+```sh
246+$ jsonpp eval rfc6901_example.json "/foo/1"
247+OK: "baz"
248+```
249+250+Index 1 gives us the second element.
251+252+### Empty String as Key
253+254+JSON allows empty strings as object keys:
255+256+```sh
257+$ jsonpp eval rfc6901_example.json "/"
258+OK: 0
259+```
260+261+The pointer `/` has one token: the empty string. This accesses the member
262+with an empty name.
263+264+### Keys with Special Characters
265+266+Now for the escape sequences:
267+268+```sh
269+$ jsonpp eval rfc6901_example.json "/a~1b"
270+OK: 1
271+```
272+273+The token `a~1b` unescapes to `a/b`, which is the key name.
274+275+```sh
276+$ jsonpp eval rfc6901_example.json "/m~0n"
277+OK: 8
278+```
279+280+The token `m~0n` unescapes to `m~n`.
281+282+### Other Special Characters (No Escaping Needed)
283+284+Most characters don't need escaping in JSON Pointer strings:
285+286+```sh
287+$ jsonpp eval rfc6901_example.json "/c%d"
288+OK: 2
289+```
290+291+```sh
292+$ jsonpp eval rfc6901_example.json "/e^f"
293+OK: 3
294+```
295+296+```sh
297+$ jsonpp eval rfc6901_example.json "/g|h"
298+OK: 4
299+```
300+301+```sh
302+$ jsonpp eval rfc6901_example.json "/ "
303+OK: 7
304+```
305+306+Even a space is a valid key character!
307+308+### Error Conditions
309+310+What happens when we try to access something that doesn't exist?
311+312+```sh
313+$ jsonpp eval rfc6901_example.json "/nonexistent"
314+ERROR: JSON Pointer: member 'nonexistent' not found
315+File "-":
316+```
317+318+Or an out-of-bounds array index:
319+320+```sh
321+$ jsonpp eval rfc6901_example.json "/foo/99"
322+ERROR: JSON Pointer: index 99 out of bounds (array has 2 elements)
323+File "-":
324+```
325+326+Or try to index into a non-container:
327+328+```sh
329+$ jsonpp eval rfc6901_example.json "/foo/0/invalid"
330+ERROR: JSON Pointer: cannot index into string with 'invalid'
331+File "-":
332+```
333+334+### Array Index Rules
335+336+RFC 6901 has specific rules for array indices. Section 4 states:
337+338+> characters comprised of digits [...] that represent an unsigned base-10
339+> integer value, making the new referenced value the array element with
340+> the zero-based index identified by the token
341+342+And importantly:
343+344+> note that leading zeros are not allowed
345+346+```sh
347+$ jsonpp parse "/foo/0"
348+OK: [Mem:foo, Nth:0]
349+```
350+351+Zero itself is fine.
352+353+```sh
354+$ jsonpp parse "/foo/01"
355+OK: [Mem:foo, Mem:01]
356+```
357+358+But `01` has a leading zero, so it's NOT treated as an array index - it
359+becomes a member name instead. This protects against accidental octal
360+interpretation.
361+362+## The End-of-Array Marker: `-`
363+364+RFC 6901, Section 4 introduces a special token:
365+366+> exactly the single character "-", making the new referenced value the
367+> (nonexistent) member after the last array element.
368+369+This is primarily useful for JSON Patch operations (RFC 6902). Let's see
370+how it parses:
371+372+```sh
373+$ jsonpp parse "/foo/-"
374+OK: [Mem:foo, End]
375+```
376+377+The `-` is recognized as a special `End` index.
378+379+However, you cannot evaluate a pointer containing `-` because it refers
380+to a position that doesn't exist:
381+382+```sh
383+$ jsonpp eval rfc6901_example.json "/foo/-"
384+ERROR: JSON Pointer: '-' (end marker) refers to nonexistent array element
385+File "-":
386+```
387+388+The RFC explains this:
389+390+> Note that the use of the "-" character to index an array will always
391+> result in such an error condition because by definition it refers to
392+> a nonexistent array element.
393+394+But we'll see later that `-` is very useful for mutation operations!
395+396+## URI Fragment Encoding
397+398+JSON Pointers can be embedded in URIs. RFC 6901, Section 6 explains:
399+400+> A JSON Pointer can be represented in a URI fragment identifier by
401+> encoding it into octets using UTF-8, while percent-encoding those
402+> characters not allowed by the fragment rule in RFC 3986.
403+404+This adds percent-encoding on top of the `~0`/`~1` escaping:
405+406+```sh
407+$ jsonpp uri-fragment "/foo"
408+OK: /foo -> /foo
409+```
410+411+Simple pointers often don't need percent-encoding.
412+413+```sh
414+$ jsonpp uri-fragment "/a~1b"
415+OK: /a~1b -> /a~1b
416+```
417+418+The `~1` escape stays as-is (it's valid in URI fragments).
419+420+```sh
421+$ jsonpp uri-fragment "/c%d"
422+OK: /c%d -> /c%25d
423+```
424+425+The `%` character must be percent-encoded as `%25` in URIs!
426+427+```sh
428+$ jsonpp uri-fragment "/ "
429+OK: / -> /%20
430+```
431+432+Spaces become `%20`.
433+434+Here's the RFC example showing the URI fragment forms:
435+436+| JSON Pointer | URI Fragment | Value |
437+|-------------|-------------|-------|
438+| `""` | `#` | whole document |
439+| `"/foo"` | `#/foo` | `["bar", "baz"]` |
440+| `"/foo/0"` | `#/foo/0` | `"bar"` |
441+| `"/"` | `#/` | `0` |
442+| `"/a~1b"` | `#/a~1b` | `1` |
443+| `"/c%d"` | `#/c%25d` | `2` |
444+| `"/ "` | `#/%20` | `7` |
445+| `"/m~0n"` | `#/m~0n` | `8` |
446+447+## Mutation Operations
448+449+While RFC 6901 defines JSON Pointer for read-only access, RFC 6902
450+(JSON Patch) uses JSON Pointer for modifications. The `jsont-pointer`
451+library provides these operations.
452+453+### Add
454+455+The `add` operation inserts a value at a location:
456+457+```sh
458+$ jsonpp add '{"foo":"bar"}' '/baz' '"qux"'
459+{"foo":"bar","baz":"qux"}
460+```
461+462+For arrays, `add` inserts BEFORE the specified index:
463+464+```sh
465+$ jsonpp add '{"foo":["a","b"]}' '/foo/1' '"X"'
466+{"foo":["a","X","b"]}
467+```
468+469+This is where the `-` marker shines - it appends to the end:
470+471+```sh
472+$ jsonpp add '{"foo":["a","b"]}' '/foo/-' '"c"'
473+{"foo":["a","b","c"]}
474+```
475+476+### Remove
477+478+The `remove` operation deletes a value:
479+480+```sh
481+$ jsonpp remove '{"foo":"bar","baz":"qux"}' '/baz'
482+{"foo":"bar"}
483+```
484+485+For arrays, it removes and shifts:
486+487+```sh
488+$ jsonpp remove '{"foo":["a","b","c"]}' '/foo/1'
489+{"foo":["a","c"]}
490+```
491+492+### Replace
493+494+The `replace` operation updates an existing value:
495+496+```sh
497+$ jsonpp replace '{"foo":"bar"}' '/foo' '"baz"'
498+{"foo":"baz"}
499+```
500+501+Unlike `add`, `replace` requires the target to already exist:
502+503+```sh
504+$ jsonpp replace '{"foo":"bar"}' '/nonexistent' '"value"'
505+ERROR: JSON Pointer: member 'nonexistent' not found
506+File "-":
507+```
508+509+### Move
510+511+The `move` operation relocates a value:
512+513+```sh
514+$ jsonpp move '{"foo":{"bar":"baz"},"qux":{}}' '/foo/bar' '/qux/thud'
515+{"foo":{},"qux":{"thud":"baz"}}
516+```
517+518+### Copy
519+520+The `copy` operation duplicates a value:
521+522+```sh
523+$ jsonpp copy '{"foo":{"bar":"baz"}}' '/foo/bar' '/foo/qux'
524+{"foo":{"bar":"baz","qux":"baz"}}
525+```
526+527+### Test
528+529+The `test` operation verifies a value (useful in JSON Patch):
530+531+```sh
532+$ jsonpp test '{"foo":"bar"}' '/foo' '"bar"'
533+true
534+```
535+536+```sh
537+$ jsonpp test '{"foo":"bar"}' '/foo' '"baz"'
538+false
539+```
540+541+## Deeply Nested Structures
542+543+JSON Pointer handles arbitrarily deep nesting:
544+545+```sh
546+$ jsonpp eval rfc6901_example.json "/foo/0"
547+OK: "bar"
548+```
549+550+For deeper structures, just add more path segments. With nested objects:
551+552+```sh
553+$ jsonpp add '{"a":{"b":{"c":"d"}}}' '/a/b/x' '"y"'
554+{"a":{"b":{"c":"d","x":"y"}}}
555+```
556+557+With nested arrays:
558+559+```sh
560+$ jsonpp add '{"arr":[[1,2],[3,4]]}' '/arr/0/1' '99'
561+{"arr":[[1,99,2],[3,4]]}
562+```
563+564+## Summary
565+566+JSON Pointer (RFC 6901) provides a simple but powerful way to address
567+values within JSON documents:
568+569+1. **Syntax**: Pointers are strings of `/`-separated reference tokens
570+2. **Escaping**: Use `~0` for `~` and `~1` for `/` in tokens
571+3. **Evaluation**: Tokens navigate through objects (by key) and arrays (by index)
572+4. **URI Encoding**: Pointers can be percent-encoded for use in URIs
573+5. **Mutations**: Combined with JSON Patch (RFC 6902), pointers enable structured updates
574+575+The `jsont-pointer` library implements all of this with type-safe OCaml
576+interfaces, integration with the `jsont` codec system, and proper error
577+handling for malformed pointers and missing values.
···48Error: nonexistent member:
49 $ ./test_pointer.exe eval data/rfc6901_example.json "/nonexistent"
50 ERROR: JSON Pointer: member 'nonexistent' not found
05152Error: index out of bounds:
53 $ ./test_pointer.exe eval data/rfc6901_example.json "/foo/2"
54 ERROR: JSON Pointer: index 2 out of bounds (array has 2 elements)
055 $ ./test_pointer.exe eval data/rfc6901_example.json "/foo/99"
56 ERROR: JSON Pointer: index 99 out of bounds (array has 2 elements)
05758Error: invalid array index (not a valid integer):
59 $ ./test_pointer.exe eval data/rfc6901_example.json "/foo/bar"
60 ERROR: JSON Pointer: invalid array index 'bar'
06162Error: end marker not allowed in get:
63 $ ./test_pointer.exe eval data/rfc6901_example.json "/foo/-"
64 ERROR: JSON Pointer: '-' (end marker) refers to nonexistent array element
06566Error: navigating through primitive (string):
67 $ ./test_pointer.exe eval data/rfc6901_example.json "/foo/0/0"
68 ERROR: JSON Pointer: cannot index into string with '0'
069 $ ./test_pointer.exe eval data/rfc6901_example.json "/foo/0/bar"
70 ERROR: JSON Pointer: cannot index into string with 'bar'
07172Nested evaluation with deep nesting:
73 $ cat data/nested.json
···48Error: nonexistent member:
49 $ ./test_pointer.exe eval data/rfc6901_example.json "/nonexistent"
50 ERROR: JSON Pointer: member 'nonexistent' not found
51+ File "-":
5253Error: index out of bounds:
54 $ ./test_pointer.exe eval data/rfc6901_example.json "/foo/2"
55 ERROR: JSON Pointer: index 2 out of bounds (array has 2 elements)
56+ File "-":
57 $ ./test_pointer.exe eval data/rfc6901_example.json "/foo/99"
58 ERROR: JSON Pointer: index 99 out of bounds (array has 2 elements)
59+ File "-":
6061Error: invalid array index (not a valid integer):
62 $ ./test_pointer.exe eval data/rfc6901_example.json "/foo/bar"
63 ERROR: JSON Pointer: invalid array index 'bar'
64+ File "-":
6566Error: end marker not allowed in get:
67 $ ./test_pointer.exe eval data/rfc6901_example.json "/foo/-"
68 ERROR: JSON Pointer: '-' (end marker) refers to nonexistent array element
69+ File "-":
7071Error: navigating through primitive (string):
72 $ ./test_pointer.exe eval data/rfc6901_example.json "/foo/0/0"
73 ERROR: JSON Pointer: cannot index into string with '0'
74+ File "-":
75 $ ./test_pointer.exe eval data/rfc6901_example.json "/foo/0/bar"
76 ERROR: JSON Pointer: cannot index into string with 'bar'
77+ File "-":
7879Nested evaluation with deep nesting:
80 $ cat data/nested.json
+5
test/mutations.t
···79Error: remove nonexistent:
80 $ ./test_pointer.exe remove '{"foo":"bar"}' '/baz'
81 ERROR: JSON Pointer: member 'baz' not found for remove
08283Error: replace nonexistent:
84 $ ./test_pointer.exe replace '{"foo":"bar"}' '/baz' '"qux"'
85 ERROR: JSON Pointer: member 'baz' not found
08687Error: add to out of bounds index:
88 $ ./test_pointer.exe add '{"foo":["bar"]}' '/foo/5' '"qux"'
89 ERROR: JSON Pointer: index 5 out of bounds for add (array has 1 elements)
09091Add nested path (parent must exist):
92 $ ./test_pointer.exe add '{"foo":{}}' '/foo/bar' '"baz"'
···137Error: remove from nonexistent array index:
138 $ ./test_pointer.exe remove '{"foo":["bar"]}' '/foo/5'
139 ERROR: JSON Pointer: index 5 out of bounds for remove
0140141Error: move to descendant of source (path doesn't exist after removal):
142 $ ./test_pointer.exe move '{"a":{"b":"c"}}' '/a' '/a/b'
143 ERROR: JSON Pointer: member 'a' not found
0144145Copy to same location (no-op essentially):
146 $ ./test_pointer.exe copy '{"foo":"bar"}' '/foo' '/foo'
···79Error: remove nonexistent:
80 $ ./test_pointer.exe remove '{"foo":"bar"}' '/baz'
81 ERROR: JSON Pointer: member 'baz' not found for remove
82+ File "-":
8384Error: replace nonexistent:
85 $ ./test_pointer.exe replace '{"foo":"bar"}' '/baz' '"qux"'
86 ERROR: JSON Pointer: member 'baz' not found
87+ File "-":
8889Error: add to out of bounds index:
90 $ ./test_pointer.exe add '{"foo":["bar"]}' '/foo/5' '"qux"'
91 ERROR: JSON Pointer: index 5 out of bounds for add (array has 1 elements)
92+ File "-":
9394Add nested path (parent must exist):
95 $ ./test_pointer.exe add '{"foo":{}}' '/foo/bar' '"baz"'
···140Error: remove from nonexistent array index:
141 $ ./test_pointer.exe remove '{"foo":["bar"]}' '/foo/5'
142 ERROR: JSON Pointer: index 5 out of bounds for remove
143+ File "-":
144145Error: move to descendant of source (path doesn't exist after removal):
146 $ ./test_pointer.exe move '{"a":{"b":"c"}}' '/a' '/a/b'
147 ERROR: JSON Pointer: member 'a' not found
148+ File "-":
149150Copy to same location (no-op essentially):
151 $ ./test_pointer.exe copy '{"foo":"bar"}' '/foo' '/foo'
+6-35
test/test_pointer.ml
···7 close_in ic;
8 s
910-(* Convert Yojson.Safe.t to Jsont.json *)
11-let rec yojson_to_jsont (y : Yojson.Safe.t) : Jsont.json =
12- match y with
13- | `Null -> Jsont.Json.null ()
14- | `Bool b -> Jsont.Json.bool b
15- | `Int i -> Jsont.Json.number (float_of_int i)
16- | `Float f -> Jsont.Json.number f
17- | `String s -> Jsont.Json.string s
18- | `List l -> Jsont.Json.list (List.map yojson_to_jsont l)
19- | `Assoc pairs ->
20- let members = List.map (fun (k, v) ->
21- Jsont.Json.mem (Jsont.Json.name k) (yojson_to_jsont v)
22- ) pairs in
23- Jsont.Json.object' members
24- | `Intlit s -> Jsont.Json.number (float_of_string s)
25-26-(* Convert Jsont.json to Yojson.Safe.t for output *)
27-let rec jsont_to_yojson (j : Jsont.json) : Yojson.Safe.t =
28- match j with
29- | Jsont.Null _ -> `Null
30- | Jsont.Bool (b, _) -> `Bool b
31- | Jsont.Number (f, _) ->
32- if Float.is_integer f && Float.abs f < 2e15 then
33- `Int (int_of_float f)
34- else
35- `Float f
36- | Jsont.String (s, _) -> `String s
37- | Jsont.Array (l, _) -> `List (List.map jsont_to_yojson l)
38- | Jsont.Object (members, _) ->
39- `Assoc (List.map (fun ((name, _), v) ->
40- (name, jsont_to_yojson v)
41- ) members)
42-43let parse_json s =
44- yojson_to_jsont (Yojson.Safe.from_string s)
004546let json_to_string json =
47- Yojson.Safe.to_string (jsont_to_yojson json)
004849(* Test: parse pointer and print indices *)
50let test_parse pointer_str =
···7 close_in ic;
8 s
900000000000000000000000000000000010let parse_json s =
11+ match Jsont_bytesrw.decode_string Jsont.json s with
12+ | Ok json -> json
13+ | Error e -> failwith e
1415let json_to_string json =
16+ match Jsont_bytesrw.encode_string Jsont.json json with
17+ | Ok s -> s
18+ | Error e -> failwith e
1920(* Test: parse pointer and print indices *)
21let test_parse pointer_str =