anil.recoil.org
RFC6901 JSON Pointer implementation in OCaml using jsont
docs
+262
-151
doc/tutorial.md
+262
-151
doc/tutorial.md
···
28
29
The JSON Pointer `/users/0/name` refers to the string `"Alice"`.
30
31
## Syntax: Reference Tokens
32
33
RFC 6901, Section 3 defines the syntax:
···
79
80
Multiple tokens navigate deeper into nested structures.
81
82
-
### Invalid Syntax
83
-
84
-
What happens if a pointer doesn't start with `/`?
85
-
86
-
```sh
87
-
$ jsonpp parse "foo"
88
-
ERROR: Invalid JSON Pointer: must be empty or start with '/': foo
89
-
```
90
-
91
-
The RFC is strict: non-empty pointers MUST start with `/`.
92
-
93
-
## Escaping Special Characters
94
-
95
-
RFC 6901, Section 3 explains the escaping rules:
96
-
97
-
> Because the characters '~' (%x7E) and '/' (%x2F) have special meanings
98
-
> in JSON Pointer, '~' needs to be encoded as '~0' and '/' needs to be
99
-
> encoded as '~1' when these characters appear in a reference token.
100
-
101
-
Why these specific characters?
102
-
- `/` separates tokens, so it must be escaped inside a token
103
-
- `~` is the escape character itself, so it must also be escaped
104
-
105
-
The escape sequences are:
106
-
- `~0` represents `~` (tilde)
107
-
- `~1` represents `/` (forward slash)
108
-
109
-
Let's see escaping in action:
110
-
111
-
```sh
112
-
$ jsonpp escape "hello"
113
-
hello
114
-
```
115
-
116
-
No special characters, no escaping needed.
117
-
118
-
```sh
119
-
$ jsonpp escape "a/b"
120
-
a~1b
121
-
```
122
-
123
-
The `/` becomes `~1`.
124
-
125
-
```sh
126
-
$ jsonpp escape "a~b"
127
-
a~0b
128
-
```
129
-
130
-
The `~` becomes `~0`.
131
132
-
```sh
133
-
$ jsonpp escape "~/"
134
-
~0~1
135
-
```
136
-
137
-
Both characters are escaped.
138
-
139
-
### Unescaping
140
-
141
-
And the reverse process:
142
-
143
-
```sh
144
-
$ jsonpp unescape "a~1b"
145
-
OK: a/b
146
-
```
147
148
-
```sh
149
-
$ jsonpp unescape "a~0b"
150
-
OK: a~b
151
```
152
153
-
### The Order Matters!
154
155
-
RFC 6901, Section 4 is careful to specify the unescaping order:
156
-
157
-
> Evaluation of each reference token begins by decoding any escaped
158
-
> character sequence. This is performed by first transforming any
159
-
> occurrence of the sequence '~1' to '/', and then transforming any
160
-
> occurrence of the sequence '~0' to '~'. By performing the substitutions
161
-
> in this order, an implementation avoids the error of turning '~01' first
162
-
> into '~1' and then into '/', which would be incorrect (the string '~01'
163
-
> correctly becomes '~1' after transformation).
164
165
-
Let's verify this tricky case:
166
167
```sh
168
-
$ jsonpp unescape "~01"
169
-
OK: ~1
170
```
171
172
-
If we unescaped `~0` first, `~01` would become `~1`, which would then become
173
-
`/`. But that's wrong! The sequence `~01` should become the literal string
174
-
`~1` (a tilde followed by the digit one).
175
-
176
-
Invalid escape sequences are rejected:
177
-
178
-
```sh
179
-
$ jsonpp unescape "~2"
180
-
ERROR: Invalid JSON Pointer: invalid escape sequence ~2
181
-
```
182
-
183
-
```sh
184
-
$ jsonpp unescape "hello~"
185
-
ERROR: Invalid JSON Pointer: incomplete escape sequence at end
186
-
```
187
188
## Evaluation: Navigating JSON
189
···
195
> the document. Each reference token in the JSON Pointer is evaluated
196
> sequentially.
197
198
Let's use the example JSON document from RFC 6901, Section 5:
199
200
```sh
···
222
OK: {"foo":["bar","baz"],"":0,"a/b":1,"c%d":2,"e^f":3,"g|h":4,"i\\j":5,"k\"l":6," ":7,"m~n":8}
223
```
224
225
-
The empty pointer returns the whole document.
226
227
### Object Member Access
228
···
263
264
### Keys with Special Characters
265
266
-
Now for the escape sequences:
267
268
```sh
269
$ jsonpp eval rfc6901_example.json "/a~1b"
270
OK: 1
271
```
272
273
-
The token `a~1b` unescapes to `a/b`, which is the key name.
274
275
```sh
276
$ jsonpp eval rfc6901_example.json "/m~0n"
277
OK: 8
278
```
279
280
-
The token `m~0n` unescapes to `m~n`.
281
282
### Other Special Characters (No Escaping Needed)
283
···
329
$ jsonpp eval rfc6901_example.json "/foo/0/invalid"
330
ERROR: JSON Pointer: cannot index into string with 'invalid'
331
File "-":
332
```
333
334
### Array Index Rules
···
393
394
But we'll see later that `-` is very useful for mutation operations!
395
396
-
## URI Fragment Encoding
397
-
398
-
JSON Pointers can be embedded in URIs. RFC 6901, Section 6 explains:
399
-
400
-
> A JSON Pointer can be represented in a URI fragment identifier by
401
-
> encoding it into octets using UTF-8, while percent-encoding those
402
-
> characters not allowed by the fragment rule in RFC 3986.
403
-
404
-
This adds percent-encoding on top of the `~0`/`~1` escaping:
405
-
406
-
```sh
407
-
$ jsonpp uri-fragment "/foo"
408
-
OK: /foo -> /foo
409
-
```
410
-
411
-
Simple pointers often don't need percent-encoding.
412
-
413
-
```sh
414
-
$ jsonpp uri-fragment "/a~1b"
415
-
OK: /a~1b -> /a~1b
416
-
```
417
-
418
-
The `~1` escape stays as-is (it's valid in URI fragments).
419
-
420
-
```sh
421
-
$ jsonpp uri-fragment "/c%d"
422
-
OK: /c%d -> /c%25d
423
-
```
424
-
425
-
The `%` character must be percent-encoded as `%25` in URIs!
426
-
427
-
```sh
428
-
$ jsonpp uri-fragment "/ "
429
-
OK: / -> /%20
430
-
```
431
-
432
-
Spaces become `%20`.
433
-
434
-
Here's the RFC example showing the URI fragment forms:
435
-
436
-
| JSON Pointer | URI Fragment | Value |
437
-
|-------------|-------------|-------|
438
-
| `""` | `#` | whole document |
439
-
| `"/foo"` | `#/foo` | `["bar", "baz"]` |
440
-
| `"/foo/0"` | `#/foo/0` | `"bar"` |
441
-
| `"/"` | `#/` | `0` |
442
-
| `"/a~1b"` | `#/a~1b` | `1` |
443
-
| `"/c%d"` | `#/c%25d` | `2` |
444
-
| `"/ "` | `#/%20` | `7` |
445
-
| `"/m~0n"` | `#/m~0n` | `8` |
446
-
447
## Mutation Operations
448
449
While RFC 6901 defines JSON Pointer for read-only access, RFC 6902
···
459
{"foo":"bar","baz":"qux"}
460
```
461
462
For arrays, `add` inserts BEFORE the specified index:
463
464
```sh
···
538
false
539
```
540
541
## Deeply Nested Structures
542
543
JSON Pointer handles arbitrarily deep nesting:
···
561
{"arr":[[1,99,2],[3,4]]}
562
```
563
564
## Summary
565
566
JSON Pointer (RFC 6901) provides a simple but powerful way to address
567
values within JSON documents:
568
569
1. **Syntax**: Pointers are strings of `/`-separated reference tokens
570
-
2. **Escaping**: Use `~0` for `~` and `~1` for `/` in tokens
571
3. **Evaluation**: Tokens navigate through objects (by key) and arrays (by index)
572
4. **URI Encoding**: Pointers can be percent-encoded for use in URIs
573
5. **Mutations**: Combined with JSON Patch (RFC 6902), pointers enable structured updates
···
28
29
The JSON Pointer `/users/0/name` refers to the string `"Alice"`.
30
31
+
In OCaml, this is represented by the `Jsont_pointer.t` type - a sequence
32
+
of navigation steps from the document root to a target value.
33
+
34
## Syntax: Reference Tokens
35
36
RFC 6901, Section 3 defines the syntax:
···
82
83
Multiple tokens navigate deeper into nested structures.
84
85
+
### The Index Type
86
87
+
Each reference token becomes an `Index.t` value in the library:
88
89
+
```ocaml
90
+
type t =
91
+
| Mem of string (* Object member access *)
92
+
| Nth of int (* Array index access *)
93
+
| End (* The special "-" marker for append operations *)
94
```
95
96
+
The `Mem` variant holds the **unescaped** member name - you work with the
97
+
actual key string (like `"a/b"`) and the library handles any escaping needed
98
+
for the JSON Pointer string representation.
99
100
+
### Invalid Syntax
101
102
+
What happens if a pointer doesn't start with `/`?
103
104
```sh
105
+
$ jsonpp parse "foo"
106
+
ERROR: Invalid JSON Pointer: must be empty or start with '/': foo
107
```
108
109
+
The RFC is strict: non-empty pointers MUST start with `/`.
110
111
## Evaluation: Navigating JSON
112
···
118
> the document. Each reference token in the JSON Pointer is evaluated
119
> sequentially.
120
121
+
In the library, this is the `Jsont_pointer.get` function:
122
+
123
+
```ocaml
124
+
val get : t -> Jsont.json -> Jsont.json
125
+
```
126
+
127
Let's use the example JSON document from RFC 6901, Section 5:
128
129
```sh
···
151
OK: {"foo":["bar","baz"],"":0,"a/b":1,"c%d":2,"e^f":3,"g|h":4,"i\\j":5,"k\"l":6," ":7,"m~n":8}
152
```
153
154
+
The empty pointer returns the whole document. In OCaml, this is
155
+
`Jsont_pointer.root`:
156
+
157
+
```ocaml
158
+
val root : t
159
+
(** The empty pointer that references the whole document. *)
160
+
```
161
162
### Object Member Access
163
···
198
199
### Keys with Special Characters
200
201
+
The RFC example includes keys with `/` and `~` characters:
202
203
```sh
204
$ jsonpp eval rfc6901_example.json "/a~1b"
205
OK: 1
206
```
207
208
+
The token `a~1b` refers to the key `a/b`. We'll explain this escaping
209
+
[below](#escaping-special-characters).
210
211
```sh
212
$ jsonpp eval rfc6901_example.json "/m~0n"
213
OK: 8
214
```
215
216
+
The token `m~0n` refers to the key `m~n`.
217
+
218
+
**Important**: When using the OCaml library programmatically, you don't need
219
+
to worry about escaping. The `Index.Mem` variant holds the literal key name:
220
+
221
+
```ocaml
222
+
(* To access the key "a/b", just use the literal string *)
223
+
let pointer = Jsont_pointer.make [Mem "a/b"]
224
+
225
+
(* The library escapes it when converting to string *)
226
+
let s = Jsont_pointer.to_string pointer (* "/a~1b" *)
227
+
```
228
229
### Other Special Characters (No Escaping Needed)
230
···
276
$ jsonpp eval rfc6901_example.json "/foo/0/invalid"
277
ERROR: JSON Pointer: cannot index into string with 'invalid'
278
File "-":
279
+
```
280
+
281
+
The library provides both exception-raising and result-returning variants:
282
+
283
+
```ocaml
284
+
val get : t -> Jsont.json -> Jsont.json
285
+
val get_result : t -> Jsont.json -> (Jsont.json, Jsont.Error.t) result
286
+
val find : t -> Jsont.json -> Jsont.json option
287
```
288
289
### Array Index Rules
···
348
349
But we'll see later that `-` is very useful for mutation operations!
350
351
## Mutation Operations
352
353
While RFC 6901 defines JSON Pointer for read-only access, RFC 6902
···
363
{"foo":"bar","baz":"qux"}
364
```
365
366
+
In OCaml:
367
+
368
+
```ocaml
369
+
val add : t -> Jsont.json -> value:Jsont.json -> Jsont.json
370
+
```
371
+
372
For arrays, `add` inserts BEFORE the specified index:
373
374
```sh
···
448
false
449
```
450
451
+
## Escaping Special Characters
452
+
453
+
RFC 6901, Section 3 explains the escaping rules:
454
+
455
+
> Because the characters '\~' (%x7E) and '/' (%x2F) have special meanings
456
+
> in JSON Pointer, '\~' needs to be encoded as '\~0' and '/' needs to be
457
+
> encoded as '\~1' when these characters appear in a reference token.
458
+
459
+
Why these specific characters?
460
+
- `/` separates tokens, so it must be escaped inside a token
461
+
- `~` is the escape character itself, so it must also be escaped
462
+
463
+
The escape sequences are:
464
+
- `~0` represents `~` (tilde)
465
+
- `~1` represents `/` (forward slash)
466
+
467
+
### The Library Handles Escaping Automatically
468
+
469
+
**Important**: When using `jsont-pointer` programmatically, you rarely need
470
+
to think about escaping. The `Index.Mem` variant stores unescaped strings,
471
+
and escaping happens automatically during serialization:
472
+
473
+
```ocaml
474
+
(* Create a pointer to key "a/b" - no escaping needed *)
475
+
let p = Jsont_pointer.make [Mem "a/b"]
476
+
477
+
(* Serialize to string - escaping happens automatically *)
478
+
let s = Jsont_pointer.to_string p (* Returns "/a~1b" *)
479
+
480
+
(* Parse from string - unescaping happens automatically *)
481
+
let p' = Jsont_pointer.of_string "/a~1b"
482
+
(* p' contains [Mem "a/b"] - the unescaped key *)
483
+
```
484
+
485
+
The `Token` module exposes the escaping functions if you need them:
486
+
487
+
```ocaml
488
+
module Token : sig
489
+
val escape : string -> string (* "a/b" -> "a~1b" *)
490
+
val unescape : string -> string (* "a~1b" -> "a/b" *)
491
+
end
492
+
```
493
+
494
+
### Escaping in Action
495
+
496
+
Let's see escaping with the CLI tool:
497
+
498
+
```sh
499
+
$ jsonpp escape "hello"
500
+
hello
501
+
```
502
+
503
+
No special characters, no escaping needed.
504
+
505
+
```sh
506
+
$ jsonpp escape "a/b"
507
+
a~1b
508
+
```
509
+
510
+
The `/` becomes `~1`.
511
+
512
+
```sh
513
+
$ jsonpp escape "a~b"
514
+
a~0b
515
+
```
516
+
517
+
The `~` becomes `~0`.
518
+
519
+
```sh
520
+
$ jsonpp escape "~/"
521
+
~0~1
522
+
```
523
+
524
+
Both characters are escaped.
525
+
526
+
### Unescaping
527
+
528
+
And the reverse process:
529
+
530
+
```sh
531
+
$ jsonpp unescape "a~1b"
532
+
OK: a/b
533
+
```
534
+
535
+
```sh
536
+
$ jsonpp unescape "a~0b"
537
+
OK: a~b
538
+
```
539
+
540
+
### The Order Matters!
541
+
542
+
RFC 6901, Section 4 is careful to specify the unescaping order:
543
+
544
+
> Evaluation of each reference token begins by decoding any escaped
545
+
> character sequence. This is performed by first transforming any
546
+
> occurrence of the sequence '~1' to '/', and then transforming any
547
+
> occurrence of the sequence '~0' to '~'. By performing the substitutions
548
+
> in this order, an implementation avoids the error of turning '~01' first
549
+
> into '~1' and then into '/', which would be incorrect (the string '~01'
550
+
> correctly becomes '~1' after transformation).
551
+
552
+
Let's verify this tricky case:
553
+
554
+
```sh
555
+
$ jsonpp unescape "~01"
556
+
OK: ~1
557
+
```
558
+
559
+
If we unescaped `~0` first, `~01` would become `~1`, which would then become
560
+
`/`. But that's wrong! The sequence `~01` should become the literal string
561
+
`~1` (a tilde followed by the digit one).
562
+
563
+
Invalid escape sequences are rejected:
564
+
565
+
```sh
566
+
$ jsonpp unescape "~2"
567
+
ERROR: Invalid JSON Pointer: invalid escape sequence ~2
568
+
```
569
+
570
+
```sh
571
+
$ jsonpp unescape "hello~"
572
+
ERROR: Invalid JSON Pointer: incomplete escape sequence at end
573
+
```
574
+
575
+
## URI Fragment Encoding
576
+
577
+
JSON Pointers can be embedded in URIs. RFC 6901, Section 6 explains:
578
+
579
+
> A JSON Pointer can be represented in a URI fragment identifier by
580
+
> encoding it into octets using UTF-8, while percent-encoding those
581
+
> characters not allowed by the fragment rule in RFC 3986.
582
+
583
+
This adds percent-encoding on top of the `~0`/`~1` escaping:
584
+
585
+
```sh
586
+
$ jsonpp uri-fragment "/foo"
587
+
OK: /foo -> /foo
588
+
```
589
+
590
+
Simple pointers often don't need percent-encoding.
591
+
592
+
```sh
593
+
$ jsonpp uri-fragment "/a~1b"
594
+
OK: /a~1b -> /a~1b
595
+
```
596
+
597
+
The `~1` escape stays as-is (it's valid in URI fragments).
598
+
599
+
```sh
600
+
$ jsonpp uri-fragment "/c%d"
601
+
OK: /c%d -> /c%25d
602
+
```
603
+
604
+
The `%` character must be percent-encoded as `%25` in URIs!
605
+
606
+
```sh
607
+
$ jsonpp uri-fragment "/ "
608
+
OK: / -> /%20
609
+
```
610
+
611
+
Spaces become `%20`.
612
+
613
+
The library provides functions for URI fragment encoding:
614
+
615
+
```ocaml
616
+
val to_uri_fragment : t -> string
617
+
val of_uri_fragment : string -> t
618
+
val jsont_uri_fragment : t Jsont.t
619
+
```
620
+
621
+
Here's the RFC example showing the URI fragment forms:
622
+
623
+
| JSON Pointer | URI Fragment | Value |
624
+
|-------------|-------------|-------|
625
+
| `""` | `#` | whole document |
626
+
| `"/foo"` | `#/foo` | `["bar", "baz"]` |
627
+
| `"/foo/0"` | `#/foo/0` | `"bar"` |
628
+
| `"/"` | `#/` | `0` |
629
+
| `"/a~1b"` | `#/a~1b` | `1` |
630
+
| `"/c%d"` | `#/c%25d` | `2` |
631
+
| `"/ "` | `#/%20` | `7` |
632
+
| `"/m~0n"` | `#/m~0n` | `8` |
633
+
634
## Deeply Nested Structures
635
636
JSON Pointer handles arbitrarily deep nesting:
···
654
{"arr":[[1,99,2],[3,4]]}
655
```
656
657
+
## Jsont Integration
658
+
659
+
The library integrates with the `Jsont` codec system for typed access:
660
+
661
+
```ocaml
662
+
(* Codec for JSON Pointers as JSON strings *)
663
+
val jsont : t Jsont.t
664
+
665
+
(* Query combinators *)
666
+
val path : ?absent:'a -> t -> 'a Jsont.t -> 'a Jsont.t
667
+
val set_path : ?allow_absent:bool -> 'a Jsont.t -> t -> 'a -> Jsont.json Jsont.t
668
+
val update_path : ?absent:'a -> t -> 'a Jsont.t -> Jsont.json Jsont.t
669
+
val delete_path : ?allow_absent:bool -> t -> Jsont.json Jsont.t
670
+
```
671
+
672
+
These allow you to use JSON Pointers with typed codecs rather than raw
673
+
`Jsont.json` values.
674
+
675
## Summary
676
677
JSON Pointer (RFC 6901) provides a simple but powerful way to address
678
values within JSON documents:
679
680
1. **Syntax**: Pointers are strings of `/`-separated reference tokens
681
+
2. **Escaping**: Use `~0` for `~` and `~1` for `/` in tokens (handled automatically by the library)
682
3. **Evaluation**: Tokens navigate through objects (by key) and arrays (by index)
683
4. **URI Encoding**: Pointers can be percent-encoded for use in URIs
684
5. **Mutations**: Combined with JSON Patch (RFC 6902), pointers enable structured updates