slices.network
Auto-indexing service and GraphQL API for AT Protocol Records
quickslice.slices.network/
atproto
gleam
graphql
refactor: reorganize database helpers into proper layers
- Merge cursor.gleam into database/queries/pagination.gleam
- Move where_clause.gleam to database/queries/where_clause.gleam
- Move where_converter.gleam to graphql/where_converter.gleam
- Update all imports across codebase
- Rename cursor_test.gleam to pagination_test.gleam
This establishes cleaner layering:
- database/queries/ for pure SQL building
- graphql/ for GraphQL-specific adapters
chadtmiller.com
+804
dev-docs/plans/2025-12-10-database-helpers-reorganization.md
+804
dev-docs/plans/2025-12-10-database-helpers-reorganization.md
···
1
+
# Database Helpers Reorganization Implementation Plan
2
+
3
+
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
4
+
5
+
**Goal:** Reorganize vestigial database helpers (`cursor`, `where_clause`, `where_converter`) from `src/` root into proper layered locations with consolidation.
6
+
7
+
**Architecture:**
8
+
- Merge `cursor.gleam` into `database/queries/pagination.gleam` (both handle pagination concerns)
9
+
- Move `where_clause.gleam` to `database/queries/where_clause.gleam` (pure SQL builder)
10
+
- Move `where_converter.gleam` to `graphql/where_converter.gleam` (GraphQL→SQL bridge belongs in GraphQL layer)
11
+
12
+
**Tech Stack:** Gleam, SQLite (sqlight)
13
+
14
+
---
15
+
16
+
## Summary of Changes
17
+
18
+
| Old Location | New Location | Action |
19
+
|--------------|--------------|--------|
20
+
| `src/cursor.gleam` | `src/database/queries/pagination.gleam` | Merge into existing file |
21
+
| `src/where_clause.gleam` | `src/database/queries/where_clause.gleam` | Move |
22
+
| `src/where_converter.gleam` | `src/graphql/where_converter.gleam` | Move (new graphql/ folder) |
23
+
24
+
### Files That Need Import Updates
25
+
26
+
**For `cursor` → `database/queries/pagination`:**
27
+
- `src/graphql_gleam.gleam` (lines 6, 120, 283, 328)
28
+
- `src/database/repositories/records.gleam` (lines 1, 553, 556, 630, 711, 714, 788, 961, 964, 1041, 1186, 1189, 1266)
29
+
- `test/cursor_test.gleam` → rename to `test/pagination_test.gleam`
30
+
31
+
**For `where_clause` → `database/queries/where_clause`:**
32
+
- `src/where_converter.gleam` (line 10) - will become `graphql/where_converter`
33
+
- `src/database/queries/aggregates.gleam` (line 14)
34
+
- `src/database/repositories/records.gleam` (line 15)
35
+
- `test/where_clause_test.gleam`
36
+
- `test/where_sql_builder_test.gleam`
37
+
- `test/where_integration_test.gleam`
38
+
- `test/where_edge_cases_test.gleam`
39
+
- `test/database_aggregation_test.gleam`
40
+
41
+
**For `where_converter` → `graphql/where_converter`:**
42
+
- `src/graphql_gleam.gleam` (line 33)
43
+
- `test/graphql_where_integration_test.gleam` (line 15)
44
+
45
+
---
46
+
47
+
## Task 1: Move where_clause.gleam to database/queries/
48
+
49
+
**Files:**
50
+
- Move: `src/where_clause.gleam` → `src/database/queries/where_clause.gleam`
51
+
52
+
**Step 1: Copy the file to new location**
53
+
54
+
```bash
55
+
cp server/src/where_clause.gleam server/src/database/queries/where_clause.gleam
56
+
```
57
+
58
+
**Step 2: Verify file exists at new location**
59
+
60
+
```bash
61
+
ls -la server/src/database/queries/where_clause.gleam
62
+
```
63
+
64
+
Expected: File exists
65
+
66
+
**Step 3: Delete old file**
67
+
68
+
```bash
69
+
rm server/src/where_clause.gleam
70
+
```
71
+
72
+
**Step 4: Run build to see what breaks**
73
+
74
+
```bash
75
+
cd server && gleam build
76
+
```
77
+
78
+
Expected: FAIL with import errors (this is expected - we'll fix in Task 4)
79
+
80
+
---
81
+
82
+
## Task 2: Move where_converter.gleam to graphql/
83
+
84
+
**Files:**
85
+
- Create: `src/graphql/` directory
86
+
- Move: `src/where_converter.gleam` → `src/graphql/where_converter.gleam`
87
+
88
+
**Step 1: Create graphql directory**
89
+
90
+
```bash
91
+
mkdir -p server/src/graphql
92
+
```
93
+
94
+
**Step 2: Copy the file to new location**
95
+
96
+
```bash
97
+
cp server/src/where_converter.gleam server/src/graphql/where_converter.gleam
98
+
```
99
+
100
+
**Step 3: Update internal import in graphql/where_converter.gleam**
101
+
102
+
Change line 10 from:
103
+
```gleam
104
+
import where_clause
105
+
```
106
+
107
+
To:
108
+
```gleam
109
+
import database/queries/where_clause
110
+
```
111
+
112
+
**Step 4: Delete old file**
113
+
114
+
```bash
115
+
rm server/src/where_converter.gleam
116
+
```
117
+
118
+
**Step 5: Run build to see what breaks**
119
+
120
+
```bash
121
+
cd server && gleam build
122
+
```
123
+
124
+
Expected: FAIL with import errors (this is expected - we'll fix in Task 4)
125
+
126
+
---
127
+
128
+
## Task 3: Merge cursor.gleam into pagination.gleam
129
+
130
+
**Files:**
131
+
- Merge: `src/cursor.gleam` → `src/database/queries/pagination.gleam`
132
+
- Delete: `src/cursor.gleam`
133
+
134
+
**Step 1: Create the merged pagination.gleam**
135
+
136
+
Replace `server/src/database/queries/pagination.gleam` with the merged content below. This combines:
137
+
- All cursor types and functions
138
+
- Existing pagination helpers
139
+
- Eliminates `RecordLike` type by using `Record` directly
140
+
141
+
```gleam
142
+
/// Pagination utilities including cursor encoding/decoding and ORDER BY building.
143
+
///
144
+
/// Cursors encode the position in a result set as base64(field1|field2|...|cid)
145
+
/// to enable stable pagination even when new records are inserted.
146
+
///
147
+
/// The cursor format:
148
+
/// - All sort field values are included in the cursor
149
+
/// - Values are separated by pipe (|) characters
150
+
/// - CID is always the last element as the ultimate tiebreaker
151
+
import database/types.{type Record}
152
+
import gleam/bit_array
153
+
import gleam/dict
154
+
import gleam/dynamic
155
+
import gleam/dynamic/decode
156
+
import gleam/float
157
+
import gleam/int
158
+
import gleam/json
159
+
import gleam/list
160
+
import gleam/option.{type Option, None, Some}
161
+
import gleam/result
162
+
import gleam/string
163
+
164
+
// ===== Cursor Types =====
165
+
166
+
/// Decoded cursor components for pagination
167
+
pub type DecodedCursor {
168
+
DecodedCursor(
169
+
/// Field values in the order they appear in sortBy
170
+
field_values: List(String),
171
+
/// CID (always the last element)
172
+
cid: String,
173
+
)
174
+
}
175
+
176
+
// ===== Base64 Encoding/Decoding =====
177
+
178
+
/// Encodes a string to URL-safe base64 without padding
179
+
pub fn encode_base64(input: String) -> String {
180
+
let bytes = bit_array.from_string(input)
181
+
bit_array.base64_url_encode(bytes, False)
182
+
}
183
+
184
+
/// Decodes a URL-safe base64 string without padding
185
+
pub fn decode_base64(input: String) -> Result(String, String) {
186
+
case bit_array.base64_url_decode(input) {
187
+
Ok(bytes) ->
188
+
case bit_array.to_string(bytes) {
189
+
Ok(str) -> Ok(str)
190
+
Error(_) -> Error("Invalid UTF-8 in cursor")
191
+
}
192
+
Error(_) -> Error("Failed to decode base64")
193
+
}
194
+
}
195
+
196
+
// ===== Field Value Extraction =====
197
+
198
+
/// Extracts a field value from a record.
199
+
///
200
+
/// Handles both table columns and JSON fields with nested paths.
201
+
pub fn extract_field_value(record: Record, field: String) -> String {
202
+
case field {
203
+
"uri" -> record.uri
204
+
"cid" -> record.cid
205
+
"did" -> record.did
206
+
"collection" -> record.collection
207
+
"indexed_at" -> record.indexed_at
208
+
_ -> extract_json_field(record.json, field)
209
+
}
210
+
}
211
+
212
+
/// Extracts a value from a JSON string using a field path
213
+
fn extract_json_field(json_str: String, field: String) -> String {
214
+
let decoder = decode.dict(decode.string, decode.dynamic)
215
+
case json.parse(json_str, decoder) {
216
+
Error(_) -> "NULL"
217
+
Ok(parsed_dict) -> {
218
+
let path_parts = string.split(field, ".")
219
+
extract_from_dict(parsed_dict, path_parts)
220
+
}
221
+
}
222
+
}
223
+
224
+
/// Recursively extracts a value from a dict using a path
225
+
fn extract_from_dict(
226
+
d: dict.Dict(String, dynamic.Dynamic),
227
+
path: List(String),
228
+
) -> String {
229
+
case path {
230
+
[] -> "NULL"
231
+
[key] -> {
232
+
case dict.get(d, key) {
233
+
Ok(val) -> dynamic_to_string(val)
234
+
Error(_) -> "NULL"
235
+
}
236
+
}
237
+
[key, ..rest] -> {
238
+
case dict.get(d, key) {
239
+
Ok(val) -> {
240
+
case decode.run(val, decode.dict(decode.string, decode.dynamic)) {
241
+
Ok(nested_dict) -> extract_from_dict(nested_dict, rest)
242
+
Error(_) -> "NULL"
243
+
}
244
+
}
245
+
Error(_) -> "NULL"
246
+
}
247
+
}
248
+
}
249
+
}
250
+
251
+
/// Converts a dynamic JSON value to a string representation
252
+
fn dynamic_to_string(value: dynamic.Dynamic) -> String {
253
+
case decode.run(value, decode.string) {
254
+
Ok(s) -> s
255
+
Error(_) ->
256
+
case decode.run(value, decode.int) {
257
+
Ok(i) -> int.to_string(i)
258
+
Error(_) ->
259
+
case decode.run(value, decode.float) {
260
+
Ok(f) -> float.to_string(f)
261
+
Error(_) ->
262
+
case decode.run(value, decode.bool) {
263
+
Ok(b) ->
264
+
case b {
265
+
True -> "true"
266
+
False -> "false"
267
+
}
268
+
Error(_) -> "NULL"
269
+
}
270
+
}
271
+
}
272
+
}
273
+
}
274
+
275
+
// ===== Cursor Generation and Decoding =====
276
+
277
+
/// Generates a cursor from a record based on the sort configuration.
278
+
///
279
+
/// Extracts all sort field values from the record and encodes them along with the CID.
280
+
/// Format: `base64(field1_value|field2_value|...|cid)`
281
+
pub fn generate_cursor_from_record(
282
+
record: Record,
283
+
sort_by: Option(List(#(String, String))),
284
+
) -> String {
285
+
let cursor_parts = case sort_by {
286
+
None -> []
287
+
Some(sort_fields) -> {
288
+
list.map(sort_fields, fn(sort_field) {
289
+
let #(field, _direction) = sort_field
290
+
extract_field_value(record, field)
291
+
})
292
+
}
293
+
}
294
+
295
+
let all_parts = list.append(cursor_parts, [record.cid])
296
+
let cursor_content = string.join(all_parts, "|")
297
+
encode_base64(cursor_content)
298
+
}
299
+
300
+
/// Decodes a base64-encoded cursor back into its components.
301
+
///
302
+
/// The cursor format is: `base64(field1|field2|...|cid)`
303
+
pub fn decode_cursor(
304
+
cursor: String,
305
+
sort_by: Option(List(#(String, String))),
306
+
) -> Result(DecodedCursor, String) {
307
+
use decoded_str <- result.try(decode_base64(cursor))
308
+
309
+
let parts = string.split(decoded_str, "|")
310
+
311
+
let expected_parts = case sort_by {
312
+
None -> 1
313
+
Some(fields) -> list.length(fields) + 1
314
+
}
315
+
316
+
case list.length(parts) == expected_parts {
317
+
False ->
318
+
Error(
319
+
"Invalid cursor format: expected "
320
+
<> int.to_string(expected_parts)
321
+
<> " parts, got "
322
+
<> int.to_string(list.length(parts)),
323
+
)
324
+
True -> {
325
+
case list.reverse(parts) {
326
+
[cid, ..rest_reversed] -> {
327
+
let field_values = list.reverse(rest_reversed)
328
+
Ok(DecodedCursor(field_values: field_values, cid: cid))
329
+
}
330
+
[] -> Error("Cursor has no parts")
331
+
}
332
+
}
333
+
}
334
+
}
335
+
336
+
// ===== Cursor WHERE Clause Building =====
337
+
338
+
/// Builds cursor-based WHERE conditions for proper multi-field pagination.
339
+
///
340
+
/// Creates progressive equality checks for stable multi-field sorting.
341
+
/// For each field, we OR together:
342
+
/// 1. field1 > cursor_value1
343
+
/// 2. field1 = cursor_value1 AND field2 > cursor_value2
344
+
/// 3. field1 = cursor_value1 AND field2 = cursor_value2 AND field3 > cursor_value3
345
+
/// ... and so on
346
+
/// Finally: all fields equal AND cid > cursor_cid
347
+
///
348
+
/// Returns: #(where_clause_sql, bind_values)
349
+
pub fn build_cursor_where_clause(
350
+
decoded_cursor: DecodedCursor,
351
+
sort_by: Option(List(#(String, String))),
352
+
is_before: Bool,
353
+
) -> #(String, List(String)) {
354
+
let sort_fields = case sort_by {
355
+
None -> []
356
+
Some(fields) -> fields
357
+
}
358
+
359
+
case list.is_empty(sort_fields) {
360
+
True -> #("1=1", [])
361
+
False -> {
362
+
let clauses =
363
+
build_progressive_clauses(
364
+
sort_fields,
365
+
decoded_cursor.field_values,
366
+
decoded_cursor.cid,
367
+
is_before,
368
+
)
369
+
370
+
let sql = "(" <> string.join(clauses.0, " OR ") <> ")"
371
+
#(sql, clauses.1)
372
+
}
373
+
}
374
+
}
375
+
376
+
/// Builds progressive equality clauses for cursor pagination
377
+
fn build_progressive_clauses(
378
+
sort_fields: List(#(String, String)),
379
+
field_values: List(String),
380
+
cid: String,
381
+
is_before: Bool,
382
+
) -> #(List(String), List(String)) {
383
+
let #(clauses, params) =
384
+
list.index_map(sort_fields, fn(field, i) {
385
+
let #(equality_parts, equality_params) = case i {
386
+
0 -> #([], [])
387
+
_ -> {
388
+
list.range(0, i - 1)
389
+
|> list.fold(#([], []), fn(eq_acc, j) {
390
+
let #(eq_parts, eq_params) = eq_acc
391
+
let prior_field =
392
+
list_at(sort_fields, j) |> result.unwrap(#("", ""))
393
+
let value = list_at(field_values, j) |> result.unwrap("")
394
+
395
+
let field_ref = build_cursor_field_reference(prior_field.0)
396
+
let new_part = field_ref <> " = ?"
397
+
let new_params = list.append(eq_params, [value])
398
+
399
+
#(list.append(eq_parts, [new_part]), new_params)
400
+
})
401
+
}
402
+
}
403
+
404
+
let value = list_at(field_values, i) |> result.unwrap("")
405
+
406
+
let comparison_op = get_comparison_operator(field.1, is_before)
407
+
let field_ref = build_cursor_field_reference(field.0)
408
+
409
+
let comparison_part = field_ref <> " " <> comparison_op <> " ?"
410
+
let all_parts = list.append(equality_parts, [comparison_part])
411
+
let all_params = list.append(equality_params, [value])
412
+
413
+
let clause = "(" <> string.join(all_parts, " AND ") <> ")"
414
+
415
+
#(clause, all_params)
416
+
})
417
+
|> list.unzip
418
+
|> fn(unzipped) {
419
+
let flattened_params = list.flatten(unzipped.1)
420
+
#(unzipped.0, flattened_params)
421
+
}
422
+
423
+
let #(final_equality_parts, final_equality_params) =
424
+
list.index_map(sort_fields, fn(field, j) {
425
+
let value = list_at(field_values, j) |> result.unwrap("")
426
+
let field_ref = build_cursor_field_reference(field.0)
427
+
#(field_ref <> " = ?", value)
428
+
})
429
+
|> list.unzip
430
+
431
+
let last_field = list.last(sort_fields) |> result.unwrap(#("", "desc"))
432
+
let cid_comparison_op = get_comparison_operator(last_field.1, is_before)
433
+
434
+
let final_parts =
435
+
list.append(final_equality_parts, ["cid " <> cid_comparison_op <> " ?"])
436
+
let final_params = list.append(final_equality_params, [cid])
437
+
438
+
let final_clause = "(" <> string.join(final_parts, " AND ") <> ")"
439
+
let all_clauses = list.append(clauses, [final_clause])
440
+
let all_params = list.append(params, final_params)
441
+
442
+
#(all_clauses, all_params)
443
+
}
444
+
445
+
/// Builds a field reference for cursor SQL queries (handles JSON fields)
446
+
fn build_cursor_field_reference(field: String) -> String {
447
+
case field {
448
+
"uri" | "cid" | "did" | "collection" | "indexed_at" -> field
449
+
_ -> {
450
+
let json_path = "$." <> string.replace(field, ".", ".")
451
+
"json_extract(json, '" <> json_path <> "')"
452
+
}
453
+
}
454
+
}
455
+
456
+
/// Gets the comparison operator based on sort direction and pagination direction
457
+
fn get_comparison_operator(direction: String, is_before: Bool) -> String {
458
+
let is_desc = string.lowercase(direction) == "desc"
459
+
460
+
case is_before {
461
+
True ->
462
+
case is_desc {
463
+
True -> ">"
464
+
False -> "<"
465
+
}
466
+
False ->
467
+
case is_desc {
468
+
True -> "<"
469
+
False -> ">"
470
+
}
471
+
}
472
+
}
473
+
474
+
/// Helper to get an element at an index from a list
475
+
fn list_at(l: List(a), index: Int) -> Result(a, Nil) {
476
+
l
477
+
|> list.drop(index)
478
+
|> list.first
479
+
}
480
+
481
+
// ===== Sort Direction Helpers =====
482
+
483
+
/// Reverses sort direction for backward pagination
484
+
pub fn reverse_sort_direction(direction: String) -> String {
485
+
case string.lowercase(direction) {
486
+
"asc" -> "desc"
487
+
"desc" -> "asc"
488
+
_ -> "asc"
489
+
}
490
+
}
491
+
492
+
/// Reverses all sort fields for backward pagination
493
+
pub fn reverse_sort_fields(
494
+
sort_fields: List(#(String, String)),
495
+
) -> List(#(String, String)) {
496
+
list.map(sort_fields, fn(field) {
497
+
let #(field_name, direction) = field
498
+
#(field_name, reverse_sort_direction(direction))
499
+
})
500
+
}
501
+
502
+
// ===== ORDER BY Building =====
503
+
504
+
/// Builds an ORDER BY clause from sort fields
505
+
/// use_table_prefix: if True, prefixes table columns with "record." for joins
506
+
pub fn build_order_by(
507
+
sort_fields: List(#(String, String)),
508
+
use_table_prefix: Bool,
509
+
) -> String {
510
+
let order_parts =
511
+
list.map(sort_fields, fn(field) {
512
+
let #(field_name, direction) = field
513
+
let table_prefix = case use_table_prefix {
514
+
True -> "record."
515
+
False -> ""
516
+
}
517
+
let field_ref = case field_name {
518
+
"uri" | "cid" | "did" | "collection" | "indexed_at" ->
519
+
table_prefix <> field_name
520
+
"createdAt" | "indexedAt" -> {
521
+
let json_field =
522
+
"json_extract(" <> table_prefix <> "json, '$." <> field_name <> "')"
523
+
"CASE
524
+
WHEN " <> json_field <> " IS NULL THEN NULL
525
+
WHEN datetime(" <> json_field <> ") IS NULL THEN NULL
526
+
ELSE " <> json_field <> "
527
+
END"
528
+
}
529
+
_ ->
530
+
"json_extract(" <> table_prefix <> "json, '$." <> field_name <> "')"
531
+
}
532
+
let dir = case string.lowercase(direction) {
533
+
"asc" -> "ASC"
534
+
_ -> "DESC"
535
+
}
536
+
field_ref <> " " <> dir <> " NULLS LAST"
537
+
})
538
+
539
+
case list.is_empty(order_parts) {
540
+
True -> {
541
+
let prefix = case use_table_prefix {
542
+
True -> "record."
543
+
False -> ""
544
+
}
545
+
prefix <> "indexed_at DESC NULLS LAST"
546
+
}
547
+
False -> string.join(order_parts, ", ")
548
+
}
549
+
}
550
+
```
551
+
552
+
**Step 2: Delete old cursor.gleam**
553
+
554
+
```bash
555
+
rm server/src/cursor.gleam
556
+
```
557
+
558
+
**Step 3: Run build to see what breaks**
559
+
560
+
```bash
561
+
cd server && gleam build
562
+
```
563
+
564
+
Expected: FAIL with import errors (we'll fix in Task 4)
565
+
566
+
---
567
+
568
+
## Task 4: Update All Imports
569
+
570
+
**Files to modify:**
571
+
- `src/graphql_gleam.gleam`
572
+
- `src/database/repositories/records.gleam`
573
+
- `src/database/queries/aggregates.gleam`
574
+
- `test/cursor_test.gleam` (rename to `test/pagination_test.gleam`)
575
+
- `test/where_clause_test.gleam`
576
+
- `test/where_sql_builder_test.gleam`
577
+
- `test/where_integration_test.gleam`
578
+
- `test/where_edge_cases_test.gleam`
579
+
- `test/database_aggregation_test.gleam`
580
+
- `test/graphql_where_integration_test.gleam`
581
+
582
+
**Step 1: Update graphql_gleam.gleam**
583
+
584
+
Change line 6 from:
585
+
```gleam
586
+
import cursor
587
+
```
588
+
To:
589
+
```gleam
590
+
import database/queries/pagination
591
+
```
592
+
593
+
Change line 33 from:
594
+
```gleam
595
+
import where_converter
596
+
```
597
+
To:
598
+
```gleam
599
+
import graphql/where_converter
600
+
```
601
+
602
+
Update all usages of `cursor.` to `pagination.`:
603
+
- Line 120: `cursor.generate_cursor_from_record` → `pagination.generate_cursor_from_record`
604
+
- Line 283: `cursor.generate_cursor_from_record` → `pagination.generate_cursor_from_record`
605
+
- Line 328: `cursor.generate_cursor_from_record` → `pagination.generate_cursor_from_record`
606
+
607
+
Remove the now-unnecessary `record_to_record_like` conversion since `pagination` now works directly with `Record`:
608
+
- Line 121: `pagination.record_to_record_like(record)` → `record`
609
+
- Similar for lines 284 and 329
610
+
611
+
**Step 2: Update database/repositories/records.gleam**
612
+
613
+
Change line 1 from:
614
+
```gleam
615
+
import cursor
616
+
```
617
+
To:
618
+
```gleam
619
+
import database/queries/pagination
620
+
```
621
+
622
+
Change line 15 from:
623
+
```gleam
624
+
import where_clause
625
+
```
626
+
To:
627
+
```gleam
628
+
import database/queries/where_clause
629
+
```
630
+
631
+
Update all `cursor.` references to `pagination.`:
632
+
- Line 553: `cursor.decode_cursor` → `pagination.decode_cursor`
633
+
- Line 556: `cursor.build_cursor_where_clause` → `pagination.build_cursor_where_clause`
634
+
- Line 630: `cursor.generate_cursor_from_record` → `pagination.generate_cursor_from_record`
635
+
- And similar for lines 711, 714, 788, 961, 964, 1041, 1186, 1189, 1266
636
+
637
+
Remove `record_to_record_like` calls - pass `Record` directly.
638
+
639
+
**Step 3: Update database/queries/aggregates.gleam**
640
+
641
+
Change line 14 from:
642
+
```gleam
643
+
import where_clause
644
+
```
645
+
To:
646
+
```gleam
647
+
import database/queries/where_clause
648
+
```
649
+
650
+
**Step 4: Rename and update cursor_test.gleam**
651
+
652
+
```bash
653
+
mv server/test/cursor_test.gleam server/test/pagination_test.gleam
654
+
```
655
+
656
+
Update imports in `test/pagination_test.gleam`:
657
+
658
+
Change line 1 from:
659
+
```gleam
660
+
import cursor
661
+
```
662
+
To:
663
+
```gleam
664
+
import database/queries/pagination
665
+
import database/types.{Record}
666
+
```
667
+
668
+
Update all `cursor.` to `pagination.` and `cursor.RecordLike` to `Record`:
669
+
670
+
```gleam
671
+
// Change all occurrences of cursor.RecordLike to Record
672
+
// Change all occurrences of cursor. to pagination.
673
+
```
674
+
675
+
**Step 5: Update where_clause_test.gleam**
676
+
677
+
Change line 7 from:
678
+
```gleam
679
+
import where_clause
680
+
```
681
+
To:
682
+
```gleam
683
+
import database/queries/where_clause
684
+
```
685
+
686
+
**Step 6: Update where_sql_builder_test.gleam**
687
+
688
+
Change line 8 from:
689
+
```gleam
690
+
import where_clause
691
+
```
692
+
To:
693
+
```gleam
694
+
import database/queries/where_clause
695
+
```
696
+
697
+
**Step 7: Update where_integration_test.gleam**
698
+
699
+
Change line 12 from:
700
+
```gleam
701
+
import where_clause
702
+
```
703
+
To:
704
+
```gleam
705
+
import database/queries/where_clause
706
+
```
707
+
708
+
**Step 8: Update where_edge_cases_test.gleam**
709
+
710
+
Change line 13 from:
711
+
```gleam
712
+
import where_clause
713
+
```
714
+
To:
715
+
```gleam
716
+
import database/queries/where_clause
717
+
```
718
+
719
+
**Step 9: Update database_aggregation_test.gleam**
720
+
721
+
Change line 11 from:
722
+
```gleam
723
+
import where_clause
724
+
```
725
+
To:
726
+
```gleam
727
+
import database/queries/where_clause
728
+
```
729
+
730
+
**Step 10: Update graphql_where_integration_test.gleam**
731
+
732
+
Change line 15 from:
733
+
```gleam
734
+
import where_converter
735
+
```
736
+
To:
737
+
```gleam
738
+
import graphql/where_converter
739
+
```
740
+
741
+
**Step 11: Run build to verify**
742
+
743
+
```bash
744
+
cd server && gleam build
745
+
```
746
+
747
+
Expected: PASS
748
+
749
+
**Step 12: Run tests to verify**
750
+
751
+
```bash
752
+
cd server && gleam test
753
+
```
754
+
755
+
Expected: All tests pass
756
+
757
+
**Step 13: Commit**
758
+
759
+
```bash
760
+
git add -A
761
+
git commit -m "refactor: reorganize database helpers into proper layers
762
+
763
+
- Merge cursor.gleam into database/queries/pagination.gleam
764
+
- Move where_clause.gleam to database/queries/where_clause.gleam
765
+
- Move where_converter.gleam to graphql/where_converter.gleam
766
+
- Update all imports across codebase
767
+
- Rename cursor_test.gleam to pagination_test.gleam
768
+
769
+
This establishes cleaner layering:
770
+
- database/queries/ for pure SQL building
771
+
- graphql/ for GraphQL-specific adapters"
772
+
```
773
+
774
+
---
775
+
776
+
## Final Verification
777
+
778
+
After all tasks complete:
779
+
780
+
```bash
781
+
cd server && gleam build && gleam test
782
+
```
783
+
784
+
Expected: Build succeeds, all tests pass.
785
+
786
+
### New File Structure
787
+
788
+
```
789
+
server/src/
790
+
├── database/
791
+
│ └── queries/
792
+
│ ├── aggregates.gleam (updated import)
793
+
│ ├── pagination.gleam (merged: cursor + pagination)
794
+
│ └── where_clause.gleam (moved from src/)
795
+
├── graphql/
796
+
│ └── where_converter.gleam (moved from src/, new folder)
797
+
└── graphql_gleam.gleam (updated imports)
798
+
```
799
+
800
+
### Future Opportunity
801
+
802
+
The new `graphql/` folder sets up for consolidating other GraphQL-related code:
803
+
- `graphql_gleam.gleam` could move to `graphql/resolver.gleam` or similar
804
+
- Other GraphQL utilities could be grouped here
-366
server/src/cursor.gleam
-366
server/src/cursor.gleam
···
1
-
/// Cursor-based pagination utilities.
2
-
///
3
-
/// Cursors encode the position in a result set as base64(field1|field2|...|cid)
4
-
/// to enable stable pagination even when new records are inserted.
5
-
///
6
-
/// The cursor format:
7
-
/// - All sort field values are included in the cursor
8
-
/// - Values are separated by pipe (|) characters
9
-
/// - CID is always the last element as the ultimate tiebreaker
10
-
import gleam/bit_array
11
-
import gleam/dict
12
-
import gleam/dynamic
13
-
import gleam/dynamic/decode
14
-
import gleam/float
15
-
import gleam/int
16
-
import gleam/json
17
-
import gleam/list
18
-
import gleam/option.{type Option, None, Some}
19
-
import gleam/result
20
-
import gleam/string
21
-
22
-
/// Decoded cursor components for pagination
23
-
pub type DecodedCursor {
24
-
DecodedCursor(
25
-
/// Field values in the order they appear in sortBy
26
-
field_values: List(String),
27
-
/// CID (always the last element)
28
-
cid: String,
29
-
)
30
-
}
31
-
32
-
/// Encodes a string to URL-safe base64 without padding
33
-
pub fn encode_base64(input: String) -> String {
34
-
let bytes = bit_array.from_string(input)
35
-
let encoded = bit_array.base64_url_encode(bytes, False)
36
-
encoded
37
-
}
38
-
39
-
/// Decodes a URL-safe base64 string without padding
40
-
pub fn decode_base64(input: String) -> Result(String, String) {
41
-
case bit_array.base64_url_decode(input) {
42
-
Ok(bytes) ->
43
-
case bit_array.to_string(bytes) {
44
-
Ok(str) -> Ok(str)
45
-
Error(_) -> Error("Invalid UTF-8 in cursor")
46
-
}
47
-
Error(_) -> Error("Failed to decode base64")
48
-
}
49
-
}
50
-
51
-
/// Record-like type for cursor generation
52
-
/// This allows cursor to work with any record type without importing database
53
-
pub type RecordLike {
54
-
RecordLike(
55
-
uri: String,
56
-
cid: String,
57
-
did: String,
58
-
collection: String,
59
-
json: String,
60
-
indexed_at: String,
61
-
)
62
-
}
63
-
64
-
/// Extracts a field value from a record.
65
-
///
66
-
/// Handles both table columns and JSON fields with nested paths.
67
-
pub fn extract_field_value(record: RecordLike, field: String) -> String {
68
-
case field {
69
-
"uri" -> record.uri
70
-
"cid" -> record.cid
71
-
"did" -> record.did
72
-
"collection" -> record.collection
73
-
"indexed_at" -> record.indexed_at
74
-
_ -> extract_json_field(record.json, field)
75
-
}
76
-
}
77
-
78
-
/// Extracts a value from a JSON string using a field path
79
-
fn extract_json_field(json_str: String, field: String) -> String {
80
-
// Parse the JSON as a dictionary
81
-
let decoder = decode.dict(decode.string, decode.dynamic)
82
-
case json.parse(json_str, decoder) {
83
-
Error(_) -> "NULL"
84
-
Ok(dict) -> {
85
-
// Split field path by dots for nested access
86
-
let path_parts = string.split(field, ".")
87
-
88
-
// Navigate through the JSON structure
89
-
extract_from_dict(dict, path_parts)
90
-
}
91
-
}
92
-
}
93
-
94
-
/// Recursively extracts a value from a dict using a path
95
-
fn extract_from_dict(
96
-
dict: dict.Dict(String, dynamic.Dynamic),
97
-
path: List(String),
98
-
) -> String {
99
-
case path {
100
-
[] -> "NULL"
101
-
[key] -> {
102
-
// Final key - extract and convert to string
103
-
case dict.get(dict, key) {
104
-
Ok(val) -> dynamic_to_string(val)
105
-
Error(_) -> "NULL"
106
-
}
107
-
}
108
-
[key, ..rest] -> {
109
-
// Intermediate key - try to decode as nested dict
110
-
case dict.get(dict, key) {
111
-
Ok(val) -> {
112
-
case decode.run(val, decode.dict(decode.string, decode.dynamic)) {
113
-
Ok(nested_dict) -> extract_from_dict(nested_dict, rest)
114
-
Error(_) -> "NULL"
115
-
}
116
-
}
117
-
Error(_) -> "NULL"
118
-
}
119
-
}
120
-
}
121
-
}
122
-
123
-
/// Converts a dynamic JSON value to a string representation
124
-
fn dynamic_to_string(value: dynamic.Dynamic) -> String {
125
-
// Try to decode as string
126
-
case decode.run(value, decode.string) {
127
-
Ok(s) -> s
128
-
Error(_) ->
129
-
// Try as int
130
-
case decode.run(value, decode.int) {
131
-
Ok(i) -> int.to_string(i)
132
-
Error(_) ->
133
-
// Try as float
134
-
case decode.run(value, decode.float) {
135
-
Ok(f) -> float.to_string(f)
136
-
Error(_) ->
137
-
// Try as bool
138
-
case decode.run(value, decode.bool) {
139
-
Ok(b) ->
140
-
case b {
141
-
True -> "true"
142
-
False -> "false"
143
-
}
144
-
Error(_) -> "NULL"
145
-
}
146
-
}
147
-
}
148
-
}
149
-
}
150
-
151
-
/// Generates a cursor from a record based on the sort configuration.
152
-
///
153
-
/// Extracts all sort field values from the record and encodes them along with the CID.
154
-
/// Format: `base64(field1_value|field2_value|...|cid)`
155
-
pub fn generate_cursor_from_record(
156
-
record: RecordLike,
157
-
sort_by: Option(List(#(String, String))),
158
-
) -> String {
159
-
let cursor_parts = case sort_by {
160
-
None -> []
161
-
Some(sort_fields) -> {
162
-
list.map(sort_fields, fn(sort_field) {
163
-
let #(field, _direction) = sort_field
164
-
extract_field_value(record, field)
165
-
})
166
-
}
167
-
}
168
-
169
-
// Always add CID as the final tiebreaker
170
-
let all_parts = list.append(cursor_parts, [record.cid])
171
-
172
-
// Join with pipe and encode
173
-
let cursor_content = string.join(all_parts, "|")
174
-
encode_base64(cursor_content)
175
-
}
176
-
177
-
/// Decodes a base64-encoded cursor back into its components.
178
-
///
179
-
/// The cursor format is: `base64(field1|field2|...|cid)`
180
-
pub fn decode_cursor(
181
-
cursor: String,
182
-
sort_by: Option(List(#(String, String))),
183
-
) -> Result(DecodedCursor, String) {
184
-
use decoded_str <- result.try(decode_base64(cursor))
185
-
186
-
let parts = string.split(decoded_str, "|")
187
-
188
-
// Validate cursor format matches sortBy fields
189
-
let expected_parts = case sort_by {
190
-
None -> 1
191
-
Some(fields) -> list.length(fields) + 1
192
-
}
193
-
194
-
case list.length(parts) == expected_parts {
195
-
False ->
196
-
Error(
197
-
"Invalid cursor format: expected "
198
-
<> int.to_string(expected_parts)
199
-
<> " parts, got "
200
-
<> int.to_string(list.length(parts)),
201
-
)
202
-
True -> {
203
-
// Last part is the CID
204
-
case list.reverse(parts) {
205
-
[cid, ..rest_reversed] -> {
206
-
let field_values = list.reverse(rest_reversed)
207
-
Ok(DecodedCursor(field_values: field_values, cid: cid))
208
-
}
209
-
[] -> Error("Cursor has no parts")
210
-
}
211
-
}
212
-
}
213
-
}
214
-
215
-
/// Builds cursor-based WHERE conditions for proper multi-field pagination.
216
-
///
217
-
/// Creates progressive equality checks for stable multi-field sorting.
218
-
/// For each field, we OR together:
219
-
/// 1. field1 > cursor_value1
220
-
/// 2. field1 = cursor_value1 AND field2 > cursor_value2
221
-
/// 3. field1 = cursor_value1 AND field2 = cursor_value2 AND field3 > cursor_value3
222
-
/// ... and so on
223
-
/// Finally: all fields equal AND cid > cursor_cid
224
-
///
225
-
/// Returns: #(where_clause_sql, bind_values)
226
-
pub fn build_cursor_where_clause(
227
-
decoded_cursor: DecodedCursor,
228
-
sort_by: Option(List(#(String, String))),
229
-
is_before: Bool,
230
-
) -> #(String, List(String)) {
231
-
let sort_fields = case sort_by {
232
-
None -> []
233
-
Some(fields) -> fields
234
-
}
235
-
236
-
case list.is_empty(sort_fields) {
237
-
True -> #("1=1", [])
238
-
False -> {
239
-
let clauses =
240
-
build_progressive_clauses(
241
-
sort_fields,
242
-
decoded_cursor.field_values,
243
-
decoded_cursor.cid,
244
-
is_before,
245
-
)
246
-
247
-
let sql = "(" <> string.join(clauses.0, " OR ") <> ")"
248
-
#(sql, clauses.1)
249
-
}
250
-
}
251
-
}
252
-
253
-
/// Builds progressive equality clauses for cursor pagination
254
-
fn build_progressive_clauses(
255
-
sort_fields: List(#(String, String)),
256
-
field_values: List(String),
257
-
cid: String,
258
-
is_before: Bool,
259
-
) -> #(List(String), List(String)) {
260
-
let _field_count = list.length(sort_fields)
261
-
262
-
// Build clauses for each level
263
-
let #(clauses, params) =
264
-
list.index_map(sort_fields, fn(field, i) {
265
-
// Build equality checks for fields [0..i-1]
266
-
let #(equality_parts, equality_params) = case i {
267
-
0 -> #([], [])
268
-
_ -> {
269
-
list.range(0, i - 1)
270
-
|> list.fold(#([], []), fn(eq_acc, j) {
271
-
let #(eq_parts, eq_params) = eq_acc
272
-
let prior_field =
273
-
list_at(sort_fields, j) |> result.unwrap(#("", ""))
274
-
let value = list_at(field_values, j) |> result.unwrap("")
275
-
276
-
let field_ref = build_field_reference(prior_field.0)
277
-
let new_part = field_ref <> " = ?"
278
-
let new_params = list.append(eq_params, [value])
279
-
280
-
#(list.append(eq_parts, [new_part]), new_params)
281
-
})
282
-
}
283
-
}
284
-
285
-
// Add comparison for current field
286
-
let value = list_at(field_values, i) |> result.unwrap("")
287
-
288
-
let comparison_op = get_comparison_operator(field.1, is_before)
289
-
let field_ref = build_field_reference(field.0)
290
-
291
-
let comparison_part = field_ref <> " " <> comparison_op <> " ?"
292
-
let all_parts = list.append(equality_parts, [comparison_part])
293
-
let all_params = list.append(equality_params, [value])
294
-
295
-
// Combine with AND
296
-
let clause = "(" <> string.join(all_parts, " AND ") <> ")"
297
-
298
-
#(clause, all_params)
299
-
})
300
-
|> list.unzip
301
-
|> fn(unzipped) {
302
-
// Flatten the params lists
303
-
let flattened_params = list.flatten(unzipped.1)
304
-
#(unzipped.0, flattened_params)
305
-
}
306
-
307
-
// Add final clause: all fields equal AND cid comparison
308
-
let #(final_equality_parts, final_equality_params) =
309
-
list.index_map(sort_fields, fn(field, j) {
310
-
let value = list_at(field_values, j) |> result.unwrap("")
311
-
let field_ref = build_field_reference(field.0)
312
-
#(field_ref <> " = ?", value)
313
-
})
314
-
|> list.unzip
315
-
316
-
// CID comparison uses the direction of the last sort field
317
-
let last_field = list.last(sort_fields) |> result.unwrap(#("", "desc"))
318
-
let cid_comparison_op = get_comparison_operator(last_field.1, is_before)
319
-
320
-
let final_parts =
321
-
list.append(final_equality_parts, ["cid " <> cid_comparison_op <> " ?"])
322
-
let final_params = list.append(final_equality_params, [cid])
323
-
324
-
let final_clause = "(" <> string.join(final_parts, " AND ") <> ")"
325
-
let all_clauses = list.append(clauses, [final_clause])
326
-
let all_params = list.append(params, final_params)
327
-
328
-
#(all_clauses, all_params)
329
-
}
330
-
331
-
/// Builds a field reference for SQL queries (handles JSON fields)
332
-
fn build_field_reference(field: String) -> String {
333
-
case field {
334
-
"uri" | "cid" | "did" | "collection" | "indexed_at" -> field
335
-
_ -> {
336
-
// JSON field - use json_extract with JSON path
337
-
let json_path = "$." <> string.replace(field, ".", ".")
338
-
"json_extract(json, '" <> json_path <> "')"
339
-
}
340
-
}
341
-
}
342
-
343
-
/// Gets the comparison operator based on sort direction and pagination direction
344
-
fn get_comparison_operator(direction: String, is_before: Bool) -> String {
345
-
let is_desc = string.lowercase(direction) == "desc"
346
-
347
-
case is_before {
348
-
True ->
349
-
case is_desc {
350
-
True -> ">"
351
-
False -> "<"
352
-
}
353
-
False ->
354
-
case is_desc {
355
-
True -> "<"
356
-
False -> ">"
357
-
}
358
-
}
359
-
}
360
-
361
-
/// Helper to get an element at an index from a list
362
-
fn list_at(list: List(a), index: Int) -> Result(a, Nil) {
363
-
list
364
-
|> list.drop(index)
365
-
|> list.first
366
-
}
+1
-1
server/src/database/queries/aggregates.gleam
+1
-1
server/src/database/queries/aggregates.gleam
···
1
+
import database/queries/where_clause
1
2
import database/types.{
2
3
type DateInterval, type GroupByField, Day, Hour, Month, SimpleField,
3
4
TruncatedField, Week,
···
11
12
import gleam/string
12
13
import lexicon_graphql/output/aggregate
13
14
import sqlight
14
-
import where_clause
15
15
16
16
// ===== Aggregation Support =====
17
17
+335
-14
server/src/database/queries/pagination.gleam
+335
-14
server/src/database/queries/pagination.gleam
···
1
-
import cursor
1
+
/// Pagination utilities including cursor encoding/decoding and ORDER BY building.
2
+
///
3
+
/// Cursors encode the position in a result set as base64(field1|field2|...|cid)
4
+
/// to enable stable pagination even when new records are inserted.
5
+
///
6
+
/// The cursor format:
7
+
/// - All sort field values are included in the cursor
8
+
/// - Values are separated by pipe (|) characters
9
+
/// - CID is always the last element as the ultimate tiebreaker
2
10
import database/types.{type Record}
11
+
import gleam/bit_array
12
+
import gleam/dict
13
+
import gleam/dynamic
14
+
import gleam/dynamic/decode
15
+
import gleam/float
16
+
import gleam/int
17
+
import gleam/json
3
18
import gleam/list
19
+
import gleam/option.{type Option, None, Some}
20
+
import gleam/result
4
21
import gleam/string
5
22
6
-
// ===== Pagination Helper Functions =====
23
+
// ===== Cursor Types =====
7
24
8
-
/// Converts a Record to a cursor.RecordLike for cursor encoding
9
-
pub fn record_to_record_like(record: Record) -> cursor.RecordLike {
10
-
cursor.RecordLike(
11
-
uri: record.uri,
12
-
cid: record.cid,
13
-
did: record.did,
14
-
collection: record.collection,
15
-
json: record.json,
16
-
indexed_at: record.indexed_at,
25
+
/// Decoded cursor components for pagination
26
+
pub type DecodedCursor {
27
+
DecodedCursor(
28
+
/// Field values in the order they appear in sortBy
29
+
field_values: List(String),
30
+
/// CID (always the last element)
31
+
cid: String,
17
32
)
18
33
}
19
34
35
+
// ===== Base64 Encoding/Decoding =====
36
+
37
+
/// Encodes a string to URL-safe base64 without padding
38
+
pub fn encode_base64(input: String) -> String {
39
+
let bytes = bit_array.from_string(input)
40
+
bit_array.base64_url_encode(bytes, False)
41
+
}
42
+
43
+
/// Decodes a URL-safe base64 string without padding
44
+
pub fn decode_base64(input: String) -> Result(String, String) {
45
+
case bit_array.base64_url_decode(input) {
46
+
Ok(bytes) ->
47
+
case bit_array.to_string(bytes) {
48
+
Ok(str) -> Ok(str)
49
+
Error(_) -> Error("Invalid UTF-8 in cursor")
50
+
}
51
+
Error(_) -> Error("Failed to decode base64")
52
+
}
53
+
}
54
+
55
+
// ===== Field Value Extraction =====
56
+
57
+
/// Extracts a field value from a record.
58
+
///
59
+
/// Handles both table columns and JSON fields with nested paths.
60
+
pub fn extract_field_value(record: Record, field: String) -> String {
61
+
case field {
62
+
"uri" -> record.uri
63
+
"cid" -> record.cid
64
+
"did" -> record.did
65
+
"collection" -> record.collection
66
+
"indexed_at" -> record.indexed_at
67
+
_ -> extract_json_field(record.json, field)
68
+
}
69
+
}
70
+
71
+
/// Extracts a value from a JSON string using a field path
72
+
fn extract_json_field(json_str: String, field: String) -> String {
73
+
let decoder = decode.dict(decode.string, decode.dynamic)
74
+
case json.parse(json_str, decoder) {
75
+
Error(_) -> "NULL"
76
+
Ok(parsed_dict) -> {
77
+
let path_parts = string.split(field, ".")
78
+
extract_from_dict(parsed_dict, path_parts)
79
+
}
80
+
}
81
+
}
82
+
83
+
/// Recursively extracts a value from a dict using a path
84
+
fn extract_from_dict(
85
+
d: dict.Dict(String, dynamic.Dynamic),
86
+
path: List(String),
87
+
) -> String {
88
+
case path {
89
+
[] -> "NULL"
90
+
[key] -> {
91
+
case dict.get(d, key) {
92
+
Ok(val) -> dynamic_to_string(val)
93
+
Error(_) -> "NULL"
94
+
}
95
+
}
96
+
[key, ..rest] -> {
97
+
case dict.get(d, key) {
98
+
Ok(val) -> {
99
+
case decode.run(val, decode.dict(decode.string, decode.dynamic)) {
100
+
Ok(nested_dict) -> extract_from_dict(nested_dict, rest)
101
+
Error(_) -> "NULL"
102
+
}
103
+
}
104
+
Error(_) -> "NULL"
105
+
}
106
+
}
107
+
}
108
+
}
109
+
110
+
/// Converts a dynamic JSON value to a string representation
111
+
fn dynamic_to_string(value: dynamic.Dynamic) -> String {
112
+
case decode.run(value, decode.string) {
113
+
Ok(s) -> s
114
+
Error(_) ->
115
+
case decode.run(value, decode.int) {
116
+
Ok(i) -> int.to_string(i)
117
+
Error(_) ->
118
+
case decode.run(value, decode.float) {
119
+
Ok(f) -> float.to_string(f)
120
+
Error(_) ->
121
+
case decode.run(value, decode.bool) {
122
+
Ok(b) ->
123
+
case b {
124
+
True -> "true"
125
+
False -> "false"
126
+
}
127
+
Error(_) -> "NULL"
128
+
}
129
+
}
130
+
}
131
+
}
132
+
}
133
+
134
+
// ===== Cursor Generation and Decoding =====
135
+
136
+
/// Generates a cursor from a record based on the sort configuration.
137
+
///
138
+
/// Extracts all sort field values from the record and encodes them along with the CID.
139
+
/// Format: `base64(field1_value|field2_value|...|cid)`
140
+
pub fn generate_cursor_from_record(
141
+
record: Record,
142
+
sort_by: Option(List(#(String, String))),
143
+
) -> String {
144
+
let cursor_parts = case sort_by {
145
+
None -> []
146
+
Some(sort_fields) -> {
147
+
list.map(sort_fields, fn(sort_field) {
148
+
let #(field, _direction) = sort_field
149
+
extract_field_value(record, field)
150
+
})
151
+
}
152
+
}
153
+
154
+
let all_parts = list.append(cursor_parts, [record.cid])
155
+
let cursor_content = string.join(all_parts, "|")
156
+
encode_base64(cursor_content)
157
+
}
158
+
159
+
/// Decodes a base64-encoded cursor back into its components.
160
+
///
161
+
/// The cursor format is: `base64(field1|field2|...|cid)`
162
+
pub fn decode_cursor(
163
+
cursor: String,
164
+
sort_by: Option(List(#(String, String))),
165
+
) -> Result(DecodedCursor, String) {
166
+
use decoded_str <- result.try(decode_base64(cursor))
167
+
168
+
let parts = string.split(decoded_str, "|")
169
+
170
+
let expected_parts = case sort_by {
171
+
None -> 1
172
+
Some(fields) -> list.length(fields) + 1
173
+
}
174
+
175
+
case list.length(parts) == expected_parts {
176
+
False ->
177
+
Error(
178
+
"Invalid cursor format: expected "
179
+
<> int.to_string(expected_parts)
180
+
<> " parts, got "
181
+
<> int.to_string(list.length(parts)),
182
+
)
183
+
True -> {
184
+
case list.reverse(parts) {
185
+
[cid, ..rest_reversed] -> {
186
+
let field_values = list.reverse(rest_reversed)
187
+
Ok(DecodedCursor(field_values: field_values, cid: cid))
188
+
}
189
+
[] -> Error("Cursor has no parts")
190
+
}
191
+
}
192
+
}
193
+
}
194
+
195
+
// ===== Cursor WHERE Clause Building =====
196
+
197
+
/// Builds cursor-based WHERE conditions for proper multi-field pagination.
198
+
///
199
+
/// Creates progressive equality checks for stable multi-field sorting.
200
+
/// For each field, we OR together:
201
+
/// 1. field1 > cursor_value1
202
+
/// 2. field1 = cursor_value1 AND field2 > cursor_value2
203
+
/// 3. field1 = cursor_value1 AND field2 = cursor_value2 AND field3 > cursor_value3
204
+
/// ... and so on
205
+
/// Finally: all fields equal AND cid > cursor_cid
206
+
///
207
+
/// Returns: #(where_clause_sql, bind_values)
208
+
pub fn build_cursor_where_clause(
209
+
decoded_cursor: DecodedCursor,
210
+
sort_by: Option(List(#(String, String))),
211
+
is_before: Bool,
212
+
) -> #(String, List(String)) {
213
+
let sort_fields = case sort_by {
214
+
None -> []
215
+
Some(fields) -> fields
216
+
}
217
+
218
+
case list.is_empty(sort_fields) {
219
+
True -> #("1=1", [])
220
+
False -> {
221
+
let clauses =
222
+
build_progressive_clauses(
223
+
sort_fields,
224
+
decoded_cursor.field_values,
225
+
decoded_cursor.cid,
226
+
is_before,
227
+
)
228
+
229
+
let sql = "(" <> string.join(clauses.0, " OR ") <> ")"
230
+
#(sql, clauses.1)
231
+
}
232
+
}
233
+
}
234
+
235
+
/// Builds progressive equality clauses for cursor pagination
236
+
fn build_progressive_clauses(
237
+
sort_fields: List(#(String, String)),
238
+
field_values: List(String),
239
+
cid: String,
240
+
is_before: Bool,
241
+
) -> #(List(String), List(String)) {
242
+
let #(clauses, params) =
243
+
list.index_map(sort_fields, fn(field, i) {
244
+
let #(equality_parts, equality_params) = case i {
245
+
0 -> #([], [])
246
+
_ -> {
247
+
list.range(0, i - 1)
248
+
|> list.fold(#([], []), fn(eq_acc, j) {
249
+
let #(eq_parts, eq_params) = eq_acc
250
+
let prior_field =
251
+
list_at(sort_fields, j) |> result.unwrap(#("", ""))
252
+
let value = list_at(field_values, j) |> result.unwrap("")
253
+
254
+
let field_ref = build_cursor_field_reference(prior_field.0)
255
+
let new_part = field_ref <> " = ?"
256
+
let new_params = list.append(eq_params, [value])
257
+
258
+
#(list.append(eq_parts, [new_part]), new_params)
259
+
})
260
+
}
261
+
}
262
+
263
+
let value = list_at(field_values, i) |> result.unwrap("")
264
+
265
+
let comparison_op = get_comparison_operator(field.1, is_before)
266
+
let field_ref = build_cursor_field_reference(field.0)
267
+
268
+
let comparison_part = field_ref <> " " <> comparison_op <> " ?"
269
+
let all_parts = list.append(equality_parts, [comparison_part])
270
+
let all_params = list.append(equality_params, [value])
271
+
272
+
let clause = "(" <> string.join(all_parts, " AND ") <> ")"
273
+
274
+
#(clause, all_params)
275
+
})
276
+
|> list.unzip
277
+
|> fn(unzipped) {
278
+
let flattened_params = list.flatten(unzipped.1)
279
+
#(unzipped.0, flattened_params)
280
+
}
281
+
282
+
let #(final_equality_parts, final_equality_params) =
283
+
list.index_map(sort_fields, fn(field, j) {
284
+
let value = list_at(field_values, j) |> result.unwrap("")
285
+
let field_ref = build_cursor_field_reference(field.0)
286
+
#(field_ref <> " = ?", value)
287
+
})
288
+
|> list.unzip
289
+
290
+
let last_field = list.last(sort_fields) |> result.unwrap(#("", "desc"))
291
+
let cid_comparison_op = get_comparison_operator(last_field.1, is_before)
292
+
293
+
let final_parts =
294
+
list.append(final_equality_parts, ["cid " <> cid_comparison_op <> " ?"])
295
+
let final_params = list.append(final_equality_params, [cid])
296
+
297
+
let final_clause = "(" <> string.join(final_parts, " AND ") <> ")"
298
+
let all_clauses = list.append(clauses, [final_clause])
299
+
let all_params = list.append(params, final_params)
300
+
301
+
#(all_clauses, all_params)
302
+
}
303
+
304
+
/// Builds a field reference for cursor SQL queries (handles JSON fields)
305
+
fn build_cursor_field_reference(field: String) -> String {
306
+
case field {
307
+
"uri" | "cid" | "did" | "collection" | "indexed_at" -> field
308
+
_ -> {
309
+
let json_path = "$." <> string.replace(field, ".", ".")
310
+
"json_extract(json, '" <> json_path <> "')"
311
+
}
312
+
}
313
+
}
314
+
315
+
/// Gets the comparison operator based on sort direction and pagination direction
316
+
fn get_comparison_operator(direction: String, is_before: Bool) -> String {
317
+
let is_desc = string.lowercase(direction) == "desc"
318
+
319
+
case is_before {
320
+
True ->
321
+
case is_desc {
322
+
True -> ">"
323
+
False -> "<"
324
+
}
325
+
False ->
326
+
case is_desc {
327
+
True -> "<"
328
+
False -> ">"
329
+
}
330
+
}
331
+
}
332
+
333
+
/// Helper to get an element at an index from a list
334
+
fn list_at(l: List(a), index: Int) -> Result(a, Nil) {
335
+
l
336
+
|> list.drop(index)
337
+
|> list.first
338
+
}
339
+
340
+
// ===== Sort Direction Helpers =====
341
+
20
342
/// Reverses sort direction for backward pagination
21
343
pub fn reverse_sort_direction(direction: String) -> String {
22
344
case string.lowercase(direction) {
···
36
358
})
37
359
}
38
360
361
+
// ===== ORDER BY Building =====
362
+
39
363
/// Builds an ORDER BY clause from sort fields
40
364
/// use_table_prefix: if True, prefixes table columns with "record." for joins
41
365
pub fn build_order_by(
···
52
376
let field_ref = case field_name {
53
377
"uri" | "cid" | "did" | "collection" | "indexed_at" ->
54
378
table_prefix <> field_name
55
-
// For JSON fields, check if they look like dates and handle accordingly
56
379
"createdAt" | "indexedAt" -> {
57
-
// Use CASE to treat invalid dates as NULL for sorting
58
380
let json_field =
59
381
"json_extract(" <> table_prefix <> "json, '$." <> field_name <> "')"
60
382
"CASE
···
70
392
"asc" -> "ASC"
71
393
_ -> "DESC"
72
394
}
73
-
// Always put NULLs last regardless of sort direction
74
395
field_ref <> " " <> dir <> " NULLS LAST"
75
396
})
76
397
+13
-18
server/src/database/repositories/records.gleam
+13
-18
server/src/database/repositories/records.gleam
···
1
-
import cursor
2
1
import database/queries/pagination
2
+
import database/queries/where_clause
3
3
import database/types.{
4
4
type CollectionStat, type InsertResult, type Record, CollectionStat, Inserted,
5
5
Record, Skipped,
···
12
12
import gleam/result
13
13
import gleam/string
14
14
import sqlight
15
-
import where_clause
16
15
17
16
// ===== Helper Functions =====
18
17
···
550
549
// Add cursor condition if present
551
550
let #(final_where_parts, final_bind_values) = case cursor_opt {
552
551
Some(cursor_str) -> {
553
-
case cursor.decode_cursor(cursor_str, sort_by) {
552
+
case pagination.decode_cursor(cursor_str, sort_by) {
554
553
Ok(decoded_cursor) -> {
555
554
let #(cursor_where, cursor_params) =
556
-
cursor.build_cursor_where_clause(
555
+
pagination.build_cursor_where_clause(
557
556
decoded_cursor,
558
557
sort_by,
559
558
!is_forward,
···
626
625
// Generate next cursor if there are more results
627
626
let next_cursor = case has_more, list.last(final_records) {
628
627
True, Ok(last_record) -> {
629
-
let record_like = pagination.record_to_record_like(last_record)
630
-
Some(cursor.generate_cursor_from_record(record_like, sort_by))
628
+
Some(pagination.generate_cursor_from_record(last_record, sort_by))
631
629
}
632
630
_, _ -> None
633
631
}
···
708
706
// Add cursor condition if present
709
707
let #(final_where_parts, final_bind_values) = case cursor_opt {
710
708
Some(cursor_str) -> {
711
-
case cursor.decode_cursor(cursor_str, sort_by) {
709
+
case pagination.decode_cursor(cursor_str, sort_by) {
712
710
Ok(decoded_cursor) -> {
713
711
let #(cursor_where, cursor_params) =
714
-
cursor.build_cursor_where_clause(
712
+
pagination.build_cursor_where_clause(
715
713
decoded_cursor,
716
714
sort_by,
717
715
!is_forward,
···
784
782
// Generate next cursor if there are more results
785
783
let next_cursor = case has_more, list.last(final_records) {
786
784
True, Ok(last_record) -> {
787
-
let record_like = pagination.record_to_record_like(last_record)
788
-
Some(cursor.generate_cursor_from_record(record_like, sort_by))
785
+
Some(pagination.generate_cursor_from_record(last_record, sort_by))
789
786
}
790
787
_, _ -> None
791
788
}
···
958
955
// Add cursor condition if present
959
956
let #(final_where_parts, final_bind_values) = case cursor_opt {
960
957
Some(cursor_str) -> {
961
-
case cursor.decode_cursor(cursor_str, sort_by) {
958
+
case pagination.decode_cursor(cursor_str, sort_by) {
962
959
Ok(decoded_cursor) -> {
963
960
let #(cursor_where, cursor_params) =
964
-
cursor.build_cursor_where_clause(
961
+
pagination.build_cursor_where_clause(
965
962
decoded_cursor,
966
963
sort_by,
967
964
!is_forward,
···
1037
1034
// Generate next cursor if there are more results
1038
1035
let next_cursor = case has_more, list.last(final_records) {
1039
1036
True, Ok(last_record) -> {
1040
-
let record_like = pagination.record_to_record_like(last_record)
1041
-
Some(cursor.generate_cursor_from_record(record_like, sort_by))
1037
+
Some(pagination.generate_cursor_from_record(last_record, sort_by))
1042
1038
}
1043
1039
_, _ -> None
1044
1040
}
···
1183
1179
// Add cursor condition if present
1184
1180
let #(final_where_parts, final_bind_values) = case cursor_opt {
1185
1181
Some(cursor_str) -> {
1186
-
case cursor.decode_cursor(cursor_str, sort_by) {
1182
+
case pagination.decode_cursor(cursor_str, sort_by) {
1187
1183
Ok(decoded_cursor) -> {
1188
1184
let #(cursor_where, cursor_params) =
1189
-
cursor.build_cursor_where_clause(
1185
+
pagination.build_cursor_where_clause(
1190
1186
decoded_cursor,
1191
1187
sort_by,
1192
1188
!is_forward,
···
1262
1258
// Generate next cursor if there are more results
1263
1259
let next_cursor = case has_more, list.last(final_records) {
1264
1260
True, Ok(last_record) -> {
1265
-
let record_like = pagination.record_to_record_like(last_record)
1266
-
Some(cursor.generate_cursor_from_record(record_like, sort_by))
1261
+
Some(pagination.generate_cursor_from_record(last_record, sort_by))
1267
1262
}
1268
1263
_, _ -> None
1269
1264
}
+5
-12
server/src/graphql_gleam.gleam
+5
-12
server/src/graphql_gleam.gleam
···
3
3
/// This module provides GraphQL schema building and query execution
4
4
import atproto_auth
5
5
import backfill
6
-
import cursor
7
6
import database/queries/aggregates
8
7
import database/queries/pagination
9
8
import database/repositories/actors
···
20
19
import gleam/option
21
20
import gleam/result
22
21
import gleam/string
22
+
import graphql/where_converter
23
23
import lexicon_graphql
24
24
import lexicon_graphql/input/aggregate
25
25
import lexicon_graphql/query/dataloader
···
30
30
import swell/executor
31
31
import swell/schema
32
32
import swell/value
33
-
import where_converter
34
33
35
34
/// Build a GraphQL schema from database lexicons
36
35
///
···
117
116
let graphql_value = record_to_graphql_value(record, db)
118
117
// Generate cursor for this record
119
118
let record_cursor =
120
-
cursor.generate_cursor_from_record(
121
-
pagination.record_to_record_like(record),
119
+
pagination.generate_cursor_from_record(
120
+
record,
122
121
pagination_params.sort_by,
123
122
)
124
123
#(graphql_value, record_cursor)
···
280
279
list.map(record_list, fn(record) {
281
280
let graphql_value = record_to_graphql_value(record, db)
282
281
let cursor =
283
-
cursor.generate_cursor_from_record(
284
-
pagination.record_to_record_like(record),
285
-
db_sort_by,
286
-
)
282
+
pagination.generate_cursor_from_record(record, db_sort_by)
287
283
#(graphql_value, cursor)
288
284
})
289
285
···
325
321
list.map(record_list, fn(record) {
326
322
let graphql_value = record_to_graphql_value(record, db)
327
323
let cursor =
328
-
cursor.generate_cursor_from_record(
329
-
pagination.record_to_record_like(record),
330
-
db_sort_by,
331
-
)
324
+
pagination.generate_cursor_from_record(record, db_sort_by)
332
325
#(graphql_value, cursor)
333
326
})
334
327
server/src/where_clause.gleam
server/src/database/queries/where_clause.gleam
server/src/where_clause.gleam
server/src/database/queries/where_clause.gleam
+1
-1
server/src/where_converter.gleam
server/src/graphql/where_converter.gleam
+1
-1
server/src/where_converter.gleam
server/src/graphql/where_converter.gleam
···
2
2
///
3
3
/// This module bridges the gap between the GraphQL layer (lexicon_graphql/input/where)
4
4
/// and the database layer (where_clause with sqlight types).
5
+
import database/queries/where_clause
5
6
import gleam/dict
6
7
import gleam/list
7
8
import gleam/option.{type Option}
8
9
import lexicon_graphql/input/where
9
10
import sqlight
10
-
import where_clause
11
11
12
12
/// Convert a where.WhereValue to a sqlight.Value
13
13
fn convert_value(value: where.WhereValue) -> sqlight.Value {
+58
-51
server/test/cursor_test.gleam
server/test/pagination_test.gleam
+58
-51
server/test/cursor_test.gleam
server/test/pagination_test.gleam
···
1
-
import cursor
1
+
import database/queries/pagination
2
+
import database/types.{Record}
2
3
import gleam/option.{None, Some}
3
4
import gleeunit/should
4
5
5
6
/// Test encoding a cursor with no sort fields (just CID)
6
7
pub fn encode_cursor_no_sort_test() {
7
8
let record =
8
-
cursor.RecordLike(
9
+
Record(
9
10
uri: "at://did:plc:test/app.bsky.feed.post/123",
10
11
cid: "bafytest123",
11
12
did: "did:plc:test",
···
14
15
indexed_at: "2025-01-15 12:00:00",
15
16
)
16
17
17
-
let result = cursor.generate_cursor_from_record(record, None)
18
+
let result = pagination.generate_cursor_from_record(record, None)
18
19
19
20
// Decode the base64 to verify it's just the CID
20
-
let decoded = cursor.decode_base64(result)
21
+
let decoded = pagination.decode_base64(result)
21
22
should.be_ok(decoded)
22
23
|> should.equal("bafytest123")
23
24
}
···
25
26
/// Test encoding a cursor with single sort field
26
27
pub fn encode_cursor_single_field_test() {
27
28
let record =
28
-
cursor.RecordLike(
29
+
Record(
29
30
uri: "at://did:plc:test/app.bsky.feed.post/123",
30
31
cid: "bafytest123",
31
32
did: "did:plc:test",
···
36
37
37
38
let sort_by = Some([#("indexed_at", "desc")])
38
39
39
-
let result = cursor.generate_cursor_from_record(record, sort_by)
40
+
let result = pagination.generate_cursor_from_record(record, sort_by)
40
41
41
42
// Decode the base64 to verify format
42
-
let decoded = cursor.decode_base64(result)
43
+
let decoded = pagination.decode_base64(result)
43
44
should.be_ok(decoded)
44
45
|> should.equal("2025-01-15 12:00:00|bafytest123")
45
46
}
···
47
48
/// Test encoding a cursor with JSON field
48
49
pub fn encode_cursor_json_field_test() {
49
50
let record =
50
-
cursor.RecordLike(
51
+
Record(
51
52
uri: "at://did:plc:test/app.bsky.feed.post/123",
52
53
cid: "bafytest123",
53
54
did: "did:plc:test",
···
58
59
59
60
let sort_by = Some([#("text", "desc")])
60
61
61
-
let result = cursor.generate_cursor_from_record(record, sort_by)
62
+
let result = pagination.generate_cursor_from_record(record, sort_by)
62
63
63
-
let decoded = cursor.decode_base64(result)
64
+
let decoded = pagination.decode_base64(result)
64
65
should.be_ok(decoded)
65
66
|> should.equal("Hello world|bafytest123")
66
67
}
···
68
69
/// Test encoding a cursor with nested JSON field
69
70
pub fn encode_cursor_nested_json_field_test() {
70
71
let record =
71
-
cursor.RecordLike(
72
+
Record(
72
73
uri: "at://did:plc:test/app.bsky.feed.post/123",
73
74
cid: "bafytest123",
74
75
did: "did:plc:test",
···
79
80
80
81
let sort_by = Some([#("author.name", "asc")])
81
82
82
-
let result = cursor.generate_cursor_from_record(record, sort_by)
83
+
let result = pagination.generate_cursor_from_record(record, sort_by)
83
84
84
-
let decoded = cursor.decode_base64(result)
85
+
let decoded = pagination.decode_base64(result)
85
86
should.be_ok(decoded)
86
87
|> should.equal("Alice|bafytest123")
87
88
}
···
89
90
/// Test encoding a cursor with multiple sort fields
90
91
pub fn encode_cursor_multi_field_test() {
91
92
let record =
92
-
cursor.RecordLike(
93
+
Record(
93
94
uri: "at://did:plc:test/app.bsky.feed.post/123",
94
95
cid: "bafytest123",
95
96
did: "did:plc:test",
···
100
101
101
102
let sort_by = Some([#("text", "desc"), #("createdAt", "desc")])
102
103
103
-
let result = cursor.generate_cursor_from_record(record, sort_by)
104
+
let result = pagination.generate_cursor_from_record(record, sort_by)
104
105
105
-
let decoded = cursor.decode_base64(result)
106
+
let decoded = pagination.decode_base64(result)
106
107
should.be_ok(decoded)
107
108
|> should.equal("Hello|2025-01-15T12:00:00Z|bafytest123")
108
109
}
···
112
113
let sort_by = Some([#("indexed_at", "desc")])
113
114
114
115
// Create a cursor: "2025-01-15 12:00:00|bafytest123"
115
-
let cursor_str = cursor.encode_base64("2025-01-15 12:00:00|bafytest123")
116
+
let cursor_str = pagination.encode_base64("2025-01-15 12:00:00|bafytest123")
116
117
117
-
let result = cursor.decode_cursor(cursor_str, sort_by)
118
+
let result = pagination.decode_cursor(cursor_str, sort_by)
118
119
119
120
should.be_ok(result)
120
121
|> fn(decoded) {
···
131
132
let sort_by = Some([#("text", "desc"), #("createdAt", "desc")])
132
133
133
134
let cursor_str =
134
-
cursor.encode_base64("Hello|2025-01-15T12:00:00Z|bafytest123")
135
+
pagination.encode_base64("Hello|2025-01-15T12:00:00Z|bafytest123")
135
136
136
-
let result = cursor.decode_cursor(cursor_str, sort_by)
137
+
let result = pagination.decode_cursor(cursor_str, sort_by)
137
138
138
139
should.be_ok(result)
139
140
|> fn(decoded) {
···
151
152
152
153
// Cursor has 2 fields but sort_by only has 1
153
154
let cursor_str =
154
-
cursor.encode_base64("Hello|2025-01-15T12:00:00Z|bafytest123")
155
+
pagination.encode_base64("Hello|2025-01-15T12:00:00Z|bafytest123")
155
156
156
-
let result = cursor.decode_cursor(cursor_str, sort_by)
157
+
let result = pagination.decode_cursor(cursor_str, sort_by)
157
158
158
159
should.be_error(result)
159
160
}
···
162
163
pub fn decode_cursor_invalid_base64_test() {
163
164
let sort_by = Some([#("text", "desc")])
164
165
165
-
let result = cursor.decode_cursor("not-valid-base64!!!", sort_by)
166
+
let result = pagination.decode_cursor("not-valid-base64!!!", sort_by)
166
167
167
168
should.be_error(result)
168
169
}
···
170
171
/// Test extracting table column values
171
172
pub fn extract_field_value_table_column_test() {
172
173
let record =
173
-
cursor.RecordLike(
174
+
Record(
174
175
uri: "at://did:plc:test/app.bsky.feed.post/123",
175
176
cid: "bafytest123",
176
177
did: "did:plc:test",
···
179
180
indexed_at: "2025-01-15 12:00:00",
180
181
)
181
182
182
-
cursor.extract_field_value(record, "uri")
183
+
pagination.extract_field_value(record, "uri")
183
184
|> should.equal("at://did:plc:test/app.bsky.feed.post/123")
184
185
185
-
cursor.extract_field_value(record, "cid")
186
+
pagination.extract_field_value(record, "cid")
186
187
|> should.equal("bafytest123")
187
188
188
-
cursor.extract_field_value(record, "did")
189
+
pagination.extract_field_value(record, "did")
189
190
|> should.equal("did:plc:test")
190
191
191
-
cursor.extract_field_value(record, "collection")
192
+
pagination.extract_field_value(record, "collection")
192
193
|> should.equal("app.bsky.feed.post")
193
194
194
-
cursor.extract_field_value(record, "indexed_at")
195
+
pagination.extract_field_value(record, "indexed_at")
195
196
|> should.equal("2025-01-15 12:00:00")
196
197
}
197
198
198
199
/// Test extracting JSON field values
199
200
pub fn extract_field_value_json_test() {
200
201
let record =
201
-
cursor.RecordLike(
202
+
Record(
202
203
uri: "at://did:plc:test/app.bsky.feed.post/123",
203
204
cid: "bafytest123",
204
205
did: "did:plc:test",
···
207
208
indexed_at: "2025-01-15 12:00:00",
208
209
)
209
210
210
-
cursor.extract_field_value(record, "text")
211
+
pagination.extract_field_value(record, "text")
211
212
|> should.equal("Hello world")
212
213
213
-
cursor.extract_field_value(record, "createdAt")
214
+
pagination.extract_field_value(record, "createdAt")
214
215
|> should.equal("2025-01-15T12:00:00Z")
215
216
216
-
cursor.extract_field_value(record, "likeCount")
217
+
pagination.extract_field_value(record, "likeCount")
217
218
|> should.equal("42")
218
219
}
219
220
220
221
/// Test extracting nested JSON field values
221
222
pub fn extract_field_value_nested_json_test() {
222
223
let record =
223
-
cursor.RecordLike(
224
+
Record(
224
225
uri: "at://did:plc:test/app.bsky.feed.post/123",
225
226
cid: "bafytest123",
226
227
did: "did:plc:test",
···
229
230
indexed_at: "2025-01-15 12:00:00",
230
231
)
231
232
232
-
cursor.extract_field_value(record, "author.name")
233
+
pagination.extract_field_value(record, "author.name")
233
234
|> should.equal("Alice")
234
235
235
-
cursor.extract_field_value(record, "author.did")
236
+
pagination.extract_field_value(record, "author.did")
236
237
|> should.equal("did:plc:alice")
237
238
}
238
239
239
240
/// Test extracting missing JSON field returns NULL
240
241
pub fn extract_field_value_missing_test() {
241
242
let record =
242
-
cursor.RecordLike(
243
+
Record(
243
244
uri: "at://did:plc:test/app.bsky.feed.post/123",
244
245
cid: "bafytest123",
245
246
did: "did:plc:test",
···
248
249
indexed_at: "2025-01-15 12:00:00",
249
250
)
250
251
251
-
cursor.extract_field_value(record, "nonexistent")
252
+
pagination.extract_field_value(record, "nonexistent")
252
253
|> should.equal("NULL")
253
254
254
-
cursor.extract_field_value(record, "author.name")
255
+
pagination.extract_field_value(record, "author.name")
255
256
|> should.equal("NULL")
256
257
}
257
258
···
260
261
/// Test building WHERE clause for single field DESC
261
262
pub fn build_where_single_field_desc_test() {
262
263
let decoded =
263
-
cursor.DecodedCursor(
264
+
pagination.DecodedCursor(
264
265
field_values: ["2025-01-15 12:00:00"],
265
266
cid: "bafytest123",
266
267
)
267
268
268
269
let sort_by = Some([#("indexed_at", "desc")])
269
270
270
-
let #(sql, params) = cursor.build_cursor_where_clause(decoded, sort_by, False)
271
+
let #(sql, params) =
272
+
pagination.build_cursor_where_clause(decoded, sort_by, False)
271
273
272
274
// For DESC: indexed_at < cursor_value OR (indexed_at = cursor_value AND cid < cursor_cid)
273
275
sql
···
284
286
/// Test building WHERE clause for single field ASC
285
287
pub fn build_where_single_field_asc_test() {
286
288
let decoded =
287
-
cursor.DecodedCursor(
289
+
pagination.DecodedCursor(
288
290
field_values: ["2025-01-15 12:00:00"],
289
291
cid: "bafytest123",
290
292
)
291
293
292
294
let sort_by = Some([#("indexed_at", "asc")])
293
295
294
-
let #(sql, params) = cursor.build_cursor_where_clause(decoded, sort_by, False)
296
+
let #(sql, params) =
297
+
pagination.build_cursor_where_clause(decoded, sort_by, False)
295
298
296
299
// For ASC: indexed_at > cursor_value OR (indexed_at = cursor_value AND cid > cursor_cid)
297
300
sql
···
308
311
/// Test building WHERE clause for JSON field
309
312
pub fn build_where_json_field_test() {
310
313
let decoded =
311
-
cursor.DecodedCursor(field_values: ["Hello world"], cid: "bafytest123")
314
+
pagination.DecodedCursor(field_values: ["Hello world"], cid: "bafytest123")
312
315
313
316
let sort_by = Some([#("text", "desc")])
314
317
315
-
let #(sql, params) = cursor.build_cursor_where_clause(decoded, sort_by, False)
318
+
let #(sql, params) =
319
+
pagination.build_cursor_where_clause(decoded, sort_by, False)
316
320
317
321
// JSON fields use json_extract
318
322
sql
···
327
331
/// Test building WHERE clause for nested JSON field
328
332
pub fn build_where_nested_json_field_test() {
329
333
let decoded =
330
-
cursor.DecodedCursor(field_values: ["Alice"], cid: "bafytest123")
334
+
pagination.DecodedCursor(field_values: ["Alice"], cid: "bafytest123")
331
335
332
336
let sort_by = Some([#("author.name", "asc")])
333
337
334
-
let #(sql, params) = cursor.build_cursor_where_clause(decoded, sort_by, False)
338
+
let #(sql, params) =
339
+
pagination.build_cursor_where_clause(decoded, sort_by, False)
335
340
336
341
// Nested JSON fields use $.path.to.field
337
342
sql
···
346
351
/// Test building WHERE clause for multiple fields
347
352
pub fn build_where_multi_field_test() {
348
353
let decoded =
349
-
cursor.DecodedCursor(
354
+
pagination.DecodedCursor(
350
355
field_values: ["Hello", "2025-01-15T12:00:00Z"],
351
356
cid: "bafytest123",
352
357
)
353
358
354
359
let sort_by = Some([#("text", "desc"), #("createdAt", "desc")])
355
360
356
-
let #(sql, params) = cursor.build_cursor_where_clause(decoded, sort_by, False)
361
+
let #(sql, params) =
362
+
pagination.build_cursor_where_clause(decoded, sort_by, False)
357
363
358
364
// Multi-field: progressive equality checks
359
365
// (text < ?) OR (text = ? AND createdAt < ?) OR (text = ? AND createdAt = ? AND cid < ?)
···
376
382
/// Test building WHERE clause for backward pagination (before)
377
383
pub fn build_where_backward_test() {
378
384
let decoded =
379
-
cursor.DecodedCursor(
385
+
pagination.DecodedCursor(
380
386
field_values: ["2025-01-15 12:00:00"],
381
387
cid: "bafytest123",
382
388
)
···
384
390
let sort_by = Some([#("indexed_at", "desc")])
385
391
386
392
// is_before = True reverses the comparison operators
387
-
let #(sql, params) = cursor.build_cursor_where_clause(decoded, sort_by, True)
393
+
let #(sql, params) =
394
+
pagination.build_cursor_where_clause(decoded, sort_by, True)
388
395
389
396
// For before with DESC: indexed_at > cursor_value OR (indexed_at = cursor_value AND cid > cursor_cid)
390
397
sql
+1
-1
server/test/database_aggregation_test.gleam
+1
-1
server/test/database_aggregation_test.gleam
···
1
1
import database/queries/aggregates
2
+
import database/queries/where_clause
2
3
import database/schema/tables
3
4
import database/types
4
5
import gleam/dict
···
8
9
import gleeunit
9
10
import gleeunit/should
10
11
import sqlight
11
-
import where_clause
12
12
13
13
pub fn main() {
14
14
gleeunit.main()
+1
-1
server/test/graphql_where_integration_test.gleam
+1
-1
server/test/graphql_where_integration_test.gleam
+1
-1
server/test/where_clause_test.gleam
+1
-1
server/test/where_clause_test.gleam
+1
-1
server/test/where_edge_cases_test.gleam
+1
-1
server/test/where_edge_cases_test.gleam
···
1
1
/// Edge case and error handling tests for where clause functionality
2
2
///
3
3
/// Tests various edge cases, error conditions, and potential SQL injection attempts
4
+
import database/queries/where_clause
4
5
import gleam/dict
5
6
import gleam/list
6
7
import gleam/option.{None, Some}
···
10
11
import lexicon_graphql/input/where as where_input
11
12
import sqlight
12
13
import swell/value
13
-
import where_clause
14
14
15
15
pub fn main() {
16
16
gleeunit.main()
+1
-1
server/test/where_integration_test.gleam
+1
-1
server/test/where_integration_test.gleam
+1
-1
server/test/where_sql_builder_test.gleam
+1
-1
server/test/where_sql_builder_test.gleam