zzstoatzz.io
prefect server in zig
add database migrations planning document
- docs/migrations.md: comprehensive analysis of migration options
- zmig (sqlite-only zig tool)
- atlas (external go-based tool, multi-dialect)
- minimal DIY approach
- hybrid recommendation (atlas for dev, embedded SQL for runtime)
- phased implementation plan
- references to python prefect alembic patterns
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
+1
docs/README.md
+1
docs/README.md
···
6
6
7
7
- [web server](./web-server.md) - http layer, route dispatch
8
8
- [database](./database.md) - sqlite + postgres persistence
9
+
- [migrations](./migrations.md) - schema versioning strategy (planning)
9
10
- [services](./services.md) - background workers
10
11
11
12
broker and messaging details in `src/broker/CLAUDE.md` and `src/services/CLAUDE.md`.
+8
docs/database.md
+8
docs/database.md
···
122
122
123
123
the `Transaction` type ensures all statements execute on the same connection, which is critical for postgres where the pool might otherwise give different connections for each query.
124
124
125
+
## migrations
126
+
127
+
**current state**: no migration system. schema is applied via `CREATE TABLE IF NOT EXISTS` on startup.
128
+
129
+
see [migrations.md](./migrations.md) for planning on proper migration support.
130
+
131
+
see [python-reference/](./python-reference/) for how prefect python handles migrations with alembic.
132
+
125
133
## testing
126
134
127
135
```bash
+165
docs/migrations.md
+165
docs/migrations.md
···
1
+
# database migrations
2
+
3
+
## current state: no migration system
4
+
5
+
we currently have no migration system. schema changes are:
6
+
- embedded in `src/db/schema/sqlite.zig` and `src/db/schema/postgres.zig`
7
+
- applied via `CREATE TABLE IF NOT EXISTS` on startup
8
+
- manually applied to existing databases (`ALTER TABLE ...`)
9
+
10
+
this works for development but **will not work for production**.
11
+
12
+
## requirements
13
+
14
+
1. **version tracking** - know which migrations have been applied
15
+
2. **dual dialect support** - sqlite and postgres, possibly with different SQL
16
+
3. **forward migrations** - apply schema changes
17
+
4. **rollback capability** - undo changes (nice to have)
18
+
5. **embedded runtime** - apply migrations on server startup
19
+
6. **CLI tooling** - create/manage migrations during development
20
+
21
+
## options evaluated
22
+
23
+
### option 1: zmig (sqlite-only)
24
+
25
+
[zmig](https://github.com/Jeansidharta/zmig) - native zig migration tool
26
+
27
+
**pros:**
28
+
- pure zig, embeds migrations in binary
29
+
- timestamp-based ordering
30
+
- up/down migrations
31
+
- runtime API: `applyMigrations(db, alloc, options)`
32
+
- requires zig 0.15.2 (we use this)
33
+
34
+
**cons:**
35
+
- sqlite only - no postgres support
36
+
- would need to fork/extend for postgres
37
+
38
+
**migration format:**
39
+
```
40
+
migrations/
41
+
├── 1758503588032-add_empirical_policy.up.sql
42
+
└── 1758503588032-add_empirical_policy.down.sql
43
+
```
44
+
45
+
### option 2: atlas (external tool)
46
+
47
+
[atlas](https://atlasgo.io/) - schema-as-code migration tool (go-based)
48
+
49
+
**pros:**
50
+
- supports sqlite AND postgres
51
+
- declarative schema definitions
52
+
- automatic migration generation
53
+
- k8s operator available
54
+
- language-agnostic CLI
55
+
56
+
**cons:**
57
+
- external dependency (go binary)
58
+
- not embedded in our binary
59
+
- adds operational complexity
60
+
- schema defined separately from zig code
61
+
62
+
### option 3: minimal DIY
63
+
64
+
build minimal migration system ourselves:
65
+
66
+
**pros:**
67
+
- tailored to our exact needs
68
+
- no external dependencies
69
+
- full control over dialect differences
70
+
- embeds in binary
71
+
72
+
**cons:**
73
+
- development effort
74
+
- must handle edge cases ourselves
75
+
76
+
### option 4: hybrid approach (recommended)
77
+
78
+
**development**: use atlas CLI for migration creation/testing
79
+
**runtime**: embed migration SQL files and apply with simple zig code
80
+
81
+
```
82
+
migrations/
83
+
├── versions.txt # ordered list of migration IDs
84
+
├── 001_initial/
85
+
│ ├── sqlite.up.sql
86
+
│ ├── sqlite.down.sql
87
+
│ ├── postgres.up.sql
88
+
│ └── postgres.down.sql
89
+
└── 002_add_empirical_policy/
90
+
├── sqlite.up.sql
91
+
└── postgres.up.sql
92
+
```
93
+
94
+
runtime logic (minimal zig code):
95
+
```zig
96
+
pub fn applyMigrations(db: *Backend) !void {
97
+
// 1. create migrations table if not exists
98
+
// 2. read applied migrations from table
99
+
// 3. read embedded migrations from @embedFile
100
+
// 4. apply any pending migrations in order
101
+
// 5. record applied migrations
102
+
}
103
+
```
104
+
105
+
## migration table schema
106
+
107
+
```sql
108
+
CREATE TABLE IF NOT EXISTS _migrations (
109
+
id TEXT PRIMARY KEY, -- "001_initial"
110
+
applied_at TEXT NOT NULL, -- ISO timestamp
111
+
checksum TEXT -- SHA256 of migration SQL (optional)
112
+
);
113
+
```
114
+
115
+
## implementation phases
116
+
117
+
### phase 1: tracking (minimal)
118
+
119
+
1. add `_migrations` table to schema
120
+
2. on startup, record current "version" (e.g., "001_initial")
121
+
3. log warning if database version != code version
122
+
123
+
### phase 2: embedded migrations
124
+
125
+
1. create `migrations/` directory with SQL files
126
+
2. embed migrations in binary via `@embedFile`
127
+
3. apply pending migrations on startup
128
+
4. support `--migrate-only` CLI flag
129
+
130
+
### phase 3: development tooling
131
+
132
+
1. add `zig build migrate:new` or similar
133
+
2. generate timestamped migration directories
134
+
3. optionally integrate atlas for schema diffing
135
+
136
+
### phase 4: rollbacks (if needed)
137
+
138
+
1. add down.sql files
139
+
2. add `--rollback` CLI flag
140
+
3. track rollback state
141
+
142
+
## alembic patterns to preserve
143
+
144
+
from python prefect (see `docs/python-reference/migrations.md`):
145
+
146
+
1. **separate dialect chains** - different SQL per database
147
+
2. **migration notes** - document changes with both dialect IDs
148
+
3. **batch mode for sqlite** - table recreation for ALTER
149
+
4. **transaction per migration** - isolation
150
+
5. **migration lock** - prevent concurrent execution
151
+
152
+
## questions to resolve
153
+
154
+
1. **when to migrate?** startup vs explicit command vs both?
155
+
2. **failure handling?** abort startup? partial state?
156
+
3. **backwards compatibility?** can old code read new schema?
157
+
4. **testing?** how to test migrations in CI?
158
+
159
+
## references
160
+
161
+
- [zmig](https://github.com/Jeansidharta/zmig) - zig sqlite migrations
162
+
- [zigmigrate](https://github.com/eugenepentland/zigmigrate) - zig sqlite migrations (goose-inspired)
163
+
- [atlas](https://atlasgo.io/) - schema-as-code tool
164
+
- [flyway](https://www.bytebase.com/blog/flyway-vs-liquibase/) - java migration tool
165
+
- python prefect: `~/github.com/prefecthq/prefect/src/prefect/server/database/_migrations/`