WIP! A BB-style forum, on the ATmosphere! We're still working... we'll be back soon when we have something to show off!
node typescript hono htmx atproto

docs: add SQLite support design document

Designs dual-dialect database support (PostgreSQL + SQLite) with:
- URL-prefix detection in createDb factory
- role_permissions join table replacing permissions text[] array
- Two-phase Postgres migration with data migration script
- NixOS module and devenv changes for optional Postgres service
- Operator upgrade instructions with safety warnings

+383
+383
docs/plans/2026-02-24-sqlite-support-design.md
··· 1 + # SQLite Support Design 2 + 3 + **Date:** 2026-02-24 4 + **Status:** Approved, pending implementation plan 5 + 6 + ## Motivation 7 + 8 + Adding SQLite as a fully supported database backend alongside PostgreSQL serves two audiences: 9 + 10 + - **Self-hosters** who want to run atBB on a single server without operating a separate PostgreSQL service 11 + - **Developers and agents** who want to spin up, test, and discard a local database without any external infrastructure 12 + 13 + The implementation is transparent to all application code above the `createDb` factory — route handlers, middleware, and services remain unaware of which database they are talking to. 14 + 15 + ## Scope 16 + 17 + SQLite is a **full production deployment target**, not only a development convenience. Both PostgreSQL and SQLite are equally supported. Operators choose their backend at deploy time via `DATABASE_URL`. 18 + 19 + ## Architecture Overview 20 + 21 + The URL prefix in `DATABASE_URL` is the single decision point. Everything below `createDb` is dialect-specific; everything above it is identical for both backends. 22 + 23 + ``` 24 + DATABASE_URL=postgres://... → postgres.js driver → Drizzle (PostgreSQL dialect) 25 + DATABASE_URL=file:./data/atbb.db → @libsql/client → Drizzle (LibSQL dialect) 26 + DATABASE_URL=file::memory: → @libsql/client → Drizzle (LibSQL in-memory, tests) 27 + DATABASE_URL=libsql://... → @libsql/client → Drizzle (Turso cloud, no code changes) 28 + ``` 29 + 30 + ### File layout changes 31 + 32 + ``` 33 + packages/db/ 34 + src/ 35 + schema.ts ← PostgreSQL schema (updated: permissions column removed) 36 + schema.sqlite.ts ← SQLite schema (new: identical structure, dialect-specific column types) 37 + index.ts ← createDb() with URL-prefix detection (updated) 38 + 39 + apps/appview/ 40 + drizzle/ ← PostgreSQL migrations (existing + two new migrations) 41 + drizzle-sqlite/ ← SQLite migrations (new, single clean initial migration) 42 + drizzle.postgres.config.ts ← renamed from drizzle.config.ts 43 + drizzle.sqlite.config.ts ← new 44 + scripts/ 45 + migrate-permissions.ts ← one-time data migration script (new) 46 + ``` 47 + 48 + ## Schema Changes 49 + 50 + ### New `role_permissions` join table 51 + 52 + The `permissions text[]` array column in `roles` is replaced by a dedicated `role_permissions` table. This is a data-model improvement independent of SQLite support: it adds referential integrity (cascade delete when a role is removed) and makes "find all roles with permission X" queries natural. 53 + 54 + **Both** schema files (`schema.ts` and `schema.sqlite.ts`) define this table. The structure is identical; only the Drizzle column helper syntax differs by dialect. 55 + 56 + ```typescript 57 + // Postgres (schema.ts) 58 + export const rolePermissions = pgTable("role_permissions", { 59 + roleId: bigint("role_id", { mode: "bigint" }) 60 + .notNull() 61 + .references(() => roles.id, { onDelete: "cascade" }), 62 + permission: text("permission").notNull(), 63 + }, (t) => [ 64 + primaryKey({ columns: [t.roleId, t.permission] }), 65 + ]); 66 + 67 + // SQLite (schema.sqlite.ts) 68 + export const rolePermissions = sqliteTable("role_permissions", { 69 + roleId: integer("role_id", { mode: "bigint" }) 70 + .notNull() 71 + .references(() => roles.id, { onDelete: "cascade" }), 72 + permission: text("permission").notNull(), 73 + }, (t) => [ 74 + primaryKey({ columns: [t.roleId, t.permission] }), 75 + ]); 76 + ``` 77 + 78 + The `permissions` column is removed from `roles` in both schema files. 79 + 80 + ### Dialect differences between the two schema files 81 + 82 + | Concern | PostgreSQL (`schema.ts`) | SQLite (`schema.sqlite.ts`) | 83 + |---|---|---| 84 + | Import | `drizzle-orm/pg-core` | `drizzle-orm/sqlite-core` | 85 + | Auto-increment IDs | `bigserial` | `integer({ mode: "bigint" }).primaryKey({ autoIncrement: true })` | 86 + | Timestamps | `timestamp({ withTimezone: true, mode: "date" })` | `integer({ mode: "timestamp" })` | 87 + | Permissions | _(removed)_ | _(removed)_ | 88 + 89 + Both dialects use `mode: "bigint"` for integer IDs and `mode: "date"` / `mode: "timestamp"` for dates, so **TypeScript types are identical** (`bigint` for IDs, `Date` for timestamps). The `Database` union type works without narrowing in route handlers. 90 + 91 + ### Impact on `checkPermission` 92 + 93 + The current `role.permissions.includes(permission)` pattern is replaced by a targeted existence query — more efficient since it avoids fetching all permissions to check one: 94 + 95 + ```typescript 96 + const [match] = await ctx.db 97 + .select() 98 + .from(rolePermissions) 99 + .where(and( 100 + eq(rolePermissions.roleId, role.id), 101 + or( 102 + eq(rolePermissions.permission, permission), 103 + eq(rolePermissions.permission, "*") 104 + ) 105 + )) 106 + .limit(1); 107 + 108 + return !!match; // fail-closed: undefined → false 109 + ``` 110 + 111 + All other permission-related queries (listing a role's permissions, seeding default roles) are updated to use join or relational queries against `role_permissions`. 112 + 113 + ## `createDb` Factory 114 + 115 + `packages/db/src/index.ts` becomes: 116 + 117 + ```typescript 118 + import { drizzle as drizzlePg } from "drizzle-orm/postgres-js"; 119 + import { drizzle as drizzleSqlite } from "drizzle-orm/libsql"; 120 + import { createClient } from "@libsql/client"; 121 + import postgres from "postgres"; 122 + import * as pgSchema from "./schema.js"; 123 + import * as sqliteSchema from "./schema.sqlite.js"; 124 + 125 + export function createDb(databaseUrl: string) { 126 + if (databaseUrl.startsWith("postgres")) { 127 + return drizzlePg(postgres(databaseUrl), { schema: pgSchema }); 128 + } 129 + return drizzleSqlite( 130 + createClient({ url: databaseUrl }), 131 + { schema: sqliteSchema } 132 + ); 133 + } 134 + 135 + export type Database = ReturnType<typeof createDb>; 136 + export type Transaction = Parameters<Parameters<Database["transaction"]>[0]>[0]; 137 + export type DbOrTransaction = Database | Transaction; 138 + ``` 139 + 140 + `AppContext` is unchanged — `db: Database` continues to work for both backends. 141 + 142 + ## Drizzle Configuration 143 + 144 + The existing `apps/appview/drizzle.config.ts` is renamed and a sibling added: 145 + 146 + ```typescript 147 + // drizzle.postgres.config.ts (renamed from drizzle.config.ts) 148 + export default defineConfig({ 149 + schema: "../../packages/db/src/schema.ts", 150 + out: "./drizzle", 151 + dialect: "postgresql", 152 + dbCredentials: { url: process.env.DATABASE_URL! }, 153 + }); 154 + 155 + // drizzle.sqlite.config.ts (new) 156 + export default defineConfig({ 157 + schema: "../../packages/db/src/schema.sqlite.ts", 158 + out: "./drizzle-sqlite", 159 + dialect: "sqlite", 160 + dbCredentials: { url: process.env.DATABASE_URL! }, 161 + }); 162 + ``` 163 + 164 + `apps/appview/package.json` gains dialect-scoped scripts: 165 + 166 + ```json 167 + "db:generate": "drizzle-kit generate --config=drizzle.postgres.config.ts", 168 + "db:migrate": "drizzle-kit migrate --config=drizzle.postgres.config.ts", 169 + "db:generate:sqlite": "drizzle-kit generate --config=drizzle.sqlite.config.ts", 170 + "db:migrate:sqlite": "drizzle-kit migrate --config=drizzle.sqlite.config.ts" 171 + ``` 172 + 173 + ## Migration Strategy 174 + 175 + ### PostgreSQL (existing deployments) 176 + 177 + The `permissions text[]` column cannot be dropped atomically with data preservation. The migration is split into three ordered steps: 178 + 179 + **Step 1 — Schema migration: add `role_permissions` table** 180 + ``` 181 + apps/appview/drizzle/0011_add_role_permissions.sql 182 + CREATE TABLE role_permissions (role_id bigint, permission text, PRIMARY KEY (role_id, permission)) 183 + -- permissions column is NOT dropped here 184 + ``` 185 + 186 + **Step 2 — Data migration script (must run between steps 1 and 3)** 187 + ``` 188 + apps/appview/scripts/migrate-permissions.ts 189 + SELECT id, permissions FROM roles WHERE permissions != '{}' 190 + → INSERT INTO role_permissions (role_id, permission) VALUES (...) 191 + ON CONFLICT DO NOTHING ← idempotent, safe to re-run 192 + → Prints: "Migrated N permissions across M roles" 193 + ``` 194 + 195 + **Step 3 — Schema migration: drop the now-redundant column** 196 + ``` 197 + apps/appview/drizzle/0012_drop_permissions_column.sql 198 + ALTER TABLE roles DROP COLUMN permissions 199 + ``` 200 + 201 + ### SQLite (fresh deployments only) 202 + 203 + SQLite deployments always start clean. The `drizzle-sqlite/` folder contains a single initial migration that includes `role_permissions` from the start and has no `permissions` column. No data migration script is needed for SQLite. 204 + 205 + ## Docker Deployment 206 + 207 + The existing `docker-compose.yml` (PostgreSQL) is unchanged. A new compose file enables SQLite deployments without a database service: 208 + 209 + ```yaml 210 + # docker-compose.sqlite.yml 211 + services: 212 + appview: 213 + build: . 214 + environment: 215 + DATABASE_URL: file:/data/atbb.db 216 + volumes: 217 + - atbb_data:/data 218 + ports: 219 + - "80:80" 220 + 221 + volumes: 222 + atbb_data: 223 + ``` 224 + 225 + The Dockerfile needs no changes — the runtime binary is identical for both backends. 226 + 227 + ## NixOS Module Changes (`nix/module.nix`) 228 + 229 + A new `database.type` option is added alongside the existing `database.enable`: 230 + 231 + ```nix 232 + database = { 233 + enable = mkOption { 234 + type = types.bool; 235 + default = cfg.database.type == "postgresql"; 236 + description = "Enable local PostgreSQL service (ignored for SQLite)."; 237 + }; 238 + type = mkOption { 239 + type = types.enum [ "postgresql" "sqlite" ]; 240 + default = "postgresql"; 241 + description = "Database backend. Use 'sqlite' for embedded single-file storage."; 242 + }; 243 + name = mkOption { ... }; # unchanged, PostgreSQL only 244 + path = mkOption { 245 + type = types.path; 246 + default = "/var/lib/atbb/atbb.db"; 247 + description = "Path to SQLite database file. Only used when database.type = 'sqlite'."; 248 + }; 249 + }; 250 + ``` 251 + 252 + The `atbb-appview` service environment becomes conditional: 253 + 254 + ```nix 255 + environment = { 256 + DATABASE_URL = if cfg.database.type == "sqlite" 257 + then "file:${cfg.database.path}" 258 + else "postgres:///atbb?host=/run/postgresql"; 259 + # ...other env vars unchanged 260 + }; 261 + serviceConfig = { 262 + StateDirectory = mkIf (cfg.database.type == "sqlite") "atbb"; # creates /var/lib/atbb/ 263 + # ... 264 + }; 265 + ``` 266 + 267 + The `atbb-migrate` oneshot service runs the appropriate migration script: 268 + 269 + ```nix 270 + script = if cfg.database.type == "sqlite" 271 + then "${atbb}/bin/atbb db:migrate:sqlite" 272 + else "${atbb}/bin/atbb db:migrate"; 273 + ``` 274 + 275 + The PostgreSQL system service is conditionally enabled: 276 + 277 + ```nix 278 + services.postgresql = mkIf (cfg.database.type == "postgresql" && cfg.database.enable) { 279 + # existing config unchanged 280 + }; 281 + ``` 282 + 283 + Existing module users default to `database.type = "postgresql"`, so no existing NixOS configurations require changes. 284 + 285 + ## Nix Package Changes (`nix/package.nix`) 286 + 287 + The `drizzle-sqlite/` migrations directory must be included in the package alongside the existing `drizzle/` directory: 288 + 289 + ```nix 290 + # In installPhase, alongside existing drizzle copy: 291 + cp -r apps/appview/drizzle $out/apps/appview/drizzle 292 + cp -r apps/appview/drizzle-sqlite $out/apps/appview/drizzle-sqlite # new 293 + ``` 294 + 295 + ## devenv Changes (`devenv.nix`) 296 + 297 + The PostgreSQL service is made optional via a devenv input flag, allowing developers using SQLite to skip the database service entirely: 298 + 299 + ```nix 300 + { config, lib, pkgs, ... }: { 301 + # Postgres service — disable if using SQLite locally 302 + services.postgres = { 303 + enable = lib.mkDefault true; 304 + # ...existing config unchanged 305 + }; 306 + } 307 + ``` 308 + 309 + Developers opting into SQLite locally set in their `.env`: 310 + ``` 311 + DATABASE_URL=file:./data/atbb.db 312 + ``` 313 + and can disable the devenv Postgres service by overriding `services.postgres.enable = false` in a local `devenv.local.nix` (devenv supports this pattern via `devenv.local.nix` overrides). 314 + 315 + ## Tests 316 + 317 + The test context detects the URL and selects the appropriate setup: 318 + 319 + ```typescript 320 + // test-context.ts 321 + if (config.databaseUrl.startsWith("postgres")) { 322 + // existing postgres.js path — unchanged 323 + } else { 324 + // SQLite: run migrations programmatically (no external process needed) 325 + const { migrate } = await import("drizzle-orm/libsql/migrator"); 326 + await migrate(db, { migrationsFolder: resolvedDrizzleSqlitePath }); 327 + // No manual cleanup needed — in-memory DB is discarded with the process 328 + } 329 + ``` 330 + 331 + Developers without a running Postgres instance run the full test suite with: 332 + 333 + ```sh 334 + DATABASE_URL=file::memory: pnpm test 335 + ``` 336 + 337 + CI continues to use PostgreSQL via `DATABASE_URL=postgres://...`. 338 + 339 + --- 340 + 341 + ## Operator Migration Instructions 342 + 343 + ### Fresh SQLite installs (new deployments) 344 + 345 + No special steps required. Run the standard migration command: 346 + 347 + ```sh 348 + DATABASE_URL=file:./data/atbb.db pnpm --filter @atbb/appview db:migrate:sqlite 349 + ``` 350 + 351 + Or if using the NixOS module with `database.type = "sqlite"`, the `atbb-migrate` service handles this automatically on startup. 352 + 353 + ### Existing PostgreSQL deployments upgrading to `role_permissions` 354 + 355 + > **Warning:** Steps must be run in order. Running step 3 before step 2 will permanently destroy all role permission data. 356 + 357 + ```sh 358 + # Step 1: Add role_permissions table (does NOT drop permissions column yet) 359 + pnpm --filter @atbb/appview db:migrate 360 + 361 + # Step 2: Copy existing permission data into the new table 362 + # MUST run before step 3. Safe to re-run if interrupted. 363 + pnpm --filter @atbb/appview migrate-permissions 364 + # Verify output: "Migrated N permissions across M roles" 365 + # Do not proceed to step 3 if N is unexpectedly 0 and you had role permissions. 366 + 367 + # Step 3: Drop the now-redundant permissions column 368 + pnpm --filter @atbb/appview db:migrate 369 + ``` 370 + 371 + **Using the NixOS module:** If `autoMigrate` is enabled, steps 1 and 3 run automatically on service restart. You must run step 2 manually between the two restarts: 372 + 373 + ```sh 374 + # 1. Deploy the new build (runs step 1 automatically on atbb-migrate) 375 + # 2. SSH into the server and run the data migration: 376 + sudo -u atbb pnpm --filter @atbb/appview migrate-permissions 377 + # 3. Restart the service (runs step 3 automatically on atbb-migrate) 378 + sudo systemctl restart atbb-appview 379 + ``` 380 + 381 + ### Switching an existing PostgreSQL deployment to SQLite 382 + 383 + This requires a full data export and import — there is no automated migration path between backends. Export your data first using the atBB backup tooling (to be documented separately), then set up a fresh SQLite deployment and import. This is an unusual operation and only makes sense for very small deployments.