this repo has no description
lewis.moe
1# PDS Implementation TODOs
2
3Lewis' corrected big boy todofile
4
5## Server Infrastructure & Proxying
6- [x] Health Check
7 - [x] Implement `GET /health` endpoint (returns "OK").
8 - [x] Implement `GET /xrpc/_health` endpoint (returns "OK").
9- [x] Server Description
10 - [x] Implement `com.atproto.server.describeServer` (returns available user domains).
11- [x] XRPC Proxying
12 - [x] Implement strict forwarding for all `app.bsky.*` and `chat.bsky.*` requests to an appview.
13 - [x] Forward auth headers correctly.
14 - [x] Handle appview errors/timeouts gracefully.
15
16## Authentication & Account Management (`com.atproto.server`)
17- [x] Account Creation
18 - [x] Implement `com.atproto.server.createAccount`.
19 - [x] Validate handle format (reject invalid characters).
20 - [x] Create DID for new user (PLC directory).
21 - [x] Initialize user repository (Root commit).
22 - [x] Return access JWT and DID.
23 - [x] Create DID for new user (did:web).
24- [x] Session Management
25 - [x] Implement `com.atproto.server.createSession` (Login).
26 - [x] Implement `com.atproto.server.getSession`.
27 - [x] Implement `com.atproto.server.refreshSession`.
28 - [x] Implement `com.atproto.server.deleteSession` (Logout).
29 - [x] Implement `com.atproto.server.activateAccount`.
30 - [x] Implement `com.atproto.server.checkAccountStatus`.
31 - [x] Implement `com.atproto.server.createAppPassword`.
32 - [x] Implement `com.atproto.server.createInviteCode`.
33 - [x] Implement `com.atproto.server.createInviteCodes`.
34 - [x] Implement `com.atproto.server.deactivateAccount`.
35 - [x] Implement `com.atproto.server.deleteAccount` (user-initiated, requires password + email token).
36 - [x] Implement `com.atproto.server.getAccountInviteCodes`.
37 - [x] Implement `com.atproto.server.getServiceAuth` (Cross-service auth).
38 - [x] Implement `com.atproto.server.listAppPasswords`.
39 - [x] Implement `com.atproto.server.requestAccountDelete`.
40 - [x] Implement `com.atproto.server.requestEmailConfirmation` / `requestEmailUpdate`.
41 - [x] Implement `com.atproto.server.requestPasswordReset` / `resetPassword`.
42 - [x] Implement `com.atproto.server.reserveSigningKey`.
43 - [x] Implement `com.atproto.server.revokeAppPassword`.
44 - [x] Implement `com.atproto.server.updateEmail`.
45 - [x] Implement `com.atproto.server.confirmEmail`.
46
47## Repository Operations (`com.atproto.repo`)
48- [x] Record CRUD
49 - [x] Implement `com.atproto.repo.createRecord`.
50 - [x] Generate `rkey` (TID) if not provided.
51 - [x] Handle MST (Merkle Search Tree) insertion.
52 - [x] **Trigger Firehose Event**.
53 - [x] Implement `com.atproto.repo.putRecord`.
54 - [x] Implement `com.atproto.repo.getRecord`.
55 - [x] Implement `com.atproto.repo.deleteRecord`.
56 - [x] Implement `com.atproto.repo.listRecords`.
57 - [x] Implement `com.atproto.repo.describeRepo`.
58 - [x] Implement `com.atproto.repo.applyWrites` (Batch writes).
59 - [x] Implement `com.atproto.repo.importRepo` (Migration).
60 - [x] Implement `com.atproto.repo.listMissingBlobs`.
61- [x] Blob Management
62 - [x] Implement `com.atproto.repo.uploadBlob`.
63 - [x] Store blob (S3).
64 - [x] return `blob` ref (CID + MimeType).
65
66## Sync & Federation (`com.atproto.sync`)
67- [x] The Firehose (WebSocket)
68 - [x] Implement `com.atproto.sync.subscribeRepos`.
69 - [x] Broadcast real-time commit events.
70 - [x] Handle cursor replay (backfill).
71- [x] Bulk Export
72 - [x] Implement `com.atproto.sync.getRepo` (Return full CAR file of repo).
73 - [x] Implement `com.atproto.sync.getBlocks` (Return specific blocks via CIDs).
74 - [x] Implement `com.atproto.sync.getLatestCommit`.
75 - [x] Implement `com.atproto.sync.getRecord` (Sync version, distinct from repo.getRecord).
76 - [x] Implement `com.atproto.sync.getRepoStatus`.
77 - [x] Implement `com.atproto.sync.listRepos`.
78 - [x] Implement `com.atproto.sync.notifyOfUpdate`.
79- [x] Blob Sync
80 - [x] Implement `com.atproto.sync.getBlob`.
81 - [x] Implement `com.atproto.sync.listBlobs`.
82- [x] Crawler Interaction
83 - [x] Implement `com.atproto.sync.requestCrawl` (Notify relays to index us).
84
85## Identity (`com.atproto.identity`)
86- [x] Resolution
87 - [x] Implement `com.atproto.identity.resolveHandle` (Can be internal or proxy to PLC).
88 - [x] Implement `com.atproto.identity.updateHandle`.
89 - [x] Implement `com.atproto.identity.submitPlcOperation` / `signPlcOperation` / `requestPlcOperationSignature`.
90 - [x] Implement `com.atproto.identity.getRecommendedDidCredentials`.
91 - [x] Implement `/.well-known/did.json` (Depends on supporting did:web).
92
93## Admin Management (`com.atproto.admin`)
94- [x] Implement `com.atproto.admin.deleteAccount`.
95- [x] Implement `com.atproto.admin.disableAccountInvites`.
96- [x] Implement `com.atproto.admin.disableInviteCodes`.
97- [x] Implement `com.atproto.admin.enableAccountInvites`.
98- [x] Implement `com.atproto.admin.getAccountInfo` / `getAccountInfos`.
99- [x] Implement `com.atproto.admin.getInviteCodes`.
100- [x] Implement `com.atproto.admin.getSubjectStatus`.
101- [x] Implement `com.atproto.admin.sendEmail`.
102- [x] Implement `com.atproto.admin.updateAccountEmail`.
103- [x] Implement `com.atproto.admin.updateAccountHandle`.
104- [x] Implement `com.atproto.admin.updateAccountPassword`.
105- [x] Implement `com.atproto.admin.updateSubjectStatus`.
106
107## Moderation (`com.atproto.moderation`)
108- [x] Implement `com.atproto.moderation.createReport`.
109
110## Temp Namespace (`com.atproto.temp`)
111- [ ] Implement `com.atproto.temp.checkSignupQueue` (signup queue status for gated signups).
112
113## OAuth 2.1 Support
114Full OAuth 2.1 provider for ATProto native app authentication.
115- [x] OAuth Provider Core
116 - [x] Implement `/.well-known/oauth-protected-resource` metadata endpoint.
117 - [x] Implement `/.well-known/oauth-authorization-server` metadata endpoint.
118 - [x] Implement `/oauth/authorize` authorization endpoint (headless JSON mode).
119 - [x] Implement `/oauth/par` Pushed Authorization Request endpoint.
120 - [x] Implement `/oauth/token` token endpoint (authorization_code + refresh_token grants).
121 - [x] Implement `/oauth/jwks` JSON Web Key Set endpoint.
122 - [x] Implement `/oauth/revoke` token revocation endpoint.
123 - [x] Implement `/oauth/introspect` token introspection endpoint.
124- [x] OAuth Database Tables
125 - [x] Device table for tracking authorized devices.
126 - [x] Authorization request table.
127 - [x] Authorized client table.
128 - [x] Token table for OAuth tokens.
129 - [x] Used refresh token table (replay protection).
130 - [x] DPoP JTI tracking table.
131- [x] DPoP (Demonstrating Proof-of-Possession) support.
132- [x] Client metadata fetching and validation.
133- [x] PKCE (S256) enforcement.
134- [x] OAuth token verification extractor for protected resources.
135- [ ] Authorization UI templates (currently headless-only, returns JSON for programmatic flows).
136- [ ] Implement `private_key_jwt` signature verification (currently rejects with clear error).
137
138## OAuth Security Notes
139
140I've tried to ensure that this codebase is not vulnerable to the following:
141
142- Constant-time comparison for signature verification (prevents timing attacks)
143- HMAC-SHA256 for access token signing with configurable secret
144- Production secrets require 32+ character minimum
145- DPoP JTI replay protection via database
146- DPoP nonce validation with HMAC-based timestamps (5 min validity)
147- Refresh token rotation with reuse detection (revokes token family on reuse)
148- PKCE S256 enforced (plain not allowed)
149- Authorization code single-use enforcement
150- URL encoding for redirect parameters (prevents injection)
151- All database queries use parameterized statements (no SQL injection)
152- Deactivated/taken-down accounts blocked from OAuth authorization
153- Client ID validation on token exchange (defense-in-depth against cross-client attacks)
154
155### Auth Notes
156- Algorithm choice: Using ES256K (secp256k1 ECDSA) with per-user keys. Ref PDS uses HS256 (HMAC) with single server key. Our approach provides better key isolation but differs from reference implementation.
157 - [ ] Support the ref PDS HS256 system too.
158- Token storage: Now storing only token JTIs in session_tokens table (defense in depth against DB breaches). Refresh token family tracking enables detection of token reuse attacks.
159- Key encryption: User signing keys encrypted at rest using AES-256-GCM with keys derived via HKDF from MASTER_KEY environment variable. Migration-safe: supports both encrypted (version 1) and plaintext (version 0) keys.
160
161## PDS-Level App Endpoints
162These endpoints need to be implemented at the PDS level (not just proxied to appview).
163
164### Actor (`app.bsky.actor`)
165- [x] Implement `app.bsky.actor.getPreferences` (user preferences storage).
166- [x] Implement `app.bsky.actor.putPreferences` (update user preferences).
167- [x] Implement `app.bsky.actor.getProfile` (PDS-level with proxy fallback).
168- [x] Implement `app.bsky.actor.getProfiles` (PDS-level with proxy fallback).
169
170### Feed (`app.bsky.feed`)
171These are implemented at PDS level to enable local-first reads:
172- [ ] Implement `app.bsky.feed.getTimeline` (PDS-level with proxy).
173- [ ] Implement `app.bsky.feed.getAuthorFeed` (PDS-level with proxy).
174- [ ] Implement `app.bsky.feed.getActorLikes` (PDS-level with proxy).
175- [ ] Implement `app.bsky.feed.getPostThread` (PDS-level with proxy).
176- [ ] Implement `app.bsky.feed.getFeed` (PDS-level with proxy).
177
178### Notification (`app.bsky.notification`)
179- [ ] Implement `app.bsky.notification.registerPush` (push notification registration).
180
181## Deprecated Sync Endpoints (for compatibility)
182- [ ] Implement `com.atproto.sync.getCheckout` (deprecated, still needed for compatibility).
183- [ ] Implement `com.atproto.sync.getHead` (deprecated, still needed for compatibility).
184
185## Misc HTTP Endpoints
186- [ ] Implement `/robots.txt` endpoint.
187
188## Record Schema Validation
189- [ ] Handle this generically.
190
191## Preference Storage
192User preferences (for app.bsky.actor.getPreferences/putPreferences):
193- [x] Create preferences table for storing user app preferences.
194- [x] Implement `app.bsky.actor.getPreferences` handler (read from postgres, proxy fallback).
195- [x] Implement `app.bsky.actor.putPreferences` handler (write to postgres).
196
197## Infrastructure & Core Components
198- [x] Sequencer (Event Log)
199 - [x] Implement a `Sequencer` (backed by `repo_seq` table).
200 - [x] Implement event formatting (`commit`, `handle`, `identity`, `account`).
201 - [x] Implement database polling / event emission mechanism.
202 - [x] Implement cursor-based event replay (`requestSeqRange`).
203- [x] Repo Storage & Consistency (in postgres)
204 - [x] Implement `RepoStorage` for postgres (replaces per-user SQLite).
205 - [x] Read/Write IPLD blocks to `blocks` table (global deduplication).
206 - [x] Manage Repo Root in `repos` table.
207 - [x] Implement Atomic Repo Transactions.
208 - [x] Ensure `blocks` write, `repo_root` update, `records` index update, and `sequencer` event are committed in a single transaction.
209 - [ ] Implement concurrency control (row-level locking on `repos` table) to prevent concurrent writes to the same repo.
210- [ ] DID Cache
211 - [ ] Implement caching layer for DID resolution (Redis or in-memory).
212 - [ ] Handle cache invalidation/expiry.
213- [ ] Background Jobs
214 - [ ] Implement `Crawlers` service (debounce notifications to relays).
215- [x] Notification Service
216 - [x] Queue-based notification system with database table
217 - [x] Background worker polling for pending notifications
218 - [x] Extensible sender trait for multiple channels
219 - [x] Email sender via OS sendmail/msmtp
220 - [ ] Discord bot sender
221 - [ ] Telegram bot sender
222 - [ ] Signal bot sender
223 - [x] Helper functions for common notification types (welcome, password reset, email verification, etc.)
224 - [x] Respect user's `preferred_notification_channel` setting for non-email-specific notifications
225- [ ] Image Processing
226 - [ ] Implement image resize/formatting pipeline (for blob uploads).
227- [x] IPLD & MST
228 - [x] Implement Merkle Search Tree logic for repo signing.
229 - [x] Implement CAR (Content Addressable Archive) encoding/decoding.
230- [ ] Validation
231 - [ ] DID PLC Operations (Sign rotation keys).
232- [ ] Fix any remaining TODOs in the code, everywhere, full stop.
233
234## Web Management UI
235A single-page web app for account management. The frontend (JS framework) calls existing ATProto XRPC endpoints - no server-side rendering or bespoke HTML form handlers.
236
237### Architecture
238- [ ] Static SPA served from PDS (or separate static host)
239- [ ] Frontend authenticates via OAuth 2.1 flow (same as any ATProto client)
240- [ ] All operations use standard XRPC endpoints (existing + new PDS-specific ones below)
241- [ ] No server-side sessions or CSRF - pure API client
242
243### PDS-Specific XRPC Endpoints (new)
244Absolutely subject to change, "bspds" isn't even the real name of this pds thus far :D
245Anyway... endpoints for PDS settings not covered by standard ATProto:
246- [ ] `com.bspds.account.getNotificationPrefs` - get preferred channel, verified channels
247- [ ] `com.bspds.account.updateNotificationPrefs` - set preferred channel
248- [ ] `com.bspds.account.getNotificationHistory` - list past notifications
249- [ ] `com.bspds.account.verifyChannel` - initiate verification for Discord/Telegram/Signal
250- [ ] `com.bspds.account.confirmChannelVerification` - confirm with code
251- [ ] `com.bspds.admin.getServerStats` - user count, storage usage, etc.
252
253### Frontend Views
254Uses existing ATProto endpoints where possible:
255
256User Dashboard
257- [ ] Account overview (uses `com.atproto.server.getSession`, `com.atproto.admin.getAccountInfo`)
258- [ ] Active sessions view (needs new endpoint or extend existing)
259- [ ] App passwords (uses `com.atproto.server.listAppPasswords`, `createAppPassword`, `revokeAppPassword`)
260- [ ] Invite codes (uses `com.atproto.server.getAccountInviteCodes`, `createInviteCode`)
261
262Notification Preferences
263- [ ] Channel selector (uses `com.bspds.account.*` endpoints above)
264- [ ] Verification flows for Discord/Telegram/Signal
265- [ ] Notification history view
266
267Account Settings
268- [ ] Email change (uses `com.atproto.server.requestEmailUpdate`, `updateEmail`)
269- [ ] Password change (uses `com.atproto.server.requestPasswordReset`, `resetPassword`)
270- [ ] Handle change (uses `com.atproto.identity.updateHandle`)
271- [ ] Account deletion (uses `com.atproto.server.requestAccountDelete`, `deleteAccount`)
272- [ ] Data export (uses `com.atproto.sync.getRepo`)
273
274Admin Dashboard (privileged users only)
275- [ ] User list (uses `com.atproto.admin.getAccountInfos` with pagination)
276- [ ] User detail/actions (uses `com.atproto.admin.*` endpoints)
277- [ ] Invite management (uses `com.atproto.admin.getInviteCodes`, `disableInviteCodes`)
278- [ ] Server stats (uses `com.bspds.admin.getServerStats`)
279