this repo has no description
lewis.moe
1# PDS Implementation TODOs
2
3Lewis' corrected big boy todofile
4
5## Server Infrastructure & Proxying
6- [x] Health Check
7 - [x] Implement `GET /health` endpoint (returns "OK").
8 - [x] Implement `GET /xrpc/_health` endpoint (returns "OK").
9- [x] Server Description
10 - [x] Implement `com.atproto.server.describeServer` (returns available user domains).
11- [x] XRPC Proxying
12 - [x] Implement strict forwarding for all `app.bsky.*` and `chat.bsky.*` requests to an appview.
13 - [x] Forward auth headers correctly.
14 - [x] Handle appview errors/timeouts gracefully.
15
16## Authentication & Account Management (`com.atproto.server`)
17- [x] Account Creation
18 - [x] Implement `com.atproto.server.createAccount`.
19 - [x] Validate handle format (reject invalid characters).
20 - [x] Create DID for new user (PLC directory).
21 - [x] Initialize user repository (Root commit).
22 - [x] Return access JWT and DID.
23 - [x] Create DID for new user (did:web).
24- [x] Session Management
25 - [x] Implement `com.atproto.server.createSession` (Login).
26 - [x] Implement `com.atproto.server.getSession`.
27 - [x] Implement `com.atproto.server.refreshSession`.
28 - [x] Implement `com.atproto.server.deleteSession` (Logout).
29 - [x] Implement `com.atproto.server.activateAccount`.
30 - [x] Implement `com.atproto.server.checkAccountStatus`.
31 - [x] Implement `com.atproto.server.createAppPassword`.
32 - [x] Implement `com.atproto.server.createInviteCode`.
33 - [x] Implement `com.atproto.server.createInviteCodes`.
34 - [x] Implement `com.atproto.server.deactivateAccount`.
35 - [x] Implement `com.atproto.server.deleteAccount` (user-initiated, requires password + email token).
36 - [x] Implement `com.atproto.server.getAccountInviteCodes`.
37 - [x] Implement `com.atproto.server.getServiceAuth` (Cross-service auth).
38 - [x] Implement `com.atproto.server.listAppPasswords`.
39 - [x] Implement `com.atproto.server.requestAccountDelete`.
40 - [x] Implement `com.atproto.server.requestEmailConfirmation` / `requestEmailUpdate`.
41 - [x] Implement `com.atproto.server.requestPasswordReset` / `resetPassword`.
42 - [x] Implement `com.atproto.server.reserveSigningKey`.
43 - [x] Implement `com.atproto.server.revokeAppPassword`.
44 - [x] Implement `com.atproto.server.updateEmail`.
45 - [x] Implement `com.atproto.server.confirmEmail`.
46
47## Repository Operations (`com.atproto.repo`)
48- [x] Record CRUD
49 - [x] Implement `com.atproto.repo.createRecord`.
50 - [x] Generate `rkey` (TID) if not provided.
51 - [x] Handle MST (Merkle Search Tree) insertion.
52 - [x] **Trigger Firehose Event**.
53 - [x] Implement `com.atproto.repo.putRecord`.
54 - [x] Implement `com.atproto.repo.getRecord`.
55 - [x] Implement `com.atproto.repo.deleteRecord`.
56 - [x] Implement `com.atproto.repo.listRecords`.
57 - [x] Implement `com.atproto.repo.describeRepo`.
58 - [x] Implement `com.atproto.repo.applyWrites` (Batch writes).
59 - [x] Implement `com.atproto.repo.importRepo` (Migration).
60 - [x] Implement `com.atproto.repo.listMissingBlobs`.
61- [x] Blob Management
62 - [x] Implement `com.atproto.repo.uploadBlob`.
63 - [x] Store blob (S3).
64 - [x] return `blob` ref (CID + MimeType).
65
66## Sync & Federation (`com.atproto.sync`)
67- [x] The Firehose (WebSocket)
68 - [x] Implement `com.atproto.sync.subscribeRepos`.
69 - [x] Broadcast real-time commit events.
70 - [x] Handle cursor replay (backfill).
71- [x] Bulk Export
72 - [x] Implement `com.atproto.sync.getRepo` (Return full CAR file of repo).
73 - [x] Implement `com.atproto.sync.getBlocks` (Return specific blocks via CIDs).
74 - [x] Implement `com.atproto.sync.getLatestCommit`.
75 - [x] Implement `com.atproto.sync.getRecord` (Sync version, distinct from repo.getRecord).
76 - [x] Implement `com.atproto.sync.getRepoStatus`.
77 - [x] Implement `com.atproto.sync.listRepos`.
78 - [x] Implement `com.atproto.sync.notifyOfUpdate`.
79- [x] Blob Sync
80 - [x] Implement `com.atproto.sync.getBlob`.
81 - [x] Implement `com.atproto.sync.listBlobs`.
82- [x] Crawler Interaction
83 - [x] Implement `com.atproto.sync.requestCrawl` (Notify relays to index us).
84- [x] Deprecated Sync Endpoints (for compatibility)
85 - [x] Implement `com.atproto.sync.getCheckout` (deprecated).
86 - [x] Implement `com.atproto.sync.getHead` (deprecated).
87
88## Identity (`com.atproto.identity`)
89- [x] Resolution
90 - [x] Implement `com.atproto.identity.resolveHandle` (Can be internal or proxy to PLC).
91 - [x] Implement `com.atproto.identity.updateHandle`.
92 - [x] Implement `com.atproto.identity.submitPlcOperation` / `signPlcOperation` / `requestPlcOperationSignature`.
93 - [x] Implement `com.atproto.identity.getRecommendedDidCredentials`.
94 - [x] Implement `/.well-known/did.json` (Depends on supporting did:web).
95
96## Admin Management (`com.atproto.admin`)
97- [x] Implement `com.atproto.admin.deleteAccount`.
98- [x] Implement `com.atproto.admin.disableAccountInvites`.
99- [x] Implement `com.atproto.admin.disableInviteCodes`.
100- [x] Implement `com.atproto.admin.enableAccountInvites`.
101- [x] Implement `com.atproto.admin.getAccountInfo` / `getAccountInfos`.
102- [x] Implement `com.atproto.admin.getInviteCodes`.
103- [x] Implement `com.atproto.admin.getSubjectStatus`.
104- [x] Implement `com.atproto.admin.sendEmail`.
105- [x] Implement `com.atproto.admin.updateAccountEmail`.
106- [x] Implement `com.atproto.admin.updateAccountHandle`.
107- [x] Implement `com.atproto.admin.updateAccountPassword`.
108- [x] Implement `com.atproto.admin.updateSubjectStatus`.
109
110## Moderation (`com.atproto.moderation`)
111- [x] Implement `com.atproto.moderation.createReport`.
112
113## Temp Namespace (`com.atproto.temp`)
114- [x] Implement `com.atproto.temp.checkSignupQueue` (signup queue status for gated signups).
115
116## Misc HTTP Endpoints
117- [x] Implement `/robots.txt` endpoint.
118
119## OAuth 2.1 Support
120Full OAuth 2.1 provider for ATProto native app authentication.
121- [x] OAuth Provider Core
122 - [x] Implement `/.well-known/oauth-protected-resource` metadata endpoint.
123 - [x] Implement `/.well-known/oauth-authorization-server` metadata endpoint.
124 - [x] Implement `/oauth/authorize` authorization endpoint (with login UI).
125 - [x] Implement `/oauth/par` Pushed Authorization Request endpoint.
126 - [x] Implement `/oauth/token` token endpoint (authorization_code + refresh_token grants).
127 - [x] Implement `/oauth/jwks` JSON Web Key Set endpoint.
128 - [x] Implement `/oauth/revoke` token revocation endpoint.
129 - [x] Implement `/oauth/introspect` token introspection endpoint.
130- [x] OAuth Database Tables
131 - [x] Device table for tracking authorized devices.
132 - [x] Authorization request table.
133 - [x] Authorized client table.
134 - [x] Token table for OAuth tokens.
135 - [x] Used refresh token table (replay protection).
136 - [x] DPoP JTI tracking table.
137- [x] DPoP (Demonstrating Proof-of-Possession) support.
138- [x] Client metadata fetching and validation.
139- [x] PKCE (S256) enforcement.
140- [x] OAuth token verification extractor for protected resources.
141- [x] Authorization UI templates (HTML login form).
142- [x] Implement `private_key_jwt` signature verification with async JWKS fetching.
143- [x] HS256 JWT support (matches reference PDS).
144
145## OAuth Security Notes
146
147Security measures implemented:
148
149- Constant-time comparison for signature verification (prevents timing attacks)
150- HMAC-SHA256 for access token signing with configurable secret
151- Production secrets require 32+ character minimum
152- DPoP JTI replay protection via database
153- DPoP nonce validation with HMAC-based timestamps (5 min validity)
154- Refresh token rotation with reuse detection (revokes token family on reuse)
155- PKCE S256 enforced (plain not allowed)
156- Authorization code single-use enforcement
157- URL encoding for redirect parameters (prevents injection)
158- All database queries use parameterized statements (no SQL injection)
159- Deactivated/taken-down accounts blocked from OAuth authorization
160- Client ID validation on token exchange (defense-in-depth against cross-client attacks)
161- HTML escaping in OAuth templates (XSS prevention)
162
163### Auth Notes
164- Dual algorithm support: ES256K (secp256k1 ECDSA) with per-user keys AND HS256 (HMAC) for compatibility with reference PDS.
165- Token storage: Storing only token JTIs in session_tokens table (defense in depth against DB breaches). Refresh token family tracking enables detection of token reuse attacks.
166- Key encryption: User signing keys encrypted at rest using AES-256-GCM with keys derived via HKDF from KEY_ENCRYPTION_KEY environment variable.
167
168## PDS-Level App Endpoints
169These endpoints need to be implemented at the PDS level (not just proxied to appview).
170
171### Actor (`app.bsky.actor`)
172- [x] Implement `app.bsky.actor.getPreferences` (user preferences storage).
173- [x] Implement `app.bsky.actor.putPreferences` (update user preferences).
174- [x] Implement `app.bsky.actor.getProfile` (PDS-level with proxy fallback).
175- [x] Implement `app.bsky.actor.getProfiles` (PDS-level with proxy fallback).
176
177### Feed (`app.bsky.feed`)
178These are implemented at PDS level to enable local-first reads (read-after-write pattern):
179- [x] Implement `app.bsky.feed.getTimeline` (PDS-level with proxy + RAW).
180- [x] Implement `app.bsky.feed.getAuthorFeed` (PDS-level with proxy + RAW).
181- [x] Implement `app.bsky.feed.getActorLikes` (PDS-level with proxy + RAW).
182- [x] Implement `app.bsky.feed.getPostThread` (PDS-level with proxy + RAW + NotFound handling).
183- [x] Implement `app.bsky.feed.getFeed` (proxy to feed generator).
184
185### Notification (`app.bsky.notification`)
186- [x] Implement `app.bsky.notification.registerPush` (push notification registration, proxied).
187
188## Infrastructure & Core Components
189- [x] Sequencer (Event Log)
190 - [x] Implement a `Sequencer` (backed by `repo_seq` table).
191 - [x] Implement event formatting (`commit`, `handle`, `identity`, `account`).
192 - [x] Implement database polling / event emission mechanism.
193 - [x] Implement cursor-based event replay (`requestSeqRange`).
194- [x] Repo Storage & Consistency (in postgres)
195 - [x] Implement `RepoStorage` for postgres (replaces per-user SQLite).
196 - [x] Read/Write IPLD blocks to `blocks` table (global deduplication).
197 - [x] Manage Repo Root in `repos` table.
198 - [x] Implement Atomic Repo Transactions.
199 - [x] Ensure `blocks` write, `repo_root` update, `records` index update, and `sequencer` event are committed in a single transaction.
200 - [x] Implement concurrency control (row-level locking via FOR UPDATE).
201- [ ] DID Cache
202 - [ ] Implement caching layer for DID resolution (Redis or in-memory).
203 - [ ] Handle cache invalidation/expiry.
204- [x] Crawlers Service
205 - [x] Implement `Crawlers` service (debounce notifications to relays).
206 - [x] 20-minute notification debounce.
207 - [x] Circuit breaker for relay failures.
208- [x] Notification Service
209 - [x] Queue-based notification system with database table
210 - [x] Background worker polling for pending notifications
211 - [x] Extensible sender trait for multiple channels
212 - [x] Email sender via OS sendmail/msmtp
213 - [x] Discord webhook sender
214 - [x] Telegram bot sender
215 - [x] Signal CLI sender
216 - [x] Helper functions for common notification types (welcome, password reset, email verification, etc.)
217 - [x] Respect user's `preferred_notification_channel` setting for non-email-specific notifications
218- [x] Image Processing
219 - [x] Implement image resize/formatting pipeline (for blob uploads).
220 - [x] WebP conversion for thumbnails.
221 - [x] EXIF stripping.
222 - [x] File size limits (10MB default).
223- [x] IPLD & MST
224 - [x] Implement Merkle Search Tree logic for repo signing.
225 - [x] Implement CAR (Content Addressable Archive) encoding/decoding.
226 - [x] Cycle detection in CAR export.
227- [x] Rate Limiting
228 - [x] Per-IP rate limiting on login (10/min).
229 - [x] Per-IP rate limiting on OAuth token endpoint (30/min).
230 - [x] Per-IP rate limiting on password reset (5/hour).
231 - [x] Per-IP rate limiting on account creation (10/hour).
232- [x] Circuit Breakers
233 - [x] PLC directory circuit breaker (5 failures → open, 60s timeout).
234 - [x] Relay notification circuit breaker (10 failures → open, 30s timeout).
235- [x] Security Hardening
236 - [x] Email header injection prevention (CRLF sanitization).
237 - [x] Signal command injection prevention (phone number validation).
238 - [x] Constant-time signature comparison.
239 - [x] SSRF protection for outbound requests.
240
241## Lewis' fabulous mini-list of remaining TODOs
242- [ ] DID resolution caching (valkey).
243- [ ] Record schema validation (generic validation framework).
244- [ ] Fix any remaining TODOs in the code.
245
246## Future: Web Management UI
247A single-page web app for account management. The frontend (JS framework) calls existing ATProto XRPC endpoints - no server-side rendering or bespoke HTML form handlers.
248
249### Architecture
250- [ ] Static SPA served from PDS (or separate static host)
251- [ ] Frontend authenticates via OAuth 2.1 flow (same as any ATProto client)
252- [ ] All operations use standard XRPC endpoints (existing + new PDS-specific ones below)
253- [ ] No server-side sessions or CSRF - pure API client
254
255### PDS-Specific XRPC Endpoints (new)
256Absolutely subject to change, "bspds" isn't even the real name of this pds thus far :D
257Anyway... endpoints for PDS settings not covered by standard ATProto:
258- [ ] `com.bspds.account.getNotificationPrefs` - get preferred channel, verified channels
259- [ ] `com.bspds.account.updateNotificationPrefs` - set preferred channel
260- [ ] `com.bspds.account.getNotificationHistory` - list past notifications
261- [ ] `com.bspds.account.verifyChannel` - initiate verification for Discord/Telegram/Signal
262- [ ] `com.bspds.account.confirmChannelVerification` - confirm with code
263- [ ] `com.bspds.admin.getServerStats` - user count, storage usage, etc.
264
265### Frontend Views
266Uses existing ATProto endpoints where possible:
267
268User Dashboard
269- [ ] Account overview (uses `com.atproto.server.getSession`, `com.atproto.admin.getAccountInfo`)
270- [ ] Active sessions view (needs new endpoint or extend existing)
271- [ ] App passwords (uses `com.atproto.server.listAppPasswords`, `createAppPassword`, `revokeAppPassword`)
272- [ ] Invite codes (uses `com.atproto.server.getAccountInviteCodes`, `createInviteCode`)
273
274Notification Preferences
275- [ ] Channel selector (uses `com.bspds.account.*` endpoints above)
276- [ ] Verification flows for Discord/Telegram/Signal
277- [ ] Notification history view
278
279Account Settings
280- [ ] Email change (uses `com.atproto.server.requestEmailUpdate`, `updateEmail`)
281- [ ] Password change (uses `com.atproto.server.requestPasswordReset`, `resetPassword`)
282- [ ] Handle change (uses `com.atproto.identity.updateHandle`)
283- [ ] Account deletion (uses `com.atproto.server.requestAccountDelete`, `deleteAccount`)
284- [ ] Data export (uses `com.atproto.sync.getRepo`)
285
286Admin Dashboard (privileged users only)
287- [ ] User list (uses `com.atproto.admin.getAccountInfos` with pagination)
288- [ ] User detail/actions (uses `com.atproto.admin.*` endpoints)
289- [ ] Invite management (uses `com.atproto.admin.getInviteCodes`, `disableInviteCodes`)
290- [ ] Server stats (uses `com.bspds.admin.getServerStats`)
291