this repo has no description
lewis.moe
1# PDS Implementation TODOs
2
3Lewis' corrected big boy todofile
4
5## Server Infrastructure & Proxying
6- [x] Health Check
7 - [x] Implement `GET /health` endpoint (returns "OK").
8 - [x] Implement `GET /xrpc/_health` endpoint (returns "OK").
9- [x] Server Description
10 - [x] Implement `com.atproto.server.describeServer` (returns available user domains).
11- [x] XRPC Proxying
12 - [x] Implement strict forwarding for all `app.bsky.*` and `chat.bsky.*` requests to an appview.
13 - [x] Forward auth headers correctly.
14 - [x] Handle appview errors/timeouts gracefully.
15
16## Authentication & Account Management (`com.atproto.server`)
17- [x] Account Creation
18 - [x] Implement `com.atproto.server.createAccount`.
19 - [x] Validate handle format (reject invalid characters).
20 - [x] Create DID for new user (PLC directory).
21 - [x] Initialize user repository (Root commit).
22 - [x] Return access JWT and DID.
23 - [x] Create DID for new user (did:web).
24- [x] Session Management
25 - [x] Implement `com.atproto.server.createSession` (Login).
26 - [x] Implement `com.atproto.server.getSession`.
27 - [x] Implement `com.atproto.server.refreshSession`.
28 - [x] Implement `com.atproto.server.deleteSession` (Logout).
29 - [x] Implement `com.atproto.server.activateAccount`.
30 - [x] Implement `com.atproto.server.checkAccountStatus`.
31 - [x] Implement `com.atproto.server.createAppPassword`.
32 - [x] Implement `com.atproto.server.createInviteCode`.
33 - [x] Implement `com.atproto.server.createInviteCodes`.
34 - [x] Implement `com.atproto.server.deactivateAccount`.
35 - [x] Implement `com.atproto.server.deleteAccount` (user-initiated, requires password + email token).
36 - [x] Implement `com.atproto.server.getAccountInviteCodes`.
37 - [x] Implement `com.atproto.server.getServiceAuth` (Cross-service auth).
38 - [x] Implement `com.atproto.server.listAppPasswords`.
39 - [x] Implement `com.atproto.server.requestAccountDelete`.
40 - [x] Implement `com.atproto.server.requestEmailConfirmation` / `requestEmailUpdate`.
41 - [x] Implement `com.atproto.server.requestPasswordReset` / `resetPassword`.
42 - [x] Implement `com.atproto.server.reserveSigningKey`.
43 - [x] Implement `com.atproto.server.revokeAppPassword`.
44 - [x] Implement `com.atproto.server.updateEmail`.
45 - [x] Implement `com.atproto.server.confirmEmail`.
46
47## Repository Operations (`com.atproto.repo`)
48- [x] Record CRUD
49 - [x] Implement `com.atproto.repo.createRecord`.
50 - [x] Generate `rkey` (TID) if not provided.
51 - [x] Handle MST (Merkle Search Tree) insertion.
52 - [x] **Trigger Firehose Event**.
53 - [x] Implement `com.atproto.repo.putRecord`.
54 - [x] Implement `com.atproto.repo.getRecord`.
55 - [x] Implement `com.atproto.repo.deleteRecord`.
56 - [x] Implement `com.atproto.repo.listRecords`.
57 - [x] Implement `com.atproto.repo.describeRepo`.
58 - [x] Implement `com.atproto.repo.applyWrites` (Batch writes).
59 - [x] Implement `com.atproto.repo.importRepo` (Migration).
60 - [x] Implement `com.atproto.repo.listMissingBlobs`.
61- [x] Blob Management
62 - [x] Implement `com.atproto.repo.uploadBlob`.
63 - [x] Store blob (S3).
64 - [x] return `blob` ref (CID + MimeType).
65
66## Sync & Federation (`com.atproto.sync`)
67- [x] The Firehose (WebSocket)
68 - [x] Implement `com.atproto.sync.subscribeRepos`.
69 - [x] Broadcast real-time commit events.
70 - [x] Handle cursor replay (backfill).
71- [x] Bulk Export
72 - [x] Implement `com.atproto.sync.getRepo` (Return full CAR file of repo).
73 - [x] Implement `com.atproto.sync.getBlocks` (Return specific blocks via CIDs).
74 - [x] Implement `com.atproto.sync.getLatestCommit`.
75 - [x] Implement `com.atproto.sync.getRecord` (Sync version, distinct from repo.getRecord).
76 - [x] Implement `com.atproto.sync.getRepoStatus`.
77 - [x] Implement `com.atproto.sync.listRepos`.
78 - [x] Implement `com.atproto.sync.notifyOfUpdate`.
79- [x] Blob Sync
80 - [x] Implement `com.atproto.sync.getBlob`.
81 - [x] Implement `com.atproto.sync.listBlobs`.
82- [x] Crawler Interaction
83 - [x] Implement `com.atproto.sync.requestCrawl` (Notify relays to index us).
84- [x] Deprecated Sync Endpoints (for compatibility)
85 - [x] Implement `com.atproto.sync.getCheckout` (deprecated).
86 - [x] Implement `com.atproto.sync.getHead` (deprecated).
87
88## Identity (`com.atproto.identity`)
89- [x] Resolution
90 - [x] Implement `com.atproto.identity.resolveHandle` (Can be internal or proxy to PLC).
91 - [x] Implement `com.atproto.identity.updateHandle`.
92 - [x] Implement `com.atproto.identity.submitPlcOperation` / `signPlcOperation` / `requestPlcOperationSignature`.
93 - [x] Implement `com.atproto.identity.getRecommendedDidCredentials`.
94 - [x] Implement `/.well-known/did.json` (Depends on supporting did:web).
95
96## Admin Management (`com.atproto.admin`)
97- [x] Implement `com.atproto.admin.deleteAccount`.
98- [x] Implement `com.atproto.admin.disableAccountInvites`.
99- [x] Implement `com.atproto.admin.disableInviteCodes`.
100- [x] Implement `com.atproto.admin.enableAccountInvites`.
101- [x] Implement `com.atproto.admin.getAccountInfo` / `getAccountInfos`.
102- [x] Implement `com.atproto.admin.getInviteCodes`.
103- [x] Implement `com.atproto.admin.getSubjectStatus`.
104- [x] Implement `com.atproto.admin.sendEmail`.
105- [x] Implement `com.atproto.admin.updateAccountEmail`.
106- [x] Implement `com.atproto.admin.updateAccountHandle`.
107- [x] Implement `com.atproto.admin.updateAccountPassword`.
108- [x] Implement `com.atproto.admin.updateSubjectStatus`.
109
110## Moderation (`com.atproto.moderation`)
111- [x] Implement `com.atproto.moderation.createReport`.
112
113## Temp Namespace (`com.atproto.temp`)
114- [x] Implement `com.atproto.temp.checkSignupQueue` (signup queue status for gated signups).
115
116## Misc HTTP Endpoints
117- [x] Implement `/robots.txt` endpoint.
118
119## OAuth 2.1 Support
120Full OAuth 2.1 provider for ATProto native app authentication.
121- [x] OAuth Provider Core
122 - [x] Implement `/.well-known/oauth-protected-resource` metadata endpoint.
123 - [x] Implement `/.well-known/oauth-authorization-server` metadata endpoint.
124 - [x] Implement `/oauth/authorize` authorization endpoint (with login UI).
125 - [x] Implement `/oauth/par` Pushed Authorization Request endpoint.
126 - [x] Implement `/oauth/token` token endpoint (authorization_code + refresh_token grants).
127 - [x] Implement `/oauth/jwks` JSON Web Key Set endpoint.
128 - [x] Implement `/oauth/revoke` token revocation endpoint.
129 - [x] Implement `/oauth/introspect` token introspection endpoint.
130- [x] OAuth Database Tables
131 - [x] Device table for tracking authorized devices.
132 - [x] Authorization request table.
133 - [x] Authorized client table.
134 - [x] Token table for OAuth tokens.
135 - [x] Used refresh token table (replay protection).
136 - [x] DPoP JTI tracking table.
137- [x] DPoP (Demonstrating Proof-of-Possession) support.
138- [x] Client metadata fetching and validation.
139- [x] PKCE (S256) enforcement.
140- [x] OAuth token verification extractor for protected resources.
141- [x] Authorization UI templates (HTML login form).
142- [x] Implement `private_key_jwt` signature verification with async JWKS fetching.
143- [x] HS256 JWT support (matches reference PDS).
144
145## OAuth Security Notes
146
147Security measures implemented:
148
149- Constant-time comparison for signature verification (prevents timing attacks)
150- HMAC-SHA256 for access token signing with configurable secret
151- Production secrets require 32+ character minimum
152- DPoP JTI replay protection via database
153- DPoP nonce validation with HMAC-based timestamps (5 min validity)
154- Refresh token rotation with reuse detection (revokes token family on reuse)
155- PKCE S256 enforced (plain not allowed)
156- Authorization code single-use enforcement
157- URL encoding for redirect parameters (prevents injection)
158- All database queries use parameterized statements (no SQL injection)
159- Deactivated/taken-down accounts blocked from OAuth authorization
160- Client ID validation on token exchange (defense-in-depth against cross-client attacks)
161- HTML escaping in OAuth templates (XSS prevention)
162
163### Auth Notes
164- Dual algorithm support: ES256K (secp256k1 ECDSA) with per-user keys AND HS256 (HMAC) for compatibility with reference PDS.
165- Token storage: Storing only token JTIs in session_tokens table (defense in depth against DB breaches). Refresh token family tracking enables detection of token reuse attacks.
166- Key encryption: User signing keys encrypted at rest using AES-256-GCM with keys derived via HKDF from KEY_ENCRYPTION_KEY environment variable.
167
168## PDS-Level App Endpoints
169These endpoints need to be implemented at the PDS level (not just proxied to appview).
170
171### Actor (`app.bsky.actor`)
172- [x] Implement `app.bsky.actor.getPreferences` (user preferences storage).
173- [x] Implement `app.bsky.actor.putPreferences` (update user preferences).
174- [x] Implement `app.bsky.actor.getProfile` (PDS-level with proxy fallback).
175- [x] Implement `app.bsky.actor.getProfiles` (PDS-level with proxy fallback).
176
177### Feed (`app.bsky.feed`)
178These are implemented at PDS level to enable local-first reads (read-after-write pattern):
179- [x] Implement `app.bsky.feed.getTimeline` (PDS-level with proxy + RAW).
180- [x] Implement `app.bsky.feed.getAuthorFeed` (PDS-level with proxy + RAW).
181- [x] Implement `app.bsky.feed.getActorLikes` (PDS-level with proxy + RAW).
182- [x] Implement `app.bsky.feed.getPostThread` (PDS-level with proxy + RAW + NotFound handling).
183- [x] Implement `app.bsky.feed.getFeed` (proxy to feed generator).
184
185### Notification (`app.bsky.notification`)
186- [x] Implement `app.bsky.notification.registerPush` (push notification registration, proxied).
187
188## Infrastructure & Core Components
189- [x] Sequencer (Event Log)
190 - [x] Implement a `Sequencer` (backed by `repo_seq` table).
191 - [x] Implement event formatting (`commit`, `handle`, `identity`, `account`).
192 - [x] Implement database polling / event emission mechanism.
193 - [x] Implement cursor-based event replay (`requestSeqRange`).
194- [x] Repo Storage & Consistency (in postgres)
195 - [x] Implement `RepoStorage` for postgres (replaces per-user SQLite).
196 - [x] Read/Write IPLD blocks to `blocks` table (global deduplication).
197 - [x] Manage Repo Root in `repos` table.
198 - [x] Implement Atomic Repo Transactions.
199 - [x] Ensure `blocks` write, `repo_root` update, `records` index update, and `sequencer` event are committed in a single transaction.
200 - [x] Implement concurrency control (row-level locking via FOR UPDATE).
201- [x] DID Cache
202 - [x] Implement caching layer for DID resolution (valkey).
203 - [x] Handle cache invalidation/expiry.
204 - [x] Graceful fallback to no-cache when Valkey unavailable.
205- [x] Crawlers Service
206 - [x] Implement `Crawlers` service (debounce notifications to relays).
207 - [x] 20-minute notification debounce.
208 - [x] Circuit breaker for relay failures.
209- [x] Notification Service
210 - [x] Queue-based notification system with database table
211 - [x] Background worker polling for pending notifications
212 - [x] Extensible sender trait for multiple channels
213 - [x] Email sender via OS sendmail/msmtp
214 - [x] Discord webhook sender
215 - [x] Telegram bot sender
216 - [x] Signal CLI sender
217 - [x] Helper functions for common notification types (welcome, password reset, email verification, etc.)
218 - [x] Respect user's `preferred_notification_channel` setting for non-email-specific notifications
219- [x] Image Processing
220 - [x] Implement image resize/formatting pipeline (for blob uploads).
221 - [x] WebP conversion for thumbnails.
222 - [x] EXIF stripping.
223 - [x] File size limits (10MB default).
224- [x] IPLD & MST
225 - [x] Implement Merkle Search Tree logic for repo signing.
226 - [x] Implement CAR (Content Addressable Archive) encoding/decoding.
227 - [x] Cycle detection in CAR export.
228- [x] Rate Limiting
229 - [x] Per-IP rate limiting on login (10/min).
230 - [x] Per-IP rate limiting on OAuth token endpoint (30/min).
231 - [x] Per-IP rate limiting on password reset (5/hour).
232 - [x] Per-IP rate limiting on account creation (10/hour).
233 - [x] Per-IP rate limiting on refreshSession (60/min).
234 - [x] Per-IP rate limiting on OAuth authorize POST (10/min).
235 - [x] Per-IP rate limiting on OAuth 2FA POST (10/min).
236 - [x] Per-IP rate limiting on OAuth PAR (30/min).
237 - [x] Per-IP rate limiting on OAuth revoke/introspect (30/min).
238 - [x] Per-IP rate limiting on createAppPassword (10/min).
239 - [x] Per-IP rate limiting on email endpoints (5/hour).
240 - [x] Distributed rate limiting via Valkey/Redis (with in-memory fallback).
241- [x] Circuit Breakers
242 - [x] PLC directory circuit breaker (5 failures → open, 60s timeout).
243 - [x] Relay notification circuit breaker (10 failures → open, 30s timeout).
244- [x] Security Hardening
245 - [x] Email header injection prevention (CRLF sanitization).
246 - [x] Signal command injection prevention (phone number validation).
247 - [x] Constant-time signature comparison.
248 - [x] SSRF protection for outbound requests.
249 - [x] Timing attack protection (dummy bcrypt on user-not-found prevents account enumeration).
250
251## Lewis' fabulous mini-list of remaining TODOs
252- [x] The OAuth authorize POST endpoint has no rate limiting, allowing password brute-forcing. Fix this and audit all oauth and 2fa surface again.
253- [x] DID resolution caching (valkey).
254- [x] Record schema validation (generic validation framework).
255- [x] Fix any remaining TODOs in the code.
256
257## Future: Web Management UI
258A single-page web app for account management. The frontend (JS framework) calls existing ATProto XRPC endpoints - no server-side rendering or bespoke HTML form handlers.
259
260### Architecture
261- [ ] Static SPA served from PDS (or separate static host)
262- [ ] Frontend authenticates via OAuth 2.1 flow (same as any ATProto client)
263- [ ] All operations use standard XRPC endpoints (existing + new PDS-specific ones below)
264- [ ] No server-side sessions or CSRF - pure API client
265
266### PDS-Specific XRPC Endpoints (new)
267Absolutely subject to change, "bspds" isn't even the real name of this pds thus far :D
268Anyway... endpoints for PDS settings not covered by standard ATProto:
269- [ ] `com.bspds.account.getNotificationPrefs` - get preferred channel, verified channels
270- [ ] `com.bspds.account.updateNotificationPrefs` - set preferred channel
271- [ ] `com.bspds.account.getNotificationHistory` - list past notifications
272- [ ] `com.bspds.account.verifyChannel` - initiate verification for Discord/Telegram/Signal
273- [ ] `com.bspds.account.confirmChannelVerification` - confirm with code
274- [ ] `com.bspds.admin.getServerStats` - user count, storage usage, etc.
275
276### Frontend Views
277Uses existing ATProto endpoints where possible:
278
279User Dashboard
280- [ ] Account overview (uses `com.atproto.server.getSession`, `com.atproto.admin.getAccountInfo`)
281- [ ] Active sessions view (needs new endpoint or extend existing)
282- [ ] App passwords (uses `com.atproto.server.listAppPasswords`, `createAppPassword`, `revokeAppPassword`)
283- [ ] Invite codes (uses `com.atproto.server.getAccountInviteCodes`, `createInviteCode`)
284
285Notification Preferences
286- [ ] Channel selector (uses `com.bspds.account.*` endpoints above)
287- [ ] Verification flows for Discord/Telegram/Signal
288- [ ] Notification history view
289
290Account Settings
291- [ ] Email change (uses `com.atproto.server.requestEmailUpdate`, `updateEmail`)
292- [ ] Password change (uses `com.atproto.server.requestPasswordReset`, `resetPassword`)
293- [ ] Handle change (uses `com.atproto.identity.updateHandle`)
294- [ ] Account deletion (uses `com.atproto.server.requestAccountDelete`, `deleteAccount`)
295- [ ] Data export (uses `com.atproto.sync.getRepo`)
296
297Admin Dashboard (privileged users only)
298- [ ] User list (uses `com.atproto.admin.getAccountInfos` with pagination)
299- [ ] User detail/actions (uses `com.atproto.admin.*` endpoints)
300- [ ] Invite management (uses `com.atproto.admin.getInviteCodes`, `disableInviteCodes`)
301- [ ] Server stats (uses `com.bspds.admin.getServerStats`)
302
303## Future: private data
304I will see where the discourse about encrypted/privileged private data is at the current moment, and make an implementation that matches what the bsky team will likely do in their pds whenever they get around to it.
305Then when they come out with theirs, I can make adjustments to mine and be ready on day 1. Or 2.
306
307We want records that only authorized parties can see and decrypt. This requires some sort of federation of keys and communication between PDSes?
308Gotta figure all of this out as a first step.
309