this repo has no description
lewis.moe
1# PDS Implementation TODOs
2
3Lewis' corrected big boy todofile
4
5## Server Infrastructure & Proxying
6- [x] Health Check
7 - [x] Implement `GET /health` endpoint (returns "OK").
8 - [x] Implement `GET /xrpc/_health` endpoint (returns "OK").
9- [x] Server Description
10 - [x] Implement `com.atproto.server.describeServer` (returns available user domains).
11- [x] XRPC Proxying
12 - [x] Implement strict forwarding for all `app.bsky.*` and `chat.bsky.*` requests to an appview.
13 - [x] Forward auth headers correctly.
14 - [x] Handle appview errors/timeouts gracefully.
15
16## Authentication & Account Management (`com.atproto.server`)
17- [x] Account Creation
18 - [x] Implement `com.atproto.server.createAccount`.
19 - [x] Validate handle format (reject invalid characters).
20 - [x] Create DID for new user (PLC directory).
21 - [x] Initialize user repository (Root commit).
22 - [x] Return access JWT and DID.
23 - [x] Create DID for new user (did:web).
24- [x] Session Management
25 - [x] Implement `com.atproto.server.createSession` (Login).
26 - [x] Implement `com.atproto.server.getSession`.
27 - [x] Implement `com.atproto.server.refreshSession`.
28 - [x] Implement `com.atproto.server.deleteSession` (Logout).
29 - [x] Implement `com.atproto.server.activateAccount`.
30 - [x] Implement `com.atproto.server.checkAccountStatus`.
31 - [x] Implement `com.atproto.server.createAppPassword`.
32 - [x] Implement `com.atproto.server.createInviteCode`.
33 - [x] Implement `com.atproto.server.createInviteCodes`.
34 - [x] Implement `com.atproto.server.deactivateAccount`.
35 - [x] Implement `com.atproto.server.deleteAccount` (user-initiated, requires password + email token).
36 - [x] Implement `com.atproto.server.getAccountInviteCodes`.
37 - [x] Implement `com.atproto.server.getServiceAuth` (Cross-service auth).
38 - [x] Implement `com.atproto.server.listAppPasswords`.
39 - [x] Implement `com.atproto.server.requestAccountDelete`.
40 - [x] Implement `com.atproto.server.requestEmailConfirmation` / `requestEmailUpdate`.
41 - [x] Implement `com.atproto.server.requestPasswordReset` / `resetPassword`.
42 - [x] Implement `com.atproto.server.reserveSigningKey`.
43 - [x] Implement `com.atproto.server.revokeAppPassword`.
44 - [x] Implement `com.atproto.server.updateEmail`.
45 - [x] Implement `com.atproto.server.confirmEmail`.
46
47## Repository Operations (`com.atproto.repo`)
48- [x] Record CRUD
49 - [x] Implement `com.atproto.repo.createRecord`.
50 - [x] Generate `rkey` (TID) if not provided.
51 - [x] Handle MST (Merkle Search Tree) insertion.
52 - [x] **Trigger Firehose Event**.
53 - [x] Implement `com.atproto.repo.putRecord`.
54 - [x] Implement `com.atproto.repo.getRecord`.
55 - [x] Implement `com.atproto.repo.deleteRecord`.
56 - [x] Implement `com.atproto.repo.listRecords`.
57 - [x] Implement `com.atproto.repo.describeRepo`.
58 - [x] Implement `com.atproto.repo.applyWrites` (Batch writes).
59 - [x] Implement `com.atproto.repo.importRepo` (Migration).
60 - [x] Implement `com.atproto.repo.listMissingBlobs`.
61- [x] Blob Management
62 - [x] Implement `com.atproto.repo.uploadBlob`.
63 - [x] Store blob (S3).
64 - [x] return `blob` ref (CID + MimeType).
65
66## Sync & Federation (`com.atproto.sync`)
67- [x] The Firehose (WebSocket)
68 - [x] Implement `com.atproto.sync.subscribeRepos`.
69 - [x] Broadcast real-time commit events.
70 - [x] Handle cursor replay (backfill).
71- [x] Bulk Export
72 - [x] Implement `com.atproto.sync.getRepo` (Return full CAR file of repo).
73 - [x] Implement `com.atproto.sync.getBlocks` (Return specific blocks via CIDs).
74 - [x] Implement `com.atproto.sync.getLatestCommit`.
75 - [x] Implement `com.atproto.sync.getRecord` (Sync version, distinct from repo.getRecord).
76 - [x] Implement `com.atproto.sync.getRepoStatus`.
77 - [x] Implement `com.atproto.sync.listRepos`.
78 - [x] Implement `com.atproto.sync.notifyOfUpdate`.
79- [x] Blob Sync
80 - [x] Implement `com.atproto.sync.getBlob`.
81 - [x] Implement `com.atproto.sync.listBlobs`.
82- [x] Crawler Interaction
83 - [x] Implement `com.atproto.sync.requestCrawl` (Notify relays to index us).
84- [x] Deprecated Sync Endpoints (for compatibility)
85 - [x] Implement `com.atproto.sync.getCheckout` (deprecated).
86 - [x] Implement `com.atproto.sync.getHead` (deprecated).
87
88## Identity (`com.atproto.identity`)
89- [x] Resolution
90 - [x] Implement `com.atproto.identity.resolveHandle` (Can be internal or proxy to PLC).
91 - [x] Implement `com.atproto.identity.updateHandle`.
92 - [x] Implement `com.atproto.identity.submitPlcOperation` / `signPlcOperation` / `requestPlcOperationSignature`.
93 - [x] Implement `com.atproto.identity.getRecommendedDidCredentials`.
94 - [x] Implement `/.well-known/did.json` (Depends on supporting did:web).
95
96## Admin Management (`com.atproto.admin`)
97- [x] Implement `com.atproto.admin.deleteAccount`.
98- [x] Implement `com.atproto.admin.disableAccountInvites`.
99- [x] Implement `com.atproto.admin.disableInviteCodes`.
100- [x] Implement `com.atproto.admin.enableAccountInvites`.
101- [x] Implement `com.atproto.admin.getAccountInfo` / `getAccountInfos`.
102- [x] Implement `com.atproto.admin.getInviteCodes`.
103- [x] Implement `com.atproto.admin.getSubjectStatus`.
104- [x] Implement `com.atproto.admin.sendEmail`.
105- [x] Implement `com.atproto.admin.updateAccountEmail`.
106- [x] Implement `com.atproto.admin.updateAccountHandle`.
107- [x] Implement `com.atproto.admin.updateAccountPassword`.
108- [x] Implement `com.atproto.admin.updateSubjectStatus`.
109
110## Moderation (`com.atproto.moderation`)
111- [x] Implement `com.atproto.moderation.createReport`.
112
113## Temp Namespace (`com.atproto.temp`)
114- [x] Implement `com.atproto.temp.checkSignupQueue` (signup queue status for gated signups).
115
116## Misc HTTP Endpoints
117- [x] Implement `/robots.txt` endpoint.
118
119## OAuth 2.1 Support
120Full OAuth 2.1 provider for ATProto native app authentication.
121- [x] OAuth Provider Core
122 - [x] Implement `/.well-known/oauth-protected-resource` metadata endpoint.
123 - [x] Implement `/.well-known/oauth-authorization-server` metadata endpoint.
124 - [x] Implement `/oauth/authorize` authorization endpoint (with login UI).
125 - [x] Implement `/oauth/par` Pushed Authorization Request endpoint.
126 - [x] Implement `/oauth/token` token endpoint (authorization_code + refresh_token grants).
127 - [x] Implement `/oauth/jwks` JSON Web Key Set endpoint.
128 - [x] Implement `/oauth/revoke` token revocation endpoint.
129 - [x] Implement `/oauth/introspect` token introspection endpoint.
130- [x] OAuth Database Tables
131 - [x] Device table for tracking authorized devices.
132 - [x] Authorization request table.
133 - [x] Authorized client table.
134 - [x] Token table for OAuth tokens.
135 - [x] Used refresh token table (replay protection).
136 - [x] DPoP JTI tracking table.
137- [x] DPoP (Demonstrating Proof-of-Possession) support.
138- [x] Client metadata fetching and validation.
139- [x] PKCE (S256) enforcement.
140- [x] OAuth token verification extractor for protected resources.
141- [x] Authorization UI templates (HTML login form).
142- [x] Implement `private_key_jwt` signature verification with async JWKS fetching.
143- [x] HS256 JWT support (matches reference PDS).
144
145## OAuth Security Notes
146
147Security measures implemented:
148
149- Constant-time comparison for signature verification (prevents timing attacks)
150- HMAC-SHA256 for access token signing with configurable secret
151- Production secrets require 32+ character minimum
152- DPoP JTI replay protection via database
153- DPoP nonce validation with HMAC-based timestamps (5 min validity)
154- Refresh token rotation with reuse detection (revokes token family on reuse)
155- PKCE S256 enforced (plain not allowed)
156- Authorization code single-use enforcement
157- URL encoding for redirect parameters (prevents injection)
158- All database queries use parameterized statements (no SQL injection)
159- Deactivated/taken-down accounts blocked from OAuth authorization
160- Client ID validation on token exchange (defense-in-depth against cross-client attacks)
161- HTML escaping in OAuth templates (XSS prevention)
162
163### Auth Notes
164- Dual algorithm support: ES256K (secp256k1 ECDSA) with per-user keys AND HS256 (HMAC) for compatibility with reference PDS.
165- Token storage: Storing only token JTIs in session_tokens table (defense in depth against DB breaches). Refresh token family tracking enables detection of token reuse attacks.
166- Key encryption: User signing keys encrypted at rest using AES-256-GCM with keys derived via HKDF from KEY_ENCRYPTION_KEY environment variable.
167
168## PDS-Level App Endpoints
169These endpoints need to be implemented at the PDS level (not just proxied to appview).
170
171### Actor (`app.bsky.actor`)
172- [x] Implement `app.bsky.actor.getPreferences` (user preferences storage).
173- [x] Implement `app.bsky.actor.putPreferences` (update user preferences).
174- [x] Implement `app.bsky.actor.getProfile` (PDS-level with proxy fallback).
175- [x] Implement `app.bsky.actor.getProfiles` (PDS-level with proxy fallback).
176
177### Feed (`app.bsky.feed`)
178These are implemented at PDS level to enable local-first reads (read-after-write pattern):
179- [x] Implement `app.bsky.feed.getTimeline` (PDS-level with proxy + RAW).
180- [x] Implement `app.bsky.feed.getAuthorFeed` (PDS-level with proxy + RAW).
181- [x] Implement `app.bsky.feed.getActorLikes` (PDS-level with proxy + RAW).
182- [x] Implement `app.bsky.feed.getPostThread` (PDS-level with proxy + RAW + NotFound handling).
183- [x] Implement `app.bsky.feed.getFeed` (proxy to feed generator).
184
185### Notification (`app.bsky.notification`)
186- [x] Implement `app.bsky.notification.registerPush` (push notification registration, proxied).
187
188## Infrastructure & Core Components
189- [x] Sequencer (Event Log)
190 - [x] Implement a `Sequencer` (backed by `repo_seq` table).
191 - [x] Implement event formatting (`commit`, `handle`, `identity`, `account`).
192 - [x] Implement database polling / event emission mechanism.
193 - [x] Implement cursor-based event replay (`requestSeqRange`).
194- [x] Repo Storage & Consistency (in postgres)
195 - [x] Implement `RepoStorage` for postgres (replaces per-user SQLite).
196 - [x] Read/Write IPLD blocks to `blocks` table (global deduplication).
197 - [x] Manage Repo Root in `repos` table.
198 - [x] Implement Atomic Repo Transactions.
199 - [x] Ensure `blocks` write, `repo_root` update, `records` index update, and `sequencer` event are committed in a single transaction.
200 - [x] Implement concurrency control (row-level locking via FOR UPDATE).
201- [ ] DID Cache
202 - [ ] Implement caching layer for DID resolution (Redis or in-memory).
203 - [ ] Handle cache invalidation/expiry.
204- [x] Crawlers Service
205 - [x] Implement `Crawlers` service (debounce notifications to relays).
206 - [x] 20-minute notification debounce.
207 - [x] Circuit breaker for relay failures.
208- [x] Notification Service
209 - [x] Queue-based notification system with database table
210 - [x] Background worker polling for pending notifications
211 - [x] Extensible sender trait for multiple channels
212 - [x] Email sender via OS sendmail/msmtp
213 - [x] Discord webhook sender
214 - [x] Telegram bot sender
215 - [x] Signal CLI sender
216 - [x] Helper functions for common notification types (welcome, password reset, email verification, etc.)
217 - [x] Respect user's `preferred_notification_channel` setting for non-email-specific notifications
218- [x] Image Processing
219 - [x] Implement image resize/formatting pipeline (for blob uploads).
220 - [x] WebP conversion for thumbnails.
221 - [x] EXIF stripping.
222 - [x] File size limits (10MB default).
223- [x] IPLD & MST
224 - [x] Implement Merkle Search Tree logic for repo signing.
225 - [x] Implement CAR (Content Addressable Archive) encoding/decoding.
226 - [x] Cycle detection in CAR export.
227- [x] Rate Limiting
228 - [x] Per-IP rate limiting on login (10/min).
229 - [x] Per-IP rate limiting on OAuth token endpoint (30/min).
230 - [x] Per-IP rate limiting on password reset (5/hour).
231 - [x] Per-IP rate limiting on account creation (10/hour).
232- [x] Circuit Breakers
233 - [x] PLC directory circuit breaker (5 failures → open, 60s timeout).
234 - [x] Relay notification circuit breaker (10 failures → open, 30s timeout).
235- [x] Security Hardening
236 - [x] Email header injection prevention (CRLF sanitization).
237 - [x] Signal command injection prevention (phone number validation).
238 - [x] Constant-time signature comparison.
239 - [x] SSRF protection for outbound requests.
240
241## Lewis' fabulous mini-list of remaining TODOs
242- [ ] The OAuth authorize POST endpoint has no rate limiting, allowing password brute-forcing. Fix this and audit all oauth and 2fa surface again.
243- [ ] DID resolution caching (valkey).
244- [ ] Record schema validation (generic validation framework).
245- [ ] Fix any remaining TODOs in the code.
246
247## Future: Web Management UI
248A single-page web app for account management. The frontend (JS framework) calls existing ATProto XRPC endpoints - no server-side rendering or bespoke HTML form handlers.
249
250### Architecture
251- [ ] Static SPA served from PDS (or separate static host)
252- [ ] Frontend authenticates via OAuth 2.1 flow (same as any ATProto client)
253- [ ] All operations use standard XRPC endpoints (existing + new PDS-specific ones below)
254- [ ] No server-side sessions or CSRF - pure API client
255
256### PDS-Specific XRPC Endpoints (new)
257Absolutely subject to change, "bspds" isn't even the real name of this pds thus far :D
258Anyway... endpoints for PDS settings not covered by standard ATProto:
259- [ ] `com.bspds.account.getNotificationPrefs` - get preferred channel, verified channels
260- [ ] `com.bspds.account.updateNotificationPrefs` - set preferred channel
261- [ ] `com.bspds.account.getNotificationHistory` - list past notifications
262- [ ] `com.bspds.account.verifyChannel` - initiate verification for Discord/Telegram/Signal
263- [ ] `com.bspds.account.confirmChannelVerification` - confirm with code
264- [ ] `com.bspds.admin.getServerStats` - user count, storage usage, etc.
265
266### Frontend Views
267Uses existing ATProto endpoints where possible:
268
269User Dashboard
270- [ ] Account overview (uses `com.atproto.server.getSession`, `com.atproto.admin.getAccountInfo`)
271- [ ] Active sessions view (needs new endpoint or extend existing)
272- [ ] App passwords (uses `com.atproto.server.listAppPasswords`, `createAppPassword`, `revokeAppPassword`)
273- [ ] Invite codes (uses `com.atproto.server.getAccountInviteCodes`, `createInviteCode`)
274
275Notification Preferences
276- [ ] Channel selector (uses `com.bspds.account.*` endpoints above)
277- [ ] Verification flows for Discord/Telegram/Signal
278- [ ] Notification history view
279
280Account Settings
281- [ ] Email change (uses `com.atproto.server.requestEmailUpdate`, `updateEmail`)
282- [ ] Password change (uses `com.atproto.server.requestPasswordReset`, `resetPassword`)
283- [ ] Handle change (uses `com.atproto.identity.updateHandle`)
284- [ ] Account deletion (uses `com.atproto.server.requestAccountDelete`, `deleteAccount`)
285- [ ] Data export (uses `com.atproto.sync.getRepo`)
286
287Admin Dashboard (privileged users only)
288- [ ] User list (uses `com.atproto.admin.getAccountInfos` with pagination)
289- [ ] User detail/actions (uses `com.atproto.admin.*` endpoints)
290- [ ] Invite management (uses `com.atproto.admin.getInviteCodes`, `disableInviteCodes`)
291- [ ] Server stats (uses `com.bspds.admin.getServerStats`)
292
293## Future: private data
294I will see where the discourse about encrypted/privileged private data is at the current moment, and make an implementation that matches what the bsky team will likely do in their pds whenever they get around to it.
295Then when they come out with theirs, I can make adjustments to mine and be ready on day 1. Or 2.
296
297We want records that only authorized parties can see and decrypt. This requires some sort of federation of keys and communication between PDSes?
298Gotta figure all of this out as a first step.
299