this repo has no description
lewis.moe
1# PDS Implementation TODOs
2Lewis' corrected big boy todofile
3## Server Infrastructure & Proxying
4- [x] Health Check
5 - [x] Implement `GET /health` endpoint (returns "OK").
6 - [x] Implement `GET /xrpc/_health` endpoint (returns "OK").
7- [x] Server Description
8 - [x] Implement `com.atproto.server.describeServer` (returns available user domains).
9- [x] XRPC Proxying
10 - [x] Implement strict forwarding for all `app.bsky.*` and `chat.bsky.*` requests to an appview.
11 - [x] Forward auth headers correctly.
12 - [x] Handle appview errors/timeouts gracefully.
13## Authentication & Account Management (`com.atproto.server`)
14- [x] Account Creation
15 - [x] Implement `com.atproto.server.createAccount`.
16 - [x] Validate handle format (reject invalid characters).
17 - [x] Create DID for new user (PLC directory).
18 - [x] Initialize user repository (Root commit).
19 - [x] Return access JWT and DID.
20 - [x] Create DID for new user (did:web).
21- [x] Session Management
22 - [x] Implement `com.atproto.server.createSession` (Login).
23 - [x] Implement `com.atproto.server.getSession`.
24 - [x] Implement `com.atproto.server.refreshSession`.
25 - [x] Implement `com.atproto.server.deleteSession` (Logout).
26 - [x] Implement `com.atproto.server.activateAccount`.
27 - [x] Implement `com.atproto.server.checkAccountStatus`.
28 - [x] Implement `com.atproto.server.createAppPassword`.
29 - [x] Implement `com.atproto.server.createInviteCode`.
30 - [x] Implement `com.atproto.server.createInviteCodes`.
31 - [x] Implement `com.atproto.server.deactivateAccount`.
32 - [x] Implement `com.atproto.server.deleteAccount` (user-initiated, requires password + email token).
33 - [x] Implement `com.atproto.server.getAccountInviteCodes`.
34 - [x] Implement `com.atproto.server.getServiceAuth` (Cross-service auth).
35 - [x] Implement `com.atproto.server.listAppPasswords`.
36 - [x] Implement `com.atproto.server.requestAccountDelete`.
37 - [x] Implement `com.atproto.server.requestEmailConfirmation` / `requestEmailUpdate`.
38 - [x] Implement `com.atproto.server.requestPasswordReset` / `resetPassword`.
39 - [x] Implement `com.atproto.server.reserveSigningKey`.
40 - [x] Implement `com.atproto.server.revokeAppPassword`.
41 - [x] Implement `com.atproto.server.updateEmail`.
42 - [x] Implement `com.atproto.server.confirmEmail`.
43## Repository Operations (`com.atproto.repo`)
44- [x] Record CRUD
45 - [x] Implement `com.atproto.repo.createRecord`.
46 - [x] Generate `rkey` (TID) if not provided.
47 - [x] Handle MST (Merkle Search Tree) insertion.
48 - [x] **Trigger Firehose Event**.
49 - [x] Implement `com.atproto.repo.putRecord`.
50 - [x] Implement `com.atproto.repo.getRecord`.
51 - [x] Implement `com.atproto.repo.deleteRecord`.
52 - [x] Implement `com.atproto.repo.listRecords`.
53 - [x] Implement `com.atproto.repo.describeRepo`.
54 - [x] Implement `com.atproto.repo.applyWrites` (Batch writes).
55 - [x] Implement `com.atproto.repo.importRepo` (Migration).
56 - [x] Implement `com.atproto.repo.listMissingBlobs`.
57- [x] Blob Management
58 - [x] Implement `com.atproto.repo.uploadBlob`.
59 - [x] Store blob (S3).
60 - [x] return `blob` ref (CID + MimeType).
61## Sync & Federation (`com.atproto.sync`)
62- [x] The Firehose (WebSocket)
63 - [x] Implement `com.atproto.sync.subscribeRepos`.
64 - [x] Broadcast real-time commit events.
65 - [x] Handle cursor replay (backfill).
66- [x] Bulk Export
67 - [x] Implement `com.atproto.sync.getRepo` (Return full CAR file of repo).
68 - [x] Implement `com.atproto.sync.getBlocks` (Return specific blocks via CIDs).
69 - [x] Implement `com.atproto.sync.getLatestCommit`.
70 - [x] Implement `com.atproto.sync.getRecord` (Sync version, distinct from repo.getRecord).
71 - [x] Implement `com.atproto.sync.getRepoStatus`.
72 - [x] Implement `com.atproto.sync.listRepos`.
73 - [x] Implement `com.atproto.sync.notifyOfUpdate`.
74- [x] Blob Sync
75 - [x] Implement `com.atproto.sync.getBlob`.
76 - [x] Implement `com.atproto.sync.listBlobs`.
77- [x] Crawler Interaction
78 - [x] Implement `com.atproto.sync.requestCrawl` (Notify relays to index us).
79- [x] Deprecated Sync Endpoints (for compatibility)
80 - [x] Implement `com.atproto.sync.getCheckout` (deprecated).
81 - [x] Implement `com.atproto.sync.getHead` (deprecated).
82## Identity (`com.atproto.identity`)
83- [x] Resolution
84 - [x] Implement `com.atproto.identity.resolveHandle` (Can be internal or proxy to PLC).
85 - [x] Implement `com.atproto.identity.updateHandle`.
86 - [x] Implement `com.atproto.identity.submitPlcOperation` / `signPlcOperation` / `requestPlcOperationSignature`.
87 - [x] Implement `com.atproto.identity.getRecommendedDidCredentials`.
88 - [x] Implement `/.well-known/did.json` (Depends on supporting did:web).
89## Admin Management (`com.atproto.admin`)
90- [x] Implement `com.atproto.admin.deleteAccount`.
91- [x] Implement `com.atproto.admin.disableAccountInvites`.
92- [x] Implement `com.atproto.admin.disableInviteCodes`.
93- [x] Implement `com.atproto.admin.enableAccountInvites`.
94- [x] Implement `com.atproto.admin.getAccountInfo` / `getAccountInfos`.
95- [x] Implement `com.atproto.admin.getInviteCodes`.
96- [x] Implement `com.atproto.admin.getSubjectStatus`.
97- [x] Implement `com.atproto.admin.sendEmail`.
98- [x] Implement `com.atproto.admin.updateAccountEmail`.
99- [x] Implement `com.atproto.admin.updateAccountHandle`.
100- [x] Implement `com.atproto.admin.updateAccountPassword`.
101- [x] Implement `com.atproto.admin.updateSubjectStatus`.
102## Moderation (`com.atproto.moderation`)
103- [x] Implement `com.atproto.moderation.createReport`.
104## Temp Namespace (`com.atproto.temp`)
105- [x] Implement `com.atproto.temp.checkSignupQueue` (signup queue status for gated signups).
106## Misc HTTP Endpoints
107- [x] Implement `/robots.txt` endpoint.
108## OAuth 2.1 Support
109Full OAuth 2.1 provider for ATProto native app authentication.
110- [x] OAuth Provider Core
111 - [x] Implement `/.well-known/oauth-protected-resource` metadata endpoint.
112 - [x] Implement `/.well-known/oauth-authorization-server` metadata endpoint.
113 - [x] Implement `/oauth/authorize` authorization endpoint (with login UI).
114 - [x] Implement `/oauth/par` Pushed Authorization Request endpoint.
115 - [x] Implement `/oauth/token` token endpoint (authorization_code + refresh_token grants).
116 - [x] Implement `/oauth/jwks` JSON Web Key Set endpoint.
117 - [x] Implement `/oauth/revoke` token revocation endpoint.
118 - [x] Implement `/oauth/introspect` token introspection endpoint.
119- [x] OAuth Database Tables
120 - [x] Device table for tracking authorized devices.
121 - [x] Authorization request table.
122 - [x] Authorized client table.
123 - [x] Token table for OAuth tokens.
124 - [x] Used refresh token table (replay protection).
125 - [x] DPoP JTI tracking table.
126- [x] DPoP (Demonstrating Proof-of-Possession) support.
127- [x] Client metadata fetching and validation.
128- [x] PKCE (S256) enforcement.
129- [x] OAuth token verification extractor for protected resources.
130- [x] Authorization UI templates (HTML login form).
131- [x] Implement `private_key_jwt` signature verification with async JWKS fetching.
132- [x] HS256 JWT support (matches reference PDS).
133## OAuth Security Notes
134Security measures implemented:
135- Constant-time comparison for signature verification (prevents timing attacks)
136- HMAC-SHA256 for access token signing with configurable secret
137- Production secrets require 32+ character minimum
138- DPoP JTI replay protection via database
139- DPoP nonce validation with HMAC-based timestamps (5 min validity)
140- Refresh token rotation with reuse detection (revokes token family on reuse)
141- PKCE S256 enforced (plain not allowed)
142- Authorization code single-use enforcement
143- URL encoding for redirect parameters (prevents injection)
144- All database queries use parameterized statements (no SQL injection)
145- Deactivated/taken-down accounts blocked from OAuth authorization
146- Client ID validation on token exchange (defense-in-depth against cross-client attacks)
147- HTML escaping in OAuth templates (XSS prevention)
148### Auth Notes
149- Dual algorithm support: ES256K (secp256k1 ECDSA) with per-user keys AND HS256 (HMAC) for compatibility with reference PDS.
150- Token storage: Storing only token JTIs in session_tokens table (defense in depth against DB breaches). Refresh token family tracking enables detection of token reuse attacks.
151- Key encryption: User signing keys encrypted at rest using AES-256-GCM with keys derived via HKDF from KEY_ENCRYPTION_KEY environment variable.
152## PDS-Level App Endpoints
153These endpoints need to be implemented at the PDS level (not just proxied to appview).
154### Actor (`app.bsky.actor`)
155- [x] Implement `app.bsky.actor.getPreferences` (user preferences storage).
156- [x] Implement `app.bsky.actor.putPreferences` (update user preferences).
157- [x] Implement `app.bsky.actor.getProfile` (PDS-level with proxy fallback).
158- [x] Implement `app.bsky.actor.getProfiles` (PDS-level with proxy fallback).
159### Feed (`app.bsky.feed`)
160These are implemented at PDS level to enable local-first reads (read-after-write pattern):
161- [x] Implement `app.bsky.feed.getTimeline` (PDS-level with proxy + RAW).
162- [x] Implement `app.bsky.feed.getAuthorFeed` (PDS-level with proxy + RAW).
163- [x] Implement `app.bsky.feed.getActorLikes` (PDS-level with proxy + RAW).
164- [x] Implement `app.bsky.feed.getPostThread` (PDS-level with proxy + RAW + NotFound handling).
165- [x] Implement `app.bsky.feed.getFeed` (proxy to feed generator).
166### Notification (`app.bsky.notification`)
167- [x] Implement `app.bsky.notification.registerPush` (push notification registration, proxied).
168## Infrastructure & Core Components
169- [x] Sequencer (Event Log)
170 - [x] Implement a `Sequencer` (backed by `repo_seq` table).
171 - [x] Implement event formatting (`commit`, `handle`, `identity`, `account`).
172 - [x] Implement database polling / event emission mechanism.
173 - [x] Implement cursor-based event replay (`requestSeqRange`).
174- [x] Repo Storage & Consistency (in postgres)
175 - [x] Implement `RepoStorage` for postgres (replaces per-user SQLite).
176 - [x] Read/Write IPLD blocks to `blocks` table (global deduplication).
177 - [x] Manage Repo Root in `repos` table.
178 - [x] Implement Atomic Repo Transactions.
179 - [x] Ensure `blocks` write, `repo_root` update, `records` index update, and `sequencer` event are committed in a single transaction.
180 - [x] Implement concurrency control (row-level locking via FOR UPDATE).
181- [x] DID Cache
182 - [x] Implement caching layer for DID resolution (valkey).
183 - [x] Handle cache invalidation/expiry.
184 - [x] Graceful fallback to no-cache when valkey unavailable.
185- [x] Crawlers Service
186 - [x] Implement `Crawlers` service (debounce notifications to relays).
187 - [x] 20-minute notification debounce.
188 - [x] Circuit breaker for relay failures.
189- [x] Notification Service
190 - [x] Queue-based notification system with database table
191 - [x] Background worker polling for pending notifications
192 - [x] Extensible sender trait for multiple channels
193 - [x] Email sender via OS sendmail/msmtp
194 - [x] Discord webhook sender
195 - [x] Telegram bot sender
196 - [x] Signal CLI sender
197 - [x] Helper functions for common notification types (welcome, password reset, email verification, etc.)
198 - [x] Respect user's `preferred_notification_channel` setting for non-email-specific notifications
199- [x] Image Processing
200 - [x] Implement image resize/formatting pipeline (for blob uploads).
201 - [x] WebP conversion for thumbnails.
202 - [x] EXIF stripping.
203 - [x] File size limits (10MB default).
204- [x] IPLD & MST
205 - [x] Implement Merkle Search Tree logic for repo signing.
206 - [x] Implement CAR (Content Addressable Archive) encoding/decoding.
207 - [x] Cycle detection in CAR export.
208- [x] Rate Limiting
209 - [x] Per-IP rate limiting on login (10/min).
210 - [x] Per-IP rate limiting on OAuth token endpoint (30/min).
211 - [x] Per-IP rate limiting on password reset (5/hour).
212 - [x] Per-IP rate limiting on account creation (10/hour).
213 - [x] Per-IP rate limiting on refreshSession (60/min).
214 - [x] Per-IP rate limiting on OAuth authorize POST (10/min).
215 - [x] Per-IP rate limiting on OAuth 2FA POST (10/min).
216 - [x] Per-IP rate limiting on OAuth PAR (30/min).
217 - [x] Per-IP rate limiting on OAuth revoke/introspect (30/min).
218 - [x] Per-IP rate limiting on createAppPassword (10/min).
219 - [x] Per-IP rate limiting on email endpoints (5/hour).
220 - [x] Distributed rate limiting via valkey (with in-memory fallback).
221- [x] Circuit Breakers
222 - [x] PLC directory circuit breaker (5 failures → open, 60s timeout).
223 - [x] Relay notification circuit breaker (10 failures → open, 30s timeout).
224- [x] Security Hardening
225 - [x] Email header injection prevention (CRLF sanitization).
226 - [x] Signal command injection prevention (phone number validation).
227 - [x] Constant-time signature comparison.
228 - [x] SSRF protection for outbound requests.
229 - [x] Timing attack protection (dummy bcrypt on user-not-found prevents account enumeration).
230## Lewis' fabulous mini-list of remaining TODOs
231- [x] The OAuth authorize POST endpoint has no rate limiting, allowing password brute-forcing. Fix this and audit all oauth and 2fa surface again.
232- [x] DID resolution caching (valkey).
233- [x] Record schema validation (generic validation framework).
234- [x] Fix any remaining TODOs in the code.
235## Future: Web Management UI
236A single-page web app for account management. The frontend (JS framework) calls existing ATProto XRPC endpoints - no server-side rendering or bespoke HTML form handlers.
237### Architecture
238- [x] Static SPA served from PDS (or separate static host)
239- [ ] Frontend authenticates via OAuth 2.1 flow (same as any ATProto client)
240- [x] All operations use standard XRPC endpoints (existing + new PDS-specific ones below)
241- [x] No server-side sessions or CSRF - pure API client
242### PDS-Specific XRPC Endpoints (new)
243Absolutely subject to change, "bspds" isn't even the real name of this pds thus far :D
244Anyway... endpoints for PDS settings not covered by standard ATProto:
245- [x] `com.bspds.account.getNotificationPrefs` - get preferred channel, verified channels
246- [x] `com.bspds.account.updateNotificationPrefs` - set preferred channel
247- [x] `com.bspds.account.getNotificationHistory` - list past notifications
248- [x] `com.bspds.account.verifyChannel` - initiate verification for Discord/Telegram/Signal
249- [x] `com.bspds.account.confirmChannelVerification` - confirm with code
250- [x] `com.bspds.admin.getServerStats` - user count, storage usage, etc.
251### Frontend Views
252Uses existing ATProto endpoints where possible:
253Authentication
254- [x] Login page (uses `com.atproto.server.createSession`)
255- [x] Registration page (uses `com.atproto.server.createAccount`)
256- [x] Signup verification flow (uses `com.atproto.server.confirmSignup`, `resendVerification`)
257- [ ] Password reset flow (uses `com.atproto.server.requestPasswordReset`, `resetPassword`)
258User Dashboard
259- [x] Account overview (uses `com.atproto.server.getSession`, `com.atproto.admin.getAccountInfo`)
260- [ ] Active sessions view (needs new endpoint or extend existing)
261- [x] App passwords (uses `com.atproto.server.listAppPasswords`, `createAppPassword`, `revokeAppPassword`)
262- [x] Invite codes (uses `com.atproto.server.getAccountInviteCodes`, `createInviteCode`)
263Notification Preferences
264- [x] Channel selector (uses `com.bspds.account.*` endpoints above)
265- [x] Verification flows for Discord/Telegram/Signal
266- [ ] Notification history view
267Account Settings
268- [x] Email change (uses `com.atproto.server.requestEmailUpdate`, `updateEmail`)
269- [ ] Password change while logged in (needs new endpoint - change password with current password)
270- [x] Handle change (uses `com.atproto.identity.updateHandle`)
271- [x] Account deletion (uses `com.atproto.server.requestAccountDelete`, `deleteAccount`)
272Data Management
273- [x] Repo browser (browse collections, view/create/delete records via `com.atproto.repo.*`)
274- [ ] Data export/download (CAR file download via `com.atproto.sync.getRepo`)
275Admin Dashboard (privileged users only)
276- [ ] User list (uses `com.atproto.admin.getAccountInfos` with pagination)
277- [ ] User detail/actions (uses `com.atproto.admin.*` endpoints)
278- [ ] Invite management (uses `com.atproto.admin.getInviteCodes`, `disableInviteCodes`)
279- [ ] Server stats (uses `com.bspds.admin.getServerStats`)
280## Future: private data
281I will see where the discourse about encrypted/privileged private data is at the current moment, and make an implementation that matches what the bsky team will likely do in their pds whenever they get around to it.
282Then when they come out with theirs, I can make adjustments to mine and be ready on day 1. Or 2.
283We want records that only authorized parties can see and decrypt. This requires some sort of federation of keys and communication between PDSes?
284Gotta figure all of this out as a first step.