bretton.dev
A community based topic aggregation platform built on atproto
Splitting CLAUDE.md into ATPROTO_GUIDE.md and PROJECT_STRUCTURE.md
+339
ATPROTO_GUIDE.md
+339
ATPROTO_GUIDE.md
···
1
+
# AT Protocol Implementation Guide
2
+
3
+
This guide provides comprehensive information about implementing AT Protocol (atproto) in the Coves platform.
4
+
5
+
## Table of Contents
6
+
- [Core Concepts](#core-concepts)
7
+
- [Architecture Overview](#architecture-overview)
8
+
- [Lexicons](#lexicons)
9
+
- [XRPC](#xrpc)
10
+
- [Data Storage](#data-storage)
11
+
- [Identity & Authentication](#identity--authentication)
12
+
- [Firehose & Sync](#firehose--sync)
13
+
- [Go Implementation Patterns](#go-implementation-patterns)
14
+
- [Best Practices](#best-practices)
15
+
16
+
## Core Concepts
17
+
18
+
### What is AT Protocol?
19
+
AT Protocol is a federated social networking protocol that enables:
20
+
- **Decentralized identity** - Users own their identity (DID) and can move between providers
21
+
- **Data portability** - Users can export and migrate their full social graph and content
22
+
- **Interoperability** - Different apps can interact with the same underlying data
23
+
24
+
### Key Components
25
+
1. **DIDs (Decentralized Identifiers)** - Persistent user identifiers (e.g., `did:plc:xyz123`)
26
+
2. **Handles** - Human-readable names that resolve to DIDs (e.g., `alice.bsky.social`)
27
+
3. **Repositories** - User data stored as signed Merkle trees in CAR files
28
+
4. **Lexicons** - Schema definitions for data types and API methods
29
+
5. **XRPC** - The RPC protocol for client-server communication
30
+
6. **Firehose** - Real-time event stream of repository changes
31
+
32
+
## Architecture Overview
33
+
34
+
### Two-Database Pattern
35
+
AT Protocol requires two distinct data stores:
36
+
37
+
#### 1. Repository Database (Source of Truth)
38
+
- **Purpose**: Stores user-generated content as immutable, signed records
39
+
- **Storage**: CAR files containing Merkle trees + PostgreSQL metadata
40
+
- **Access**: Through XRPC procedures that modify repositories
41
+
- **Properties**:
42
+
- Append-only (soft deletes via tombstones)
43
+
- Cryptographically verifiable
44
+
- User-controlled and portable
45
+
46
+
#### 2. AppView Database (Query Layer)
47
+
- **Purpose**: Denormalized, indexed data optimized for queries
48
+
- **Storage**: PostgreSQL with application-specific schema
49
+
- **Access**: Through XRPC queries (read-only)
50
+
- **Properties**:
51
+
- Eventually consistent with repositories
52
+
- Can be rebuilt from repository data
53
+
- Application-specific aggregations
54
+
55
+
### Data Flow
56
+
57
+
```
58
+
Write Path:
59
+
Client → XRPC Procedure → Service → Write Repo → CAR Store
60
+
↓
61
+
Firehose Event
62
+
↓
63
+
AppView Indexer
64
+
↓
65
+
AppView Database
66
+
67
+
Read Path:
68
+
Client → XRPC Query → Service → Read Repo → AppView Database
69
+
```
70
+
71
+
## Lexicons
72
+
73
+
### What are Lexicons?
74
+
Lexicons are JSON schema files that define:
75
+
- Data types (records stored in repositories)
76
+
- API methods (queries and procedures)
77
+
- Input/output schemas for API calls
78
+
79
+
### Lexicon Structure
80
+
```json
81
+
{
82
+
"lexicon": 1,
83
+
"id": "social.coves.community.profile",
84
+
"defs": {
85
+
"main": {
86
+
"type": "record",
87
+
"key": "self",
88
+
"record": {
89
+
"type": "object",
90
+
"required": ["name", "createdAt"],
91
+
"properties": {
92
+
"name": {"type": "string", "maxLength": 64},
93
+
"description": {"type": "string", "maxLength": 256},
94
+
"rules": {"type": "array", "items": {"type": "string"}},
95
+
"createdAt": {"type": "string", "format": "datetime"}
96
+
}
97
+
}
98
+
}
99
+
}
100
+
}
101
+
```
102
+
103
+
### Lexicon Types
104
+
105
+
#### 1. Record Types
106
+
Define data structures stored in user repositories:
107
+
```json
108
+
{
109
+
"type": "record",
110
+
"key": "tid|rkey|literal",
111
+
"record": { /* schema */ }
112
+
}
113
+
```
114
+
115
+
#### 2. Query Types (Read-only)
116
+
Define read operations that don't modify state:
117
+
```json
118
+
{
119
+
"type": "query",
120
+
"parameters": { /* input schema */ },
121
+
"output": { /* response schema */ }
122
+
}
123
+
```
124
+
125
+
#### 3. Procedure Types (Write)
126
+
Define operations that modify repositories:
127
+
```json
128
+
{
129
+
"type": "procedure",
130
+
"input": { /* request body schema */ },
131
+
"output": { /* response schema */ }
132
+
}
133
+
```
134
+
135
+
### Naming Conventions
136
+
- Use reverse-DNS format: `social.coves.community.profile`
137
+
- Queries often start with `get`, `list`, or `search`
138
+
- Procedures often start with `create`, `update`, or `delete`
139
+
- Keep names descriptive but concise
140
+
141
+
## XRPC
142
+
143
+
### What is XRPC?
144
+
XRPC (Cross-Protocol RPC) is AT Protocol's HTTP-based RPC system:
145
+
- All methods live under `/xrpc/` path
146
+
- Method names map directly to Lexicon IDs
147
+
- Supports both JSON and binary data
148
+
149
+
### Request Format
150
+
```
151
+
# Query (GET)
152
+
GET /xrpc/social.coves.community.getCommunity?id=123
153
+
154
+
# Procedure (POST)
155
+
POST /xrpc/social.coves.community.createPost
156
+
Content-Type: application/json
157
+
Authorization: Bearer <token>
158
+
159
+
{"text": "Hello, Coves!"}
160
+
```
161
+
162
+
### Authentication
163
+
- Uses Bearer tokens in Authorization header
164
+
- Tokens are JWTs signed by the user's signing key
165
+
- Service auth for server-to-server calls
166
+
167
+
## Data Storage
168
+
169
+
### CAR Files
170
+
Content Addressable archive files store repository data:
171
+
- Contains IPLD blocks forming a Merkle tree
172
+
- Each block identified by CID (Content IDentifier)
173
+
- Enables cryptographic verification and efficient sync
174
+
175
+
### Record Keys (rkeys)
176
+
- Unique identifiers for records within a collection
177
+
- Can be TIDs (timestamp-based) or custom strings
178
+
- Must match pattern: `[a-zA-Z0-9._~-]{1,512}`
179
+
180
+
### Repository Structure
181
+
```
182
+
Repository (did:plc:user123)
183
+
├── social.coves.post
184
+
│ ├── 3kkreaz3amd27 (TID)
185
+
│ └── 3kkreaz3amd28 (TID)
186
+
├── social.coves.community.member
187
+
│ ├── community123
188
+
│ └── community456
189
+
└── app.bsky.actor.profile
190
+
└── self
191
+
```
192
+
193
+
## Identity & Authentication
194
+
195
+
### DIDs (Decentralized Identifiers)
196
+
- Permanent, unique identifiers for users
197
+
- Two types supported:
198
+
- `did:plc:*` - Hosted by PLC Directory
199
+
- `did:web:*` - Self-hosted
200
+
201
+
### Handle Resolution
202
+
Handles resolve to DIDs via:
203
+
1. DNS TXT record: `_atproto.alice.com → did:plc:xyz`
204
+
2. HTTPS well-known: `https://alice.com/.well-known/atproto-did`
205
+
206
+
### Authentication Flow
207
+
1. Client creates session with identifier/password
208
+
2. Server returns access/refresh tokens
209
+
3. Client uses access token for API requests
210
+
4. Refresh when access token expires
211
+
212
+
## Firehose & Sync
213
+
214
+
### Firehose Events
215
+
Real-time stream of repository changes:
216
+
- Commit events (creates, updates, deletes)
217
+
- Identity events (handle changes)
218
+
- Account events (status changes)
219
+
220
+
### Subscribing to Firehose
221
+
Connect via WebSocket to `com.atproto.sync.subscribeRepos`:
222
+
```
223
+
wss://bsky.network/xrpc/com.atproto.sync.subscribeRepos
224
+
```
225
+
226
+
### Processing Events
227
+
- Events include full record data and operation type
228
+
- Process events to update AppView database
229
+
- Handle out-of-order events with sequence numbers
230
+
231
+
## Go Implementation Patterns
232
+
233
+
### Using Indigo Library
234
+
Bluesky's official Go implementation provides:
235
+
- Lexicon code generation
236
+
- CAR file handling
237
+
- XRPC client/server
238
+
- Firehose subscription
239
+
240
+
### Code Generation
241
+
Generate Go types from Lexicons:
242
+
```bash
243
+
go run github.com/bluesky-social/indigo/cmd/lexgen \
244
+
--package coves \
245
+
--prefix social.coves \
246
+
--outdir api/coves \
247
+
lexicons/social/coves/*.json
248
+
```
249
+
250
+
### Repository Operations
251
+
```go
252
+
// Write to repository
253
+
rkey := models.GenerateTID()
254
+
err := repoStore.CreateRecord(ctx, userDID, "social.coves.post", rkey, &Post{
255
+
Text: "Hello",
256
+
CreatedAt: time.Now().Format(time.RFC3339),
257
+
})
258
+
259
+
// Read from repository
260
+
records, err := repoStore.ListRecords(ctx, userDID, "social.coves.post", limit, cursor)
261
+
```
262
+
263
+
### XRPC Handler Pattern
264
+
```go
265
+
func (s *Server) HandleGetCommunity(ctx context.Context) error {
266
+
// 1. Parse and validate input
267
+
id := xrpc.QueryParam(ctx, "id")
268
+
269
+
// 2. Call service layer
270
+
community, err := s.communityService.GetByID(ctx, id)
271
+
if err != nil {
272
+
return err
273
+
}
274
+
275
+
// 3. Return response
276
+
return xrpc.WriteJSONResponse(ctx, community)
277
+
}
278
+
```
279
+
280
+
## Best Practices
281
+
282
+
### 1. Lexicon Design
283
+
- Keep schemas focused and single-purpose
284
+
- Use references (`$ref`) for shared types
285
+
- Version carefully - Lexicons are contracts
286
+
- Document thoroughly with descriptions
287
+
288
+
### 2. Data Modeling
289
+
- Store minimal data in repositories
290
+
- Denormalize extensively in AppView
291
+
- Use record keys that are meaningful
292
+
- Plan for data portability
293
+
294
+
### 3. Performance
295
+
- Batch firehose processing
296
+
- Use database transactions wisely
297
+
- Index AppView tables appropriately
298
+
- Cache frequently accessed data
299
+
300
+
### 4. Error Handling
301
+
- Use standard XRPC error codes
302
+
- Provide meaningful error messages
303
+
- Handle network failures gracefully
304
+
- Implement proper retry logic
305
+
306
+
### 5. Security
307
+
- Validate all inputs against Lexicons
308
+
- Verify signatures on repository data
309
+
- Rate limit API endpoints
310
+
- Sanitize user-generated content
311
+
312
+
### 6. Federation
313
+
- Design for multi-instance deployment
314
+
- Handle remote user identities
315
+
- Respect instance-specific policies
316
+
- Plan for cross-instance data sync
317
+
318
+
## Common Patterns
319
+
320
+
### Handling User Content
321
+
- Always validate against Lexicon schemas
322
+
- Store in user's repository via CAR files
323
+
- Index in AppView for efficient queries
324
+
- Emit firehose events for subscribers
325
+
326
+
## Resources
327
+
328
+
### Official Documentation
329
+
- [ATProto Specifications](https://atproto.com/specs)
330
+
- [Lexicon Documentation](https://atproto.com/specs/lexicon)
331
+
- [XRPC Specification](https://atproto.com/specs/xrpc)
332
+
333
+
### Reference Implementations
334
+
- [Indigo (Go)](https://github.com/bluesky-social/indigo)
335
+
- [ATProto SDK (TypeScript)](https://github.com/bluesky-social/atproto)
336
+
337
+
### Tools
338
+
- [Lexicon CLI](https://github.com/bluesky-social/atproto/tree/main/packages/lex-cli)
339
+
- [goat CLI](https://github.com/bluesky-social/indigo/tree/main/cmd/goat)
+51
-75
CLAUDE.md
+51
-75
CLAUDE.md
···
1
1
Project:
2
-
You are a distinguished developer helping build Coves, a forum like atProto social media platform (think reddit).
2
+
You are a distinguished developer helping build Coves, a forum like atProto social media platform (think reddit / lemmy).
3
3
4
4
Human & LLM Readability Guidelines:
5
5
- Clear Module Boundaries: Each feature is a self-contained module with explicit interfaces
···
19
19
- DB: PostgreSQL
20
20
- atProto for federation & user identities
21
21
22
-
## atProto Guidelines
23
-
- Attempt to utilize bsky built indigo packages before building atProto layer functions from scratch
22
+
## atProto Guidelines
23
+
24
+
For comprehensive AT Protocol implementation details, see [ATPROTO_GUIDE.md](./ATPROTO_GUIDE.md).
25
+
26
+
Key principles:
27
+
- Utilize Bluesky's Indigo packages before building custom atProto functionality
28
+
- Everything is XRPC - no separate REST API layer needed
29
+
- Follow the two-database pattern: Repository (CAR files) and AppView (PostgreSQL)
30
+
- Design for federation and data portability from the start
24
31
25
32
# Architecture Guidelines
26
33
27
34
## Required Layered Architecture
28
35
Follow this strict separation of concerns:
29
36
```
30
-
Handler (HTTP) → Service (Business Logic) → Repository (Data Access) → Database
37
+
Handler (XRPC) → Service (Business Logic) → Repository (Data Access) → Database
31
38
```
39
+
- Handlers: XRPC request/response only
40
+
- Services: Business logic, uses both write/read repos
41
+
- Write Repos: CAR store operations
42
+
- Read Repos: AppView queries
43
+
32
44
33
45
## Directory Structure
34
-
```
35
-
internal/
36
-
├── api/
37
-
│ ├── handlers/ # HTTP request/response handling ONLY
38
-
│ └── routes/ # Route definitions
39
-
├── core/
40
-
│ └── [domain]/ # Business logic, domain models, service interfaces
41
-
│ ├── service.go # Business logic implementation
42
-
│ ├── repository.go # Data access interface
43
-
│ └── [domain].go # Domain models
44
-
└── db/
45
-
└── postgres/ # Database implementation details
46
-
└── [domain]_repo.go # Repository implementations
47
-
```
46
+
47
+
For a detailed project structure with file-level details and implementation status, see [PROJECT_STRUCTURE.md](./PROJECT_STRUCTURE.md).
48
+
49
+
The project follows a layered architecture with clear separation between:
50
+
- **XRPC handlers** - atProto API layer
51
+
- Only handle XRPC concerns: parsing requests, formatting responses
52
+
- Delegate all business logic to services
53
+
- No direct database access
54
+
- **Core business logic** - Domain services and models
55
+
- Contains all business logic
56
+
- Orchestrates between write and read repositories
57
+
- Manages transactions and complex operations
58
+
- **Data repositories** - Split between CAR store writes and AppView reads
59
+
- **Write Repositories** (`internal/atproto/carstore/*_write_repo.go`)
60
+
- Modify CAR files (source of truth)
61
+
- **Read Repositories** (`db/appview/*_read_repo.go`)
62
+
- Query denormalized PostgreSQL tables
63
+
- Optimized for performance
48
64
49
65
## Strict Prohibitions
50
66
- **NEVER** put SQL queries in handlers
51
67
- **NEVER** import database packages in handlers
52
68
- **NEVER** pass *sql.DB directly to handlers
53
-
- **NEVER** mix business logic with HTTP concerns
69
+
- **NEVER** mix business logic with XRPC concerns
54
70
- **NEVER** bypass the service layer
55
71
56
-
## Required Patterns
57
-
58
-
### Handlers (HTTP Layer)
59
-
- Only handle HTTP concerns: parsing requests, formatting responses
60
-
- Delegate all business logic to services
61
-
- No direct database access
62
-
63
-
Example:
64
-
```go
65
-
func (h *UserHandler) CreateUser(w http.ResponseWriter, r *http.Request) {
66
-
var req CreateUserRequest
67
-
json.NewDecoder(r.Body).Decode(&req)
68
-
69
-
user, err := h.userService.CreateUser(req) // Delegate to service
70
-
// Handle response formatting only
71
-
}
72
-
```
73
-
74
-
### Services (Business Layer)
75
-
- Contain all business logic and validation
76
-
- Use repository interfaces, never concrete implementations
77
-
- Handle transactions and complex operations
78
-
79
-
Example:
80
-
```go
81
-
type UserService struct {
82
-
userRepo UserRepository // Interface, not concrete type
83
-
}
84
-
```
85
-
86
-
### Repositories (Data Layer)
87
-
- Define interfaces in core/[domain]/
88
-
- Implement in db/postgres/
89
-
- Handle all SQL queries and database operations
90
-
91
-
Example:
92
-
```go
93
-
// Interface in core/users/repository.go
94
-
type UserRepository interface {
95
-
Create(user User) (*User, error)
96
-
GetByID(id int) (*User, error)
97
-
}
98
-
99
-
// Implementation in db/postgres/user_repo.go
100
-
type PostgresUserRepo struct {
101
-
db *sql.DB
102
-
}
103
-
```
104
-
105
72
## Testing Requirements
106
73
- Services must be easily mockable (use interfaces)
107
74
- Integration tests should test the full stack
···
119
86
2. Generate test file with failing tests
120
87
3. Generate implementation to pass tests
121
88
4. Generate handler with tests
122
-
5. Update routes in api/routes/
89
+
5. Update routes in xrpc/routes/
123
90
124
91
### Refactoring Checklist
125
92
Before considering a feature complete:
···
135
102
- Never modify existing migrations
136
103
- Always provide rollback migrations
137
104
138
-
139
-
140
105
## Dependency Injection
141
106
- Use constructor functions for all components
142
107
- Pass interfaces, not concrete types
···
145
110
Example dependency wiring:
146
111
```go
147
112
// main.go
148
-
userRepo := postgres.NewUserRepository(db)
149
-
userService := users.NewUserService(userRepo)
150
-
userHandler := handlers.NewUserHandler(userService)
113
+
userWriteRepo := carstore.NewUserWriteRepository(carStore)
114
+
userReadRepo := appview.NewUserReadRepository(db)
115
+
userService := users.NewUserService(userWriteRepo, userReadRepo)
116
+
userHandler := xrpc.NewUserHandler(userService)
151
117
```
152
118
153
119
## Error Handling
···
157
123
- Never expose internal error details in API responses
158
124
159
125
### Context7 Usage Guidelines:
160
-
- Always check Context7 for best practices before implementing external integrations
126
+
- Always check Context7 for best practices before implementing external integrations and packages
161
127
- Use Context7 to understand proper error handling patterns for specific libraries
162
128
- Reference Context7 for testing patterns with external dependencies
163
129
- Consult Context7 for proper configuration patterns
130
+
131
+
## XRPC Implementation
132
+
133
+
For detailed XRPC patterns and Lexicon examples, see [ATPROTO_GUIDE.md](./ATPROTO_GUIDE.md#xrpc).
134
+
135
+
### Key Points
136
+
- All client interactions go through XRPC endpoints
137
+
- Handlers validate against Lexicon schemas automatically
138
+
- Queries are read-only, procedures modify repositories
139
+
- Every endpoint must have a corresponding Lexicon definition
+76
PROJECT_STRUCTURE.md
+76
PROJECT_STRUCTURE.md
···
1
+
# Coves Project Structure
2
+
3
+
This document provides an overview of the Coves project directory structure, following atProto architecture patterns.
4
+
5
+
**Legend:**
6
+
- † = Planned but not yet implemented
7
+
- 🔒 = Security-sensitive files
8
+
9
+
```
10
+
Coves/
11
+
├── CLAUDE.md # Project guidelines and architecture decisions
12
+
├── ATPROTO_GUIDE.md # Comprehensive AT Protocol implementation guide
13
+
├── PROJECT_STRUCTURE.md # This file - project structure overview
14
+
├── LICENSE # Project license
15
+
├── README.md # Project overview and setup instructions
16
+
├── go.mod # Go module definition
17
+
├── go.sum # Go module checksums
18
+
│
19
+
├── cmd/ # Application entrypoints
20
+
├── internal/ # Private application code
21
+
│ ├── xrpc/ † # XRPC handlers (atProto API layer)
22
+
│ ├── api/ # Traditional HTTP endpoints (minimal)
23
+
│ ├── core/ # Business logic and domain models
24
+
│ ├── atproto/ # atProto-specific implementations
25
+
│ └── config/ † # Configuration management
26
+
│
27
+
├── db/ # Database layer
28
+
│ ├── appview/ † # AppView PostgreSQL queries
29
+
│ ├── postgres/ # Legacy/non-atProto database operations
30
+
│ ├── migrations/ # Database migrations
31
+
│ ├── local_dev_db_compose/ # Local development database
32
+
│ └── test_db_compose/ # Test database setup
33
+
│
34
+
├── pkg/ # Public packages (can be imported by external projects)
35
+
├── data/ # Runtime data storage
36
+
│ └── carstore/ 🔒 # CAR file storage directory
37
+
│
38
+
├── scripts/ # Development and deployment scripts
39
+
├── tests/ # Integration and e2e tests
40
+
├── docs/ † # Additional documentation
41
+
├── local_dev_data/ # Local development data
42
+
├── test_db_data/ # Test database seed data
43
+
└── build/ † # Build artifacts
44
+
```
45
+
46
+
## Implementation Status
47
+
48
+
### Completed ✓
49
+
- Basic repository structure
50
+
- User domain models
51
+
- CAR store foundation
52
+
- Lexicon schemas
53
+
- Database migrations
54
+
55
+
### In Progress 🚧
56
+
- Repository service implementation
57
+
- User service
58
+
- Basic authentication
59
+
60
+
### Planned 📋
61
+
- XRPC handlers
62
+
- AppView indexer
63
+
- Firehose implementation
64
+
- Community features
65
+
- Moderation system
66
+
- Feed algorithms
67
+
68
+
## Development Guidelines
69
+
70
+
For detailed implementation guidelines, see [CLAUDE.md](./CLAUDE.md) and [ATPROTO_GUIDE.md](./ATPROTO_GUIDE.md).
71
+
72
+
1. **Start with Lexicons**: Define data schemas first
73
+
2. **Implement Core Domain**: Create models and interfaces
74
+
3. **Build Services**: Implement business logic
75
+
4. **Add Repositories**: Create data access layers
76
+
5. **Wire XRPC**: Connect handlers last