arabica.social
Coffee journaling on ATProto (alpha)
alpha.arabica.social
coffee
fix: block browser api endpoint access, misc fixes
pdewey.com
+7
-2
BACKLOG.md
+7
-2
BACKLOG.md
···
17
17
- Dev mode -- show did, copy did in profiles (remove "logged in as <did>" from home page)
18
18
- Toggle for table view vs future post-style view
19
19
20
-
## Fixes
20
+
## Far Future Considerations
21
+
22
+
- Consider fully separating API backend from frontend service
23
+
- Currently using HTMX header checks to prevent direct browser access to internal API endpoints
24
+
- If adding mobile apps, third-party API consumers, or microservices architecture, revisit this
25
+
- For now, monolithic approach is appropriate for HTMX-based web app with decentralized storage
21
26
22
-
- Loading columns for brews table doesn't match loaded column names
27
+
## Fixes
+314
docs/jetstream-tap-evaluation.md
+314
docs/jetstream-tap-evaluation.md
···
1
+
# Jetstream and Tap Evaluation for Arabica
2
+
3
+
## Executive Summary
4
+
5
+
This document evaluates two AT Protocol synchronization tools - **Jetstream** and **Tap** - for potential integration with Arabica. These tools could help reduce API requests for the community feed feature and simplify real-time data synchronization.
6
+
7
+
**Recommendation:** Consider Jetstream for community feed improvements in the near term; Tap is overkill for Arabica's current scale but valuable for future growth.
8
+
9
+
---
10
+
11
+
## Background: Current Arabica Architecture
12
+
13
+
Arabica currently interacts with AT Protocol in two ways:
14
+
15
+
1. **Authenticated User Operations** (`internal/atproto/store.go`)
16
+
- Direct XRPC calls to user's PDS for CRUD operations
17
+
- Per-session in-memory cache (5-minute TTL)
18
+
- Each user's data stored in their own PDS
19
+
20
+
2. **Community Feed** (`internal/feed/service.go`)
21
+
- Polls registered users' PDSes to aggregate recent activity
22
+
- Fetches profiles, brews, beans, roasters, grinders, brewers from each user
23
+
- Public feed cached for 5 minutes
24
+
- **Problem:** N+1 query pattern - each registered user requires multiple API calls
25
+
26
+
### Current Feed Inefficiency
27
+
28
+
For N registered users, the feed service makes approximately:
29
+
- N profile fetches
30
+
- N x 5 collection fetches (brew, bean, roaster, grinder, brewer) for recent items
31
+
- N x 4 collection fetches for reference resolution
32
+
- **Total: ~10N API calls per feed refresh**
33
+
34
+
---
35
+
36
+
## Tool 1: Jetstream
37
+
38
+
### What It Is
39
+
40
+
Jetstream is a streaming service that consumes the AT Protocol firehose (`com.atproto.sync.subscribeRepos`) and converts it into lightweight JSON events. It's operated by Bluesky at public endpoints.
41
+
42
+
**Public Instances:**
43
+
- `jetstream1.us-east.bsky.network`
44
+
- `jetstream2.us-east.bsky.network`
45
+
- `jetstream1.us-west.bsky.network`
46
+
- `jetstream2.us-west.bsky.network`
47
+
48
+
### Key Features
49
+
50
+
| Feature | Description |
51
+
|---------|-------------|
52
+
| JSON Output | Simple JSON instead of CBOR/CAR binary encoding |
53
+
| Filtering | Filter by collection (NSID) or repo (DID) |
54
+
| Compression | ~56% smaller messages with zstd compression |
55
+
| Low Latency | Real-time event delivery |
56
+
| Easy to Use | Standard WebSocket connection |
57
+
58
+
### Jetstream Event Example
59
+
60
+
```json
61
+
{
62
+
"did": "did:plc:eygmaihciaxprqvxpfvl6flk",
63
+
"time_us": 1725911162329308,
64
+
"kind": "commit",
65
+
"commit": {
66
+
"rev": "3l3qo2vutsw2b",
67
+
"operation": "create",
68
+
"collection": "social.arabica.alpha.brew",
69
+
"rkey": "3l3qo2vuowo2b",
70
+
"record": {
71
+
"$type": "social.arabica.alpha.brew",
72
+
"method": "pourover",
73
+
"rating": 4,
74
+
"createdAt": "2024-09-09T19:46:02.102Z"
75
+
},
76
+
"cid": "bafyreidwaivazkwu67xztlmuobx35hs2lnfh3kolmgfmucldvhd3sgzcqi"
77
+
}
78
+
}
79
+
```
80
+
81
+
### How Arabica Could Use Jetstream
82
+
83
+
**Use Case: Real-time Community Feed**
84
+
85
+
Instead of polling each user's PDS every 5 minutes, Arabica could:
86
+
87
+
1. Subscribe to Jetstream filtered by:
88
+
- `wantedCollections`: `social.arabica.alpha.*`
89
+
- `wantedDids`: List of registered feed users
90
+
91
+
2. Maintain a local feed index updated in real-time
92
+
93
+
3. Serve feed directly from local index (instant response, no API calls)
94
+
95
+
**Implementation Sketch:**
96
+
97
+
```go
98
+
// Subscribe to Jetstream for Arabica collections
99
+
ws, _ := websocket.Dial("wss://jetstream1.us-east.bsky.network/subscribe?" +
100
+
"wantedCollections=social.arabica.alpha.brew&" +
101
+
"wantedCollections=social.arabica.alpha.bean&" +
102
+
"wantedDids=" + strings.Join(registeredDids, "&wantedDids="))
103
+
104
+
// Process events in background goroutine
105
+
for {
106
+
var event JetstreamEvent
107
+
ws.ReadJSON(&event)
108
+
109
+
switch event.Commit.Collection {
110
+
case "social.arabica.alpha.brew":
111
+
feedIndex.AddBrew(event.DID, event.Commit.Record)
112
+
case "social.arabica.alpha.bean":
113
+
feedIndex.AddBean(event.DID, event.Commit.Record)
114
+
}
115
+
}
116
+
```
117
+
118
+
### Jetstream Tradeoffs
119
+
120
+
| Pros | Cons |
121
+
|------|------|
122
+
| Dramatically reduces API calls | No cryptographic verification of data |
123
+
| Real-time updates (sub-second latency) | Requires persistent WebSocket connection |
124
+
| Simple JSON format | Trust relationship with Jetstream operator |
125
+
| Can filter by collection/DID | Not part of formal AT Protocol spec |
126
+
| Free public instances available | No built-in backfill mechanism |
127
+
128
+
### Jetstream Verdict for Arabica
129
+
130
+
**Recommended for:** Community feed real-time updates
131
+
132
+
**Not suitable for:** Authenticated user operations (those need direct PDS calls)
133
+
134
+
**Effort estimate:** Medium (1-2 weeks)
135
+
- Add WebSocket client for Jetstream
136
+
- Build local feed index (could use BoltDB or in-memory)
137
+
- Handle reconnection/cursor management
138
+
- Still need initial backfill via direct API
139
+
140
+
---
141
+
142
+
## Tool 2: Tap
143
+
144
+
### What It Is
145
+
146
+
Tap is a synchronization tool for AT Protocol that handles the complexity of repo synchronization. It subscribes to a Relay and outputs filtered, verified events. Tap is more comprehensive than Jetstream but requires running your own instance.
147
+
148
+
**Repository:** `github.com/bluesky-social/indigo/cmd/tap`
149
+
150
+
### Key Features
151
+
152
+
| Feature | Description |
153
+
|---------|-------------|
154
+
| Automatic Backfill | Fetches complete history when tracking new repos |
155
+
| Verification | MST integrity checks, signature validation |
156
+
| Recovery | Auto-resyncs if repo becomes desynchronized |
157
+
| Flexible Delivery | WebSocket, fire-and-forget, or webhooks |
158
+
| Filtered Output | DID and collection filtering |
159
+
160
+
### Tap Operating Modes
161
+
162
+
1. **Dynamic (default):** Add DIDs via API as needed
163
+
2. **Collection Signal:** Auto-track repos with records in specified collection
164
+
3. **Full Network:** Mirror entire AT Protocol network (resource-intensive)
165
+
166
+
### How Arabica Could Use Tap
167
+
168
+
**Use Case: Complete Feed Infrastructure**
169
+
170
+
Tap could replace the entire feed polling mechanism:
171
+
172
+
1. Run Tap instance with `TAP_SIGNAL_COLLECTION=social.arabica.alpha.brew`
173
+
2. Tap automatically discovers and tracks users who create brew records
174
+
3. Feed service consumes events from local Tap instance
175
+
4. No manual user registration needed - Tap discovers users automatically
176
+
177
+
**Collection Signal Mode:**
178
+
179
+
```bash
180
+
# Start Tap to auto-track repos with Arabica records
181
+
TAP_SIGNAL_COLLECTION=social.arabica.alpha.brew \
182
+
go run ./cmd/tap --disable-acks=true
183
+
```
184
+
185
+
**Webhook Delivery (Serverless-friendly):**
186
+
187
+
Tap can POST events to an HTTP endpoint, making it compatible with serverless architectures:
188
+
189
+
```bash
190
+
# Tap sends events to Arabica webhook
191
+
TAP_WEBHOOK_URL=https://arabica.example/api/feed-webhook \
192
+
go run ./cmd/tap
193
+
```
194
+
195
+
### Tap Tradeoffs
196
+
197
+
| Pros | Cons |
198
+
|------|------|
199
+
| Automatic backfill when adding repos | Requires running your own service |
200
+
| Full cryptographic verification | More operational complexity |
201
+
| Handles cursor management | Resource requirements (DB, network) |
202
+
| Auto-discovers users via collection signal | Overkill for small user bases |
203
+
| Webhook support for serverless | Still in beta |
204
+
205
+
### Tap Verdict for Arabica
206
+
207
+
**Recommended for:** Future growth when feed has many users
208
+
209
+
**Not suitable for:** Current scale (< 100 registered users)
210
+
211
+
**Effort estimate:** High (2-4 weeks)
212
+
- Deploy and operate Tap service
213
+
- Integrate webhook or WebSocket consumer
214
+
- Migrate feed service to consume from Tap
215
+
- Handle Tap service reliability/monitoring
216
+
217
+
---
218
+
219
+
## Comparison Matrix
220
+
221
+
| Aspect | Current Polling | Jetstream | Tap |
222
+
|--------|----------------|-----------|-----|
223
+
| API Calls per Refresh | ~10N | 0 (after connection) | 0 (after backfill) |
224
+
| Latency | 5 min cache | Real-time | Real-time |
225
+
| Backfill | Full fetch each time | Manual | Automatic |
226
+
| Verification | Trusts PDS | Trusts Jetstream | Full verification |
227
+
| Operational Cost | None | None (public) | Run own service |
228
+
| Complexity | Low | Medium | High |
229
+
| User Discovery | Manual registry | Manual | Auto via collection |
230
+
| Recommended Scale | < 50 users | 50-1000 users | 1000+ users |
231
+
232
+
---
233
+
234
+
## Recommendation
235
+
236
+
### Short Term (Now - 6 months)
237
+
238
+
**Stick with current polling + caching approach**
239
+
240
+
Rationale:
241
+
- Current implementation works
242
+
- User base is small
243
+
- Polling N users with caching is acceptable
244
+
245
+
**Consider adding Jetstream for feed** if:
246
+
- Feed latency becomes user-visible issue
247
+
- Registered users exceed ~50
248
+
- API rate limiting becomes a problem
249
+
250
+
### Medium Term (6-12 months)
251
+
252
+
**Implement Jetstream integration**
253
+
254
+
1. Add background Jetstream consumer
255
+
2. Build local feed index (BoltDB or SQLite)
256
+
3. Serve feed from local index
257
+
4. Keep polling as fallback for backfill
258
+
259
+
### Long Term (12+ months)
260
+
261
+
**Evaluate Tap when:**
262
+
- User base exceeds 500+ registered users
263
+
- Want automatic user discovery
264
+
- Need cryptographic verification for social features (likes, comments)
265
+
- Building moderation/anti-abuse features
266
+
267
+
---
268
+
269
+
## Implementation Notes
270
+
271
+
### Jetstream Client Library
272
+
273
+
Bluesky provides a Go client library:
274
+
275
+
```go
276
+
import "github.com/bluesky-social/jetstream/pkg/client"
277
+
```
278
+
279
+
### Tap TypeScript Library
280
+
281
+
For frontend integration:
282
+
283
+
```typescript
284
+
import { TapClient } from '@atproto/tap';
285
+
```
286
+
287
+
### Connection Resilience
288
+
289
+
Both tools require handling:
290
+
- WebSocket reconnection
291
+
- Cursor persistence across restarts
292
+
- Backpressure when events arrive faster than processing
293
+
294
+
### Caching Integration
295
+
296
+
Can coexist with current `SessionCache`:
297
+
- Jetstream/Tap updates the local index
298
+
- Local index serves feed requests
299
+
- SessionCache continues for authenticated user operations
300
+
301
+
---
302
+
303
+
## Related Documentation
304
+
305
+
- Jetstream GitHub: https://github.com/bluesky-social/jetstream
306
+
- Tap README: https://github.com/bluesky-social/indigo/blob/main/cmd/tap/README.md
307
+
- Jetstream Blog Post: https://docs.bsky.app/blog/jetstream
308
+
- Tap Blog Post: https://docs.bsky.app/blog/introducing-tap
309
+
310
+
---
311
+
312
+
## Note on "Constellation" and "Slingshot"
313
+
314
+
These terms don't appear to correspond to official AT Protocol tools as of this evaluation. If these refer to specific community projects or internal codenames, please provide additional context for evaluation.
+505
docs/microcosm-tools-evaluation.md
+505
docs/microcosm-tools-evaluation.md
···
1
+
# Microcosm Tools Evaluation for Arabica
2
+
3
+
## Executive Summary
4
+
5
+
This document evaluates three community-built AT Protocol infrastructure tools from [microcosm.blue](https://microcosm.blue/) - **Constellation**, **Spacedust**, and **Slingshot** - for potential integration with Arabica's community feed feature.
6
+
7
+
**Recommendation:** Adopt Constellation immediately for future social features (likes/comments). Consider Slingshot as an optional optimization for feed performance. Spacedust is ideal for real-time notifications when social features are implemented.
8
+
9
+
---
10
+
11
+
## Background: Current Arabica Architecture
12
+
13
+
### The Problem
14
+
15
+
Arabica's community feed (`internal/feed/service.go`) currently polls each registered user's PDS directly. For N registered users:
16
+
17
+
| API Call Type | Count per Refresh |
18
+
|---------------|-------------------|
19
+
| Profile fetches | N |
20
+
| Brew collections | N |
21
+
| Bean collections | N |
22
+
| Roaster collections | N |
23
+
| Grinder collections | N |
24
+
| Brewer collections | N |
25
+
| Reference resolution | ~4N |
26
+
| **Total** | **~10N API calls** |
27
+
28
+
This approach has several issues:
29
+
- **Latency**: Feed refresh is slow with many users
30
+
- **Rate limits**: Risk of PDS rate limiting
31
+
- **Reliability**: Feed fails if any PDS is slow/down
32
+
- **Scalability**: Linear growth in API calls per user
33
+
34
+
### Future Social Features
35
+
36
+
Arabica plans to add likes, comments, and follows (see `AGENTS.md`). These interactions require **backlink queries** - given a brew, find all likes pointing at it. This is impossible with current polling approach.
37
+
38
+
---
39
+
40
+
## Tool 1: Constellation (Backlink Index)
41
+
42
+
### What It Is
43
+
44
+
Constellation is a **global backlink index** that crawls every record in the AT Protocol firehose and indexes all links (AT-URIs, DIDs, URLs). It answers "who/what points at this target?" queries.
45
+
46
+
**Public Instance:** `https://constellation.microcosm.blue`
47
+
48
+
### Key Capabilities
49
+
50
+
| Feature | Description |
51
+
|---------|-------------|
52
+
| Backlink queries | Find all records linking to a target |
53
+
| Like/follow counts | Get interaction counts instantly |
54
+
| Any lexicon support | Works with `social.arabica.alpha.*` |
55
+
| DID filtering | Filter links by specific users |
56
+
| Distinct DID counts | Count unique users, not just records |
57
+
58
+
### API Examples
59
+
60
+
**Get like count for a brew:**
61
+
```bash
62
+
curl "https://constellation.microcosm.blue/links/count/distinct-dids" \
63
+
-G --data-urlencode "target=at://did:plc:xxx/social.arabica.alpha.brew/abc123" \
64
+
--data-urlencode "collection=social.arabica.alpha.like" \
65
+
--data-urlencode "path=.subject.uri"
66
+
```
67
+
68
+
**Get all users who liked a brew:**
69
+
```bash
70
+
curl "https://constellation.microcosm.blue/links/distinct-dids" \
71
+
-G --data-urlencode "target=at://did:plc:xxx/social.arabica.alpha.brew/abc123" \
72
+
--data-urlencode "collection=social.arabica.alpha.like" \
73
+
--data-urlencode "path=.subject.uri"
74
+
```
75
+
76
+
**Get all comments on a brew:**
77
+
```bash
78
+
curl "https://constellation.microcosm.blue/links" \
79
+
-G --data-urlencode "target=at://did:plc:xxx/social.arabica.alpha.brew/abc123" \
80
+
--data-urlencode "collection=social.arabica.alpha.comment" \
81
+
--data-urlencode "path=.subject.uri"
82
+
```
83
+
84
+
### How Arabica Could Use Constellation
85
+
86
+
**Use Case 1: Social Interaction Counts**
87
+
88
+
When displaying a brew in the feed, fetch interaction counts:
89
+
90
+
```go
91
+
// Get like count for a brew
92
+
func (c *ConstellationClient) GetLikeCount(ctx context.Context, brewURI string) (int, error) {
93
+
url := fmt.Sprintf("%s/links/count/distinct-dids?target=%s&collection=%s&path=%s",
94
+
c.baseURL,
95
+
url.QueryEscape(brewURI),
96
+
"social.arabica.alpha.like",
97
+
url.QueryEscape(".subject.uri"))
98
+
99
+
// Returns {"total": 42}
100
+
var result struct { Total int `json:"total"` }
101
+
// ... fetch and decode
102
+
return result.Total, nil
103
+
}
104
+
```
105
+
106
+
**Use Case 2: Comment Threads**
107
+
108
+
Fetch all comments for a brew detail page:
109
+
110
+
```go
111
+
func (c *ConstellationClient) GetComments(ctx context.Context, brewURI string) ([]Comment, error) {
112
+
// Constellation returns the AT-URIs of comment records
113
+
// Then fetch each comment from Slingshot or user's PDS
114
+
}
115
+
```
116
+
117
+
**Use Case 3: "Who liked this" List**
118
+
119
+
```go
120
+
func (c *ConstellationClient) GetLikers(ctx context.Context, brewURI string) ([]string, error) {
121
+
// Returns list of DIDs who liked this brew
122
+
// Can hydrate with profile info from Slingshot
123
+
}
124
+
```
125
+
126
+
### Constellation Tradeoffs
127
+
128
+
| Pros | Cons |
129
+
|------|------|
130
+
| Instant interaction counts (no polling) | Third-party dependency |
131
+
| Works with any lexicon including Arabica's | Not self-hosted (yet) |
132
+
| Handles likes from any PDS globally | Slight index delay (~seconds) |
133
+
| 11B+ links indexed, production-ready | Trusts Constellation operator |
134
+
| Free public instance | Query limits may apply |
135
+
136
+
### Constellation Verdict
137
+
138
+
**Essential for:** Social features (likes, comments, follows)
139
+
140
+
**Not needed for:** Current feed polling (Constellation indexes interactions, not record listings)
141
+
142
+
**Effort estimate:** Low (1 week)
143
+
- Add HTTP client for Constellation API
144
+
- Integrate counts into brew display
145
+
- Cache counts locally (5-minute TTL)
146
+
147
+
---
148
+
149
+
## Tool 2: Spacedust (Interactions Firehose)
150
+
151
+
### What It Is
152
+
153
+
Spacedust extracts **links** from every record in the AT Protocol firehose and re-emits them over WebSocket. Unlike Jetstream (which emits full records), Spacedust emits just the link relationships.
154
+
155
+
**Public Instance:** `wss://spacedust.microcosm.blue`
156
+
157
+
### Key Capabilities
158
+
159
+
| Feature | Description |
160
+
|---------|-------------|
161
+
| Real-time link events | Instantly know when someone likes/follows |
162
+
| Filter by source/target | Subscribe to specific collections or targets |
163
+
| Any lexicon support | Works with `social.arabica.alpha.*` |
164
+
| Lightweight | Just links, not full records |
165
+
166
+
### Example: Subscribe to Likes on Your Brews
167
+
168
+
```javascript
169
+
// WebSocket connection to Spacedust
170
+
const ws = new WebSocket(
171
+
"wss://spacedust.microcosm.blue/subscribe" +
172
+
"?wantedSources=social.arabica.alpha.like:subject.uri" +
173
+
"&wantedSubjects=did:plc:your-did"
174
+
);
175
+
176
+
ws.onmessage = (event) => {
177
+
const link = JSON.parse(event.data);
178
+
// { source: "at://...", target: "at://...", ... }
179
+
console.log("Someone liked your brew!");
180
+
};
181
+
```
182
+
183
+
### How Arabica Could Use Spacedust
184
+
185
+
**Use Case: Real-time Notifications**
186
+
187
+
When social features are added, Spacedust enables instant notifications:
188
+
189
+
```go
190
+
// Background goroutine subscribes to Spacedust
191
+
func (s *NotificationService) subscribeToInteractions(userDID string) {
192
+
ws := dial("wss://spacedust.microcosm.blue/subscribe" +
193
+
"?wantedSources=social.arabica.alpha.like:subject.uri" +
194
+
"&wantedSubjects=" + userDID)
195
+
196
+
for {
197
+
link := readLink(ws)
198
+
// Someone liked a brew by userDID
199
+
s.notify(userDID, "Someone liked your brew!")
200
+
}
201
+
}
202
+
```
203
+
204
+
**Use Case: Live Feed Updates**
205
+
206
+
Push new brews to connected clients without polling:
207
+
208
+
```go
209
+
// Subscribe to all Arabica brew creations
210
+
ws := dial("wss://spacedust.microcosm.blue/subscribe" +
211
+
"?wantedSources=social.arabica.alpha.brew:beanRef")
212
+
213
+
// When a link event arrives, a new brew was created
214
+
// Fetch full record from Slingshot and push to feed
215
+
```
216
+
217
+
### Spacedust Tradeoffs
218
+
219
+
| Pros | Cons |
220
+
|------|------|
221
+
| Real-time, sub-second latency | Requires persistent WebSocket |
222
+
| Lightweight link-only events | Still in v0 (missing some features) |
223
+
| Filter by collection/target | No cursor replay yet |
224
+
| Perfect for notifications | Need to hydrate records separately |
225
+
226
+
### Spacedust Verdict
227
+
228
+
**Ideal for:** Real-time notifications, live feed updates
229
+
230
+
**Not suitable for:** Current feed needs (need full records, not just links)
231
+
232
+
**Effort estimate:** Medium (2-3 weeks)
233
+
- WebSocket client with reconnection
234
+
- Notification service for social interactions
235
+
- Integration with frontend for live updates
236
+
- Depends on social features being implemented first
237
+
238
+
---
239
+
240
+
## Tool 3: Slingshot (Records & Identities Cache)
241
+
242
+
### What It Is
243
+
244
+
Slingshot is an **edge cache** for AT Protocol records and identities. It pre-caches records from the firehose and provides fast, authenticated access. Also resolves handles to DIDs with bi-directional verification.
245
+
246
+
**Public Instance:** `https://slingshot.microcosm.blue`
247
+
248
+
### Key Capabilities
249
+
250
+
| Feature | Description |
251
+
|---------|-------------|
252
+
| Fast record fetching | Pre-cached from firehose |
253
+
| Identity resolution | `resolveMiniDoc` for handle/DID |
254
+
| Bi-directional verification | Only returns verified handles |
255
+
| Works with slow PDS | Cache serves even if PDS is down |
256
+
| Standard XRPC API | Drop-in replacement for PDS calls |
257
+
258
+
### API Examples
259
+
260
+
**Resolve identity:**
261
+
```bash
262
+
curl "https://slingshot.microcosm.blue/xrpc/com.bad-example.identity.resolveMiniDoc?identifier=bad-example.com"
263
+
# Returns: { "did": "did:plc:...", "handle": "bad-example.com", "pds": "https://..." }
264
+
```
265
+
266
+
**Get record (standard XRPC):**
267
+
```bash
268
+
curl "https://slingshot.microcosm.blue/xrpc/com.atproto.repo.getRecord?repo=did:plc:xxx&collection=social.arabica.alpha.brew&rkey=abc123"
269
+
```
270
+
271
+
**List records:**
272
+
```bash
273
+
curl "https://slingshot.microcosm.blue/xrpc/com.atproto.repo.listRecords?repo=did:plc:xxx&collection=social.arabica.alpha.brew&limit=10"
274
+
```
275
+
276
+
### How Arabica Could Use Slingshot
277
+
278
+
**Use Case 1: Faster Feed Fetching**
279
+
280
+
Replace direct PDS calls with Slingshot for public data:
281
+
282
+
```go
283
+
// Before: Each user's PDS
284
+
pdsEndpoint, _ := c.GetPDSEndpoint(ctx, did)
285
+
url := fmt.Sprintf("%s/xrpc/com.atproto.repo.listRecords...", pdsEndpoint)
286
+
287
+
// After: Single Slingshot endpoint
288
+
url := fmt.Sprintf("https://slingshot.microcosm.blue/xrpc/com.atproto.repo.listRecords...")
289
+
```
290
+
291
+
**Benefits:**
292
+
- Eliminates N DNS lookups for N user PDS endpoints
293
+
- Single, fast endpoint for all public record fetches
294
+
- Continues working even if individual PDS is slow/down
295
+
- Pre-cached records = faster response times
296
+
297
+
**Use Case 2: Identity Resolution**
298
+
299
+
Replace multiple API calls with single `resolveMiniDoc`:
300
+
301
+
```go
302
+
// Before: Two calls
303
+
handle := resolveHandle(did) // Call 1
304
+
pds := resolvePDSEndpoint(did) // Call 2
305
+
306
+
// After: One call
307
+
mini := resolveMiniDoc(did)
308
+
// { handle: "user.bsky.social", pds: "https://...", did: "did:plc:..." }
309
+
```
310
+
311
+
**Use Case 3: Hydrate Records from Constellation**
312
+
313
+
When Constellation returns AT-URIs (e.g., comments on a brew), fetch the actual records from Slingshot:
314
+
315
+
```go
316
+
// Constellation returns: ["at://did:plc:a/social.arabica.alpha.comment/123", ...]
317
+
commentURIs := constellation.GetComments(ctx, brewURI)
318
+
319
+
// Fetch each comment record from Slingshot
320
+
for _, uri := range commentURIs {
321
+
record := slingshot.GetRecord(ctx, uri)
322
+
// ...
323
+
}
324
+
```
325
+
326
+
### Implementation: Slingshot-Backed PublicClient
327
+
328
+
```go
329
+
// internal/atproto/slingshot_client.go
330
+
331
+
const SlingshotBaseURL = "https://slingshot.microcosm.blue"
332
+
333
+
type SlingshotClient struct {
334
+
baseURL string
335
+
httpClient *http.Client
336
+
}
337
+
338
+
func NewSlingshotClient() *SlingshotClient {
339
+
return &SlingshotClient{
340
+
baseURL: SlingshotBaseURL,
341
+
httpClient: &http.Client{Timeout: 10 * time.Second},
342
+
}
343
+
}
344
+
345
+
// ListRecords uses Slingshot instead of user's PDS
346
+
func (c *SlingshotClient) ListRecords(ctx context.Context, did, collection string, limit int) (*PublicListRecordsOutput, error) {
347
+
// Same XRPC API, different endpoint
348
+
url := fmt.Sprintf("%s/xrpc/com.atproto.repo.listRecords?repo=%s&collection=%s&limit=%d&reverse=true",
349
+
c.baseURL, url.QueryEscape(did), url.QueryEscape(collection), limit)
350
+
// ... standard HTTP request
351
+
}
352
+
353
+
// ResolveMiniDoc gets handle + PDS in one call
354
+
func (c *SlingshotClient) ResolveMiniDoc(ctx context.Context, identifier string) (*MiniDoc, error) {
355
+
url := fmt.Sprintf("%s/xrpc/com.bad-example.identity.resolveMiniDoc?identifier=%s",
356
+
c.baseURL, url.QueryEscape(identifier))
357
+
// ... returns { did, handle, pds }
358
+
}
359
+
```
360
+
361
+
### Slingshot Tradeoffs
362
+
363
+
| Pros | Cons |
364
+
|------|------|
365
+
| Faster than direct PDS calls | Third-party dependency |
366
+
| Single endpoint for all users | May not have custom lexicons cached |
367
+
| Identity verification built-in | Not all XRPC APIs implemented |
368
+
| Resilient to slow/down PDS | Trusts Slingshot operator |
369
+
| Pre-cached from firehose | Still in v0, some features missing |
370
+
371
+
### Slingshot Verdict
372
+
373
+
**Recommended for:** Feed performance optimization, identity resolution
374
+
375
+
**Not suitable for:** Authenticated user operations (still need direct PDS)
376
+
377
+
**Effort estimate:** Low (3-5 days)
378
+
- Add SlingshotClient as optional PublicClient backend
379
+
- Feature flag to toggle between direct PDS and Slingshot
380
+
- Test with Arabica collections to ensure they're indexed
381
+
382
+
---
383
+
384
+
## Comparison: Current vs. Microcosm Tools
385
+
386
+
| Aspect | Current Polling | + Slingshot | + Constellation | + Spacedust |
387
+
|--------|-----------------|-------------|-----------------|-------------|
388
+
| Feed refresh latency | Slow (N PDS calls) | Fast (1 endpoint) | N/A | Real-time |
389
+
| Like/comment counts | Impossible | Impossible | Instant | N/A |
390
+
| Rate limit risk | High | Low | Low | None |
391
+
| PDS failure resilience | Poor | Good | N/A | N/A |
392
+
| Real-time updates | No (5min cache) | No | No | Yes |
393
+
| Effort to integrate | N/A | Low | Low | Medium |
394
+
395
+
---
396
+
397
+
## Recommendation
398
+
399
+
### Immediate (Social Features Prerequisite)
400
+
401
+
**1. Integrate Constellation when adding likes/comments**
402
+
403
+
Constellation is essential for social features. When a brew is displayed, use Constellation to:
404
+
- Show like count
405
+
- Show comment count
406
+
- Power "who liked this" lists
407
+
- Power comment threads
408
+
409
+
**Implementation priority:** Do this alongside `social.arabica.alpha.like` and `social.arabica.alpha.comment` lexicon implementation.
410
+
411
+
### Short Term (Performance Optimization)
412
+
413
+
**2. Evaluate Slingshot for feed performance**
414
+
415
+
If feed latency becomes an issue:
416
+
- Add SlingshotClient as alternative to direct PDS calls
417
+
- A/B test performance improvement
418
+
- Use for public record fetches only (keep direct PDS for authenticated writes)
419
+
420
+
**Trigger:** When registered users exceed ~20-30, or feed refresh exceeds 5 seconds
421
+
422
+
### Medium Term (Real-time Features)
423
+
424
+
**3. Add Spacedust for notifications**
425
+
426
+
When social features are live and users want notifications:
427
+
- Subscribe to Spacedust for likes/comments on user's content
428
+
- Push notifications via WebSocket to connected clients
429
+
- Optional: background job for email notifications
430
+
431
+
**Trigger:** After social features launch, when users request notifications
432
+
433
+
---
434
+
435
+
## Comparison with Official Tools (Jetstream/Tap)
436
+
437
+
See `jetstream-tap-evaluation.md` for official Bluesky tools. Key differences:
438
+
439
+
| Aspect | Microcosm Tools | Official Tools |
440
+
|--------|-----------------|----------------|
441
+
| Focus | Links/interactions | Full records |
442
+
| Backlink queries | Constellation (yes) | Not available |
443
+
| Record caching | Slingshot | Not available |
444
+
| Real-time | Spacedust (links) | Jetstream (records) |
445
+
| Self-hosting | Not yet documented | Available |
446
+
| Community | Community-supported | Bluesky-supported |
447
+
448
+
**Recommendation:** Use Microcosm tools for social features (likes/comments/follows) where backlink queries are essential. Consider Jetstream for full feed real-time if needed later.
449
+
450
+
---
451
+
452
+
## Implementation Plan
453
+
454
+
### Phase 1: Constellation Integration (with social features)
455
+
456
+
```
457
+
1. Create internal/atproto/constellation.go
458
+
- ConstellationClient with HTTP client
459
+
- GetBacklinks(), GetLinkCount(), GetDistinctDIDs()
460
+
461
+
2. Create internal/social/interactions.go
462
+
- GetBrewLikeCount(brewURI)
463
+
- GetBrewComments(brewURI)
464
+
- GetBrewLikers(brewURI)
465
+
466
+
3. Update templates to show interaction counts
467
+
- Modify feed item display
468
+
- Add like button (when like lexicon ready)
469
+
```
470
+
471
+
### Phase 2: Slingshot Optimization (optional)
472
+
473
+
```
474
+
1. Create internal/atproto/slingshot.go
475
+
- SlingshotClient implementing same interface as PublicClient
476
+
477
+
2. Add feature flag: ARABICA_USE_SLINGSHOT=true
478
+
479
+
3. Modify feed/service.go to use SlingshotClient
480
+
- Keep PublicClient as fallback
481
+
```
482
+
483
+
### Phase 3: Spacedust Notifications (future)
484
+
485
+
```
486
+
1. Create internal/notifications/spacedust.go
487
+
- WebSocket client with reconnection
488
+
- Subscribe to user's content interactions
489
+
490
+
2. Create notification storage (BoltDB)
491
+
492
+
3. Add /api/notifications endpoint for frontend polling
493
+
494
+
4. Optional: WebSocket to frontend for real-time
495
+
```
496
+
497
+
---
498
+
499
+
## Related Documentation
500
+
501
+
- Microcosm Main: https://microcosm.blue/
502
+
- Constellation API: https://constellation.microcosm.blue/
503
+
- Source Code: https://github.com/at-microcosm/microcosm-rs
504
+
- Discord: https://discord.gg/tcDfe4PGVB
505
+
- See also: `jetstream-tap-evaluation.md` for official Bluesky tools
+13
internal/bff/helpers.go
+13
internal/bff/helpers.go
···
233
233
234
234
return websiteURL
235
235
}
236
+
237
+
// EscapeJS escapes a string for safe use in JavaScript string literals.
238
+
// Handles newlines, quotes, backslashes, and other special characters.
239
+
func EscapeJS(s string) string {
240
+
// Replace special characters that would break JavaScript strings
241
+
s = strings.ReplaceAll(s, "\\", "\\\\") // Must be first
242
+
s = strings.ReplaceAll(s, "'", "\\'")
243
+
s = strings.ReplaceAll(s, "\"", "\\\"")
244
+
s = strings.ReplaceAll(s, "\n", "\\n")
245
+
s = strings.ReplaceAll(s, "\r", "\\r")
246
+
s = strings.ReplaceAll(s, "\t", "\\t")
247
+
return s
248
+
}
+1
internal/bff/render.go
+1
internal/bff/render.go
+14
internal/middleware/security.go
+14
internal/middleware/security.go
···
176
176
177
177
// getClientIP is defined in logging.go
178
178
179
+
// RequireHTMXMiddleware ensures that certain API routes are only accessible via HTMX requests.
180
+
// This prevents direct browser access to internal API endpoints that return fragments or JSON.
181
+
// Routes that need to be publicly accessible (like /api/resolve-handle) should not use this middleware.
182
+
func RequireHTMXMiddleware(next http.Handler) http.Handler {
183
+
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
184
+
// Check for HTMX request header
185
+
if r.Header.Get("HX-Request") != "true" {
186
+
http.NotFound(w, r)
187
+
return
188
+
}
189
+
next.ServeHTTP(w, r)
190
+
})
191
+
}
192
+
179
193
// MaxBodySize limits the size of request bodies
180
194
const (
181
195
MaxJSONBodySize = 1 << 20 // 1 MB for JSON requests
+9
-12
internal/routing/routing.go
+9
-12
internal/routing/routing.go
···
34
34
mux.HandleFunc("GET /.well-known/oauth-client-metadata", h.HandleWellKnownOAuth)
35
35
36
36
// API routes for handle resolution (used by login autocomplete)
37
+
// These are intentionally public and don't require HTMX headers
37
38
mux.HandleFunc("GET /api/resolve-handle", h.HandleResolveHandle)
38
39
mux.HandleFunc("GET /api/search-actors", h.HandleSearchActors)
39
40
40
-
// API route for fetching all user data (used by client-side cache)
41
+
// API route for fetching all user data (used by client-side cache via fetch())
42
+
// Auth-protected but accessible without HTMX header (called from JavaScript)
41
43
mux.HandleFunc("GET /api/data", h.HandleAPIListAll)
42
44
43
-
// Community feed partial (loaded async via HTMX)
44
-
mux.HandleFunc("GET /api/feed", h.HandleFeedPartial)
45
-
46
-
// Brew list partial (loaded async via HTMX)
47
-
mux.HandleFunc("GET /api/brews", h.HandleBrewListPartial)
48
-
49
-
// Manage page partial (loaded async via HTMX)
50
-
mux.HandleFunc("GET /api/manage", h.HandleManagePartial)
51
-
52
-
// Profile content partial (loaded async via HTMX)
53
-
mux.HandleFunc("GET /api/profile/{actor}", h.HandleProfilePartial)
45
+
// HTMX partials (loaded async via HTMX)
46
+
// These return HTML fragments and should only be accessed via HTMX
47
+
mux.Handle("GET /api/feed", middleware.RequireHTMXMiddleware(http.HandlerFunc(h.HandleFeedPartial)))
48
+
mux.Handle("GET /api/brews", middleware.RequireHTMXMiddleware(http.HandlerFunc(h.HandleBrewListPartial)))
49
+
mux.Handle("GET /api/manage", middleware.RequireHTMXMiddleware(http.HandlerFunc(h.HandleManagePartial)))
50
+
mux.Handle("GET /api/profile/{actor}", middleware.RequireHTMXMiddleware(http.HandlerFunc(h.HandleProfilePartial)))
54
51
55
52
// Page routes (must come before static files)
56
53
mux.HandleFunc("GET /{$}", h.HandleHome) // {$} means exact match
+3
-3
templates/partials/manage_content.tmpl
+3
-3
templates/partials/manage_content.tmpl
···
44
44
<td class="px-6 py-4 text-sm text-brown-900">{{.Process}}</td>
45
45
<td class="px-6 py-4 text-sm text-brown-700">{{.Description}}</td>
46
46
<td class="px-6 py-4 text-sm font-medium space-x-2">
47
-
<button @click="editBean('{{.RKey}}', '{{.Name}}', '{{.Origin}}', '{{.RoastLevel}}', '{{.Process}}', '{{.Description}}', '{{.RoasterRKey}}')"
47
+
<button @click="editBean('{{.RKey}}', '{{escapeJS .Name}}', '{{escapeJS .Origin}}', '{{.RoastLevel}}', '{{.Process}}', '{{escapeJS .Description}}', '{{.RoasterRKey}}')"
48
48
class="text-brown-700 hover:text-brown-900 font-medium">Edit</button>
49
49
<button @click="deleteBean('{{.RKey}}')"
50
50
class="text-brown-600 hover:text-brown-800 font-medium">Delete</button>
···
95
95
{{end}}
96
96
</td>
97
97
<td class="px-6 py-4 text-sm font-medium space-x-2">
98
-
<button @click="editRoaster('{{.RKey}}', '{{.Name}}', '{{.Location}}', '{{.Website}}')"
98
+
<button @click="editRoaster('{{.RKey}}', '{{escapeJS .Name}}', '{{escapeJS .Location}}', '{{escapeJS .Website}}')"
99
99
class="text-brown-700 hover:text-brown-900 font-medium">Edit</button>
100
100
<button @click="deleteRoaster('{{.RKey}}')"
101
101
class="text-brown-600 hover:text-brown-800 font-medium">Delete</button>
···
143
143
<td class="px-6 py-4 text-sm text-brown-900">{{.BurrType}}</td>
144
144
<td class="px-6 py-4 text-sm text-brown-700">{{.Notes}}</td>
145
145
<td class="px-6 py-4 text-sm font-medium space-x-2">
146
-
<button @click="editGrinder('{{.RKey}}', '{{.Name}}', '{{.GrinderType}}', '{{.BurrType}}', '{{.Notes}}')"
146
+
<button @click="editGrinder('{{.RKey}}', '{{escapeJS .Name}}', '{{.GrinderType}}', '{{.BurrType}}', '{{escapeJS .Notes}}')"
147
147
class="text-brown-700 hover:text-brown-900 font-medium">Edit</button>
148
148
<button @click="deleteGrinder('{{.RKey}}')"
149
149
class="text-brown-600 hover:text-brown-800 font-medium">Delete</button>