sprk.so
Spark feed generator template
1# Spark Feed Generator
2
3A template for creating custom feed generators for [sprk.so](https://sprk.so).
4
5## Architecture
6
7The feed generator has three main components:
8
91. **Ingester** (`ingester/`) - Consumes the Spark firehose in real-time and
10 indexes records to MongoDB. By default, it indexes `so.sprk.feed.post`
11 records. Handlers for `follow`, `like`, and `repost` are included but
12 disabled - enable them by uncommenting in `ingester/index.ts`.
13
142. **Algorithms** (`algos/`) - Define how posts are selected and sorted for your
15 feed. Each algorithm exports:
16 - `handler` - Query function that returns posts from the database
17 - `needsAuth` - Whether the feed requires user authentication
18 - `publisherDid` - The DID of the feed publisher
19 - `rkey` - Unique identifier for this algorithm
20
213. **API Server** - Exposes XRPC endpoints that Spark clients call to fetch feed
22 content (`so.sprk.feed.getFeedSkeleton`,
23 `so.sprk.feed.describeFeedGenerator`). You won't need to modify these when
24 creating a feed.
25
26## Creating Custom Feeds
27
28**For topic/community feeds:** Filter posts at the ingester level. Modify the
29handler in `ingester/handlers/post.ts` to only index posts matching your
30criteria (hashtags, keywords, specific authors, etc.).
31
32**For personalized/sorted feeds:** Create a new algorithm in `algos/`. Copy
33`simple-desc.ts` as a starting point, then modify the query logic. Register your
34algorithm in `algos/index.ts`.
35
36Example algorithm structure:
37
38```ts
39// algos/my-feed.ts
40export const info = {
41 handler: async (ctx, params, did) => {
42 // Query posts from ctx.db.models.Post
43 // Return { cursor, feed: [{ post: "at://..." }] }
44 },
45 needsAuth: false, // Set true if feed needs user's DID
46 publisherDid: "did:plc:your-did",
47 rkey: "my-feed", // at://your-did/so.sprk.feed.generator/my-feed
48} as Algorithm;
49```
50
51## Running
52
53### Prerequisites
54
55Create a `.env` file in the project root:
56
57```sh
58# Required
59FEEDGEN_DOMAIN=feeds.example.com # Your feed generator's domain
60SPRK_DB_NAME=feed-gen # MongoDB database name
61SPRK_DB_URI=mongodb://localhost:27017
62SPRK_DB_USER=username
63SPRK_DB_PASS=password
64
65# Optional
66SPRK_HOST=0.0.0.0 # Server bind address
67SPRK_PORT=3000 # Server port
68NODE_ENV=production
69```
70
71### Docker Compose (Recommended)
72
73Starts both the feed generator and a MongoDB instance:
74
75```sh
76# Start services
77docker-compose up -d
78
79# View logs
80docker-compose logs -f feed-gen
81
82# Stop services
83docker-compose down
84```
85
86### Manual Setup
87
88Requires [Deno](https://deno.com/) and a MongoDB instance.
89
90```sh
91# Install dependencies
92deno install
93
94# Start server
95deno task start
96```
97
98The server will be available at `http://localhost:3000`.
99
100## Endpoints
101
102| Endpoint | Description |
103| ---------------------------------------------- | ------------------------------------- |
104| `GET /` | Service info |
105| `GET /xrpc/_health` | Health check |
106| `GET /.well-known/did.json` | DID document for `did:web` resolution |
107| `GET /xrpc/so.sprk.feed.describeFeedGenerator` | List available feeds |
108| `GET /xrpc/so.sprk.feed.getFeedSkeleton` | Fetch feed posts |