hailey.at
this repo has no description
1# Vylet Go Monorepo
2
3## Services
4
5### CDN Service
6
7The CDN service tracks blob references from the ATProto firehose and stores them in the database for resolution and serving.
8
9#### What it does
10
11- Subscribes to the Kafka firehose topic
12- Extracts blob references from all records using `atdata.ExtractBlobs()`
13- Stores blob metadata in the `blob_refs` table with DID and CID as primary key
14- Tracks first seen time, processing time, and update time for each blob
15- Exposes Prometheus metrics for monitoring
16
17#### Running locally
18
19```bash
20# Using justfile
21just run-cdn
22
23# Or directly with go run
24go run ./cmd/cdn
25```
26
27#### Configuration
28
29Environment variables:
30- `VYLET_CDN_DATABASE_HOST` - Database server address (default: `127.0.0.1:9090`)
31- `VYLET_BOOTSTRAP_SERVERS` - Kafka bootstrap servers (default: `localhost:9092`)
32- `VYLET_CDN_INPUT_TOPIC` - Firehose topic to consume (default: `firehose-events-prod`)
33- `VYLET_CDN_CONSUMER_GROUP` - Kafka consumer group (required)
34
35#### Metrics
36
37The CDN service exposes the following Prometheus metrics:
38
39- `cdn_blobs_extracted_total` - Total number of blobs extracted from records
40- `cdn_db_operations_total{operation, status}` - Database operations by type (create/update) and status (success/error)
41- `cdn_records_processed_total{operation}` - Records processed by operation type
42
43#### API Integration
44
45Blob references tracked by the CDN service can be resolved via the API:
46
47```
48GET /xrpc/app.vylet.media.getBlob?did=<did>&cid=<cid>
49```
50
51This endpoint:
521. Fetches blob metadata from `blob_refs` table
532. Checks if the blob is taken down
543. Resolves the PDS endpoint from the DID document
554. Returns a 302 redirect to the blob on the user's PDS