hailey.at
this repo has no description
Add Tap mode (#2)
* tap mode
* add acks
* update dockercompose with tap
* update readme
+508
-37
+29
-33
README.md
+29
-33
README.md
···
3
A small service that receives events from the AT firehose and produces them to Kafka. Supports standard JSON outputs as well as [Osprey](https://github.com/roostorg/osprey)
4
formatted events.
5
6
## Usage
7
8
### Docker Compose
···
11
12
```yaml
13
environment:
14
-
ATKAFKA_RELAY_HOST: "wss://bsky.network"
15
-
ATKAFKA_BOOTSTRAP_SERVERS: "kafka:29092"
16
-
ATKAFKA_OUTPUT_TOPIC: "atproto-events"
17
-
ATKAFKA_OSPREY_COMPATIBLE: "false"
18
-
```
19
20
-
Then start:
21
22
-
```bash
23
-
docker compose up -d # Start services
24
-
```
25
-
26
-
### Docker Run
27
28
-
For standard mode:
29
30
-
```bash
31
-
docker run -d \
32
-
-e ATKAFKA_BOOTSTRAP_SERVERS=kafka:9092 \
33
-
-e ATKAFKA_OUTPUT_TOPIC=atproto-events \
34
-
-p 2112:2112 \
35
-
ghcr.io/haileyok/at-kafka:latest
36
```
37
38
-
For Osprey-compatible mode:
39
40
```bash
41
-
docker run -d \
42
-
-e ATKAFKA_BOOTSTRAP_SERVERS=kafka:9092 \
43
-
-e ATKAFKA_OUTPUT_TOPIC=atproto-events \
44
-
-e ATKAFKA_OSPREY_COMPATIBLE=true \
45
-
-p 2112:2112 \
46
-
ghcr.io/haileyok/at-kafka:latest
47
-
```
48
49
-
## Configuration
50
51
-
| Flag | Environment Variable | Default | Description |
52
-
|------|---------------------|---------|-------------|
53
-
| `--relay-host` | `ATKAFKA_RELAY_HOST` | `wss://bsky.network` | AT Protocol relay host to connect to |
54
-
| `--bootstrap-servers` | `ATKAFKA_BOOTSTRAP_SERVERS` | (required) | Comma-separated list of Kafka bootstrap servers |
55
-
| `--output-topic` | `ATKAFKA_OUTPUT_TOPIC` | (required) | Kafka topic to publish events to |
56
-
| `--osprey-compatible` | `ATKAFKA_OSPREY_COMPATIBLE` | `false` | Enable Osprey-compatible event format |
57
58
## Event Structure
59
···
143
144
- `atkafka_handled_events` - Total events that are received on the firehose and handled
145
- `atkafka_produced_events` - Total messages that are output on the bus
···
3
A small service that receives events from the AT firehose and produces them to Kafka. Supports standard JSON outputs as well as [Osprey](https://github.com/roostorg/osprey)
4
formatted events.
5
6
+
Additionally, at-kafka supports subscribing to [Tap](https://github.com/bluesky-social/indigo/tree/main/cmd/tap) if youare attempting to perform a network backfill.
7
+
8
## Usage
9
10
### Docker Compose
···
13
14
```yaml
15
environment:
16
+
# For relay mode
17
+
ATKAFKA_RELAY_HOST: "wss://bsky.network" # ATProto relay to subscribe to for events
18
19
+
# For tap mode
20
+
ATKAFKA_TAP_HOST: "ws://localhost:2480" # Tap websocket host to subscribe to for events
21
+
ATKAFKA_DISABLE_ACKS: false # Whether to disable sending of acks to Tap
22
23
+
# Kafka configuration
24
+
ATKAFKA_BOOTSTRAP_SERVERS: "kafka:29092" # Kafka bootstrap servers, comma separated
25
+
ATKAFKA_OUTPUT_TOPIC: "atproto-events" # The output topic for events
26
+
ATKAFKA_OSPREY_COMPATIBLE: false # Whether to produce Osprey-compatible events
27
28
+
# Match only Blacksky PDS users
29
+
ATKAFKA_MATCHED_SERVICES: "blacksky.app" # A comma-separated list of PDSes to emit events for
30
+
# OR ignore anyone on Bluesky PBC PDSes
31
+
ATKAFKA_IGNORED_SERVICES: "*.bsky.network" # OR a comma-separated list of PDSes to _not_ emit events for
32
33
+
# Match only Teal.fm records
34
+
ATKAFKA_MATCHED_COLLECTIONS: "fm.teal.*" # A comma-separated list of collections to emit events for
35
+
# OR ignore all Bluesky records
36
+
ATKAFKA_IGNORED_COLLECTIONS: "app.bsky.*" # OR a comma-separated list of collections to ignore events for
37
```
38
39
+
Then start:
40
41
```bash
42
+
# For normal mode
43
+
docker compose up -d
44
45
+
# For tap mode
46
+
docker compose -f docker-compose.tap.yml up -d
47
48
+
```
49
50
## Event Structure
51
···
135
136
- `atkafka_handled_events` - Total events that are received on the firehose and handled
137
- `atkafka_produced_events` - Total messages that are output on the bus
138
+
- `atkafka_plc_requests` - Total number of PLC requests that were made, if applicable, and whether they were cached
139
+
- `atkafka_api_requests` - Total number of API requests that were made, if applicable, and whether they were cached
140
+
- `atkafka_cache_size` - The size of the PLC and API caches
141
+
- `atkafka_acks_sent` - Total acks that were sent to Tap, if applicable
+10
-3
atkafka/atkafka.go
+10
-3
atkafka/atkafka.go
···
28
29
type Server struct {
30
relayHost string
31
bootstrapServers []string
32
outputTopic string
33
ospreyCompat bool
···
42
plcClient *PlcClient
43
apiClient *ApiClient
44
logger *slog.Logger
45
}
46
47
type ServerArgs struct {
48
// network params
49
-
RelayHost string
50
-
PlcHost string
51
-
ApiHost string
52
53
// for watched and ignoed services or collections, only one list may be supplied
54
// for both services and collections, wildcards are acceptable. for example:
···
113
114
s := &Server{
115
relayHost: args.RelayHost,
116
plcClient: plcClient,
117
apiClient: apiClient,
118
bootstrapServers: args.BootstrapServers,
···
28
29
type Server struct {
30
relayHost string
31
+
tapHost string
32
+
disableAcks bool
33
bootstrapServers []string
34
outputTopic string
35
ospreyCompat bool
···
44
plcClient *PlcClient
45
apiClient *ApiClient
46
logger *slog.Logger
47
+
ws *websocket.Conn
48
}
49
50
type ServerArgs struct {
51
// network params
52
+
RelayHost string
53
+
TapHost string
54
+
DisableAcks bool
55
+
PlcHost string
56
+
ApiHost string
57
58
// for watched and ignoed services or collections, only one list may be supplied
59
// for both services and collections, wildcards are acceptable. for example:
···
118
119
s := &Server{
120
relayHost: args.RelayHost,
121
+
tapHost: args.TapHost,
122
+
disableAcks: args.DisableAcks,
123
plcClient: plcClient,
124
apiClient: apiClient,
125
bootstrapServers: args.BootstrapServers,
+5
atkafka/metrics.go
+5
atkafka/metrics.go
+303
atkafka/tap.go
+303
atkafka/tap.go
···
···
1
+
package atkafka
2
+
3
+
import (
4
+
"context"
5
+
"encoding/json"
6
+
"fmt"
7
+
"net/http"
8
+
"net/url"
9
+
"os"
10
+
"os/signal"
11
+
"strings"
12
+
"syscall"
13
+
"time"
14
+
15
+
"github.com/araddon/dateparse"
16
+
"github.com/bluesky-social/indigo/atproto/syntax"
17
+
"github.com/gorilla/websocket"
18
+
"github.com/twmb/franz-go/pkg/kgo"
19
+
"golang.org/x/sync/semaphore"
20
+
)
21
+
22
+
func (s *Server) RunTapMode(ctx context.Context) error {
23
+
sema := semaphore.NewWeighted(1_000)
24
+
25
+
s.logger.Info("starting tap consumer", "tap-host", s.tapHost, "bootstrap-servers", s.bootstrapServers, "output-topic", s.outputTopic)
26
+
27
+
createCtx, cancel := context.WithTimeout(ctx, 10*time.Second)
28
+
defer cancel()
29
+
30
+
producerLogger := s.logger.With("component", "producer")
31
+
kafProducer, err := NewProducer(createCtx, producerLogger, s.bootstrapServers, s.outputTopic,
32
+
WithEnsureTopic(true),
33
+
WithTopicPartitions(200),
34
+
)
35
+
if err != nil {
36
+
return fmt.Errorf("failed to create producer: %w", err)
37
+
}
38
+
defer kafProducer.Close()
39
+
s.producer = kafProducer
40
+
s.logger.Info("created producer")
41
+
42
+
wsDialer := websocket.DefaultDialer
43
+
u, err := url.Parse(s.tapHost)
44
+
if err != nil {
45
+
return fmt.Errorf("invalid tapHost: %w", err)
46
+
}
47
+
u.Path = "/channel"
48
+
s.logger.Info("created dialer")
49
+
50
+
wsErr := make(chan error, 1)
51
+
shutdownWs := make(chan struct{}, 1)
52
+
go func() {
53
+
logger := s.logger.With("component", "websocket")
54
+
55
+
logger.Info("subscribing to tap stream", "upstream", s.tapHost)
56
+
57
+
conn, _, err := wsDialer.Dial(u.String(), http.Header{
58
+
"User-Agent": []string{"at-kafka/0.0.0"},
59
+
})
60
+
if err != nil {
61
+
wsErr <- err
62
+
return
63
+
}
64
+
s.ws = conn
65
+
66
+
for {
67
+
var evt TapEvent
68
+
err := conn.ReadJSON(&evt)
69
+
if err != nil {
70
+
logger.Error("error reading json from websocket", "err", err)
71
+
break
72
+
}
73
+
74
+
if err := sema.Acquire(ctx, 1); err != nil {
75
+
logger.Error("error acquring sema", "err", err)
76
+
break
77
+
}
78
+
79
+
go func() {
80
+
defer sema.Release(1)
81
+
s.handleTapEvent(ctx, &evt)
82
+
}()
83
+
}
84
+
85
+
<-shutdownWs
86
+
87
+
wsErr <- nil
88
+
}()
89
+
s.logger.Info("created tap consumer")
90
+
91
+
signals := make(chan os.Signal, 1)
92
+
signal.Notify(signals, syscall.SIGTERM, syscall.SIGINT)
93
+
94
+
select {
95
+
case sig := <-signals:
96
+
s.logger.Info("shutting down on signal", "signal", sig)
97
+
case err := <-wsErr:
98
+
if err != nil {
99
+
s.logger.Error("websocket error", "err", err)
100
+
} else {
101
+
s.logger.Info("websocket shutdown unexpectedly")
102
+
}
103
+
}
104
+
105
+
close(shutdownWs)
106
+
107
+
return nil
108
+
}
109
+
110
+
func (s *Server) handleTapEvent(ctx context.Context, evt *TapEvent) error {
111
+
logger := s.logger.With("component", "handleEvent")
112
+
113
+
var collection string
114
+
var actionName string
115
+
116
+
var evtKey string
117
+
var evtsToProduce [][]byte
118
+
119
+
if evt.Record != nil {
120
+
// key events by DID
121
+
evtKey = evt.Record.Did
122
+
did := evt.Record.Did
123
+
kind := evt.Record.Action
124
+
collection = evt.Record.Collection
125
+
rkey := evt.Record.Rkey
126
+
atUri := fmt.Sprintf("at://%s/%s/%s", did, collection, rkey)
127
+
128
+
skip := false
129
+
if len(s.watchedCollections) > 0 {
130
+
skip = true
131
+
for _, watchedCollection := range s.watchedCollections {
132
+
if watchedCollection == collection || strings.HasPrefix(collection, watchedCollection+".") {
133
+
skip = false
134
+
break
135
+
}
136
+
}
137
+
} else if len(s.ignoredCollections) > 0 {
138
+
for _, ignoredCollection := range s.ignoredCollections {
139
+
if ignoredCollection == collection || strings.HasPrefix(collection, ignoredCollection+".") {
140
+
skip = true
141
+
break
142
+
}
143
+
}
144
+
}
145
+
146
+
if skip {
147
+
logger.Debug("skipping event based on collection", "collection", collection)
148
+
return nil
149
+
}
150
+
151
+
actionName = "operation#" + kind
152
+
153
+
handledEvents.WithLabelValues(actionName, collection).Inc()
154
+
155
+
// create the formatted operation
156
+
atkOp := AtKafkaOp{
157
+
Action: evt.Record.Action,
158
+
Collection: collection,
159
+
Rkey: rkey,
160
+
Uri: atUri,
161
+
Cid: evt.Record.Cid,
162
+
Path: fmt.Sprintf("%s/%s", collection, rkey),
163
+
}
164
+
165
+
if evt.Record.Record != nil {
166
+
atkOp.Record = *evt.Record.Record
167
+
}
168
+
169
+
kafkaEvt := AtKafkaEvent{
170
+
Did: did,
171
+
Operation: &atkOp,
172
+
}
173
+
174
+
if evt.Record.Record != nil {
175
+
timestamp, err := parseTimeFromRecord(collection, *evt.Record.Record, rkey)
176
+
if err != nil {
177
+
return fmt.Errorf("error getting timestamp from record: %w", err)
178
+
}
179
+
kafkaEvt.Timestamp = timestamp.Format(time.RFC3339Nano)
180
+
}
181
+
182
+
evtBytes, err := json.Marshal(&kafkaEvt)
183
+
if err != nil {
184
+
return fmt.Errorf("failed to marshal kafka event: %w", err)
185
+
}
186
+
187
+
evtsToProduce = append(evtsToProduce, evtBytes)
188
+
}
189
+
190
+
for _, evtBytes := range evtsToProduce {
191
+
if err := s.produceAsyncTap(ctx, evtKey, evtBytes, evt.Id); err != nil {
192
+
return err
193
+
}
194
+
}
195
+
196
+
return nil
197
+
}
198
+
199
+
func (s *Server) produceAsyncTap(ctx context.Context, key string, msg []byte, id int64) error {
200
+
logger := s.logger.With("name", "produceAsyncTap", "key", key, "id", id)
201
+
callback := func(r *kgo.Record, err error) {
202
+
status := "ok"
203
+
ackStatus := "ok"
204
+
205
+
go func() {
206
+
producedEvents.WithLabelValues(status).Inc()
207
+
if s.ws != nil && !s.disableAcks {
208
+
acksSent.WithLabelValues(ackStatus).Inc()
209
+
}
210
+
}()
211
+
212
+
if err != nil {
213
+
logger.Error("error producing message", "err", err)
214
+
status = "error"
215
+
} else if s.ws != nil && !s.disableAcks {
216
+
if err := s.ws.WriteJSON(TapAck{
217
+
Type: "ack",
218
+
Id: id,
219
+
}); err != nil {
220
+
logger.Error("error sending ack", "err", err)
221
+
ackStatus = "error"
222
+
}
223
+
}
224
+
}
225
+
226
+
if err := s.producer.ProduceAsync(ctx, key, msg, callback); err != nil {
227
+
return fmt.Errorf("failed to produce message: %w", err)
228
+
}
229
+
230
+
return nil
231
+
}
232
+
233
+
func parseTimeFromRecord(collection string, rec map[string]any, rkey string) (*time.Time, error) {
234
+
var rkeyTime time.Time
235
+
if rkey != "self" {
236
+
rt, err := syntax.ParseTID(rkey)
237
+
if err == nil {
238
+
rkeyTime = rt.Time()
239
+
}
240
+
}
241
+
242
+
switch collection {
243
+
case "app.bsky.feed.post":
244
+
cat, ok := rec["createdAt"].(string)
245
+
if ok {
246
+
t, err := dateparse.ParseAny(cat)
247
+
if err == nil {
248
+
return &t, nil
249
+
}
250
+
251
+
if rkeyTime.IsZero() {
252
+
return timePtr(time.Now()), nil
253
+
}
254
+
}
255
+
256
+
return &rkeyTime, nil
257
+
case "app.bsky.feed.repost":
258
+
cat, ok := rec["createdAt"].(string)
259
+
if ok {
260
+
t, err := dateparse.ParseAny(cat)
261
+
if err == nil {
262
+
return &t, nil
263
+
}
264
+
265
+
if rkeyTime.IsZero() {
266
+
return nil, fmt.Errorf("failed to get a useful timestamp from record")
267
+
}
268
+
}
269
+
270
+
return &rkeyTime, nil
271
+
case "app.bsky.feed.like":
272
+
cat, ok := rec["createdAt"].(string)
273
+
if ok {
274
+
t, err := dateparse.ParseAny(cat)
275
+
if err == nil {
276
+
return &t, nil
277
+
}
278
+
279
+
if rkeyTime.IsZero() {
280
+
return nil, fmt.Errorf("failed to get a useful timestamp from record")
281
+
}
282
+
}
283
+
284
+
return &rkeyTime, nil
285
+
case "app.bsky.actor.profile":
286
+
// We can't really trust the createdat in the profile record anyway, and its very possible its missing. just use iat for this one
287
+
return timePtr(time.Now()), nil
288
+
case "app.bsky.feed.generator":
289
+
if !rkeyTime.IsZero() {
290
+
return &rkeyTime, nil
291
+
}
292
+
return timePtr(time.Now()), nil
293
+
default:
294
+
if !rkeyTime.IsZero() {
295
+
return &rkeyTime, nil
296
+
}
297
+
return timePtr(time.Now()), nil
298
+
}
299
+
}
300
+
301
+
func timePtr(t time.Time) *time.Time {
302
+
return &t
303
+
}
+30
atkafka/types.go
+30
atkafka/types.go
···
65
AccountAge int64 `json:"accountAge"`
66
Profile *bsky.ActorDefs_ProfileViewDetailed `json:"profile"`
67
}
68
+
69
+
type TapEvent struct {
70
+
Id int64 `json:"id"`
71
+
Type string `json:"type"`
72
+
Record *TapEventRecord `json:"record,omitempty"`
73
+
Identity *TapEventIdentity `json:"identity,omitempty"`
74
+
}
75
+
76
+
type TapEventRecord struct {
77
+
Live bool `json:"live"`
78
+
Rev string `json:"rev"`
79
+
Did string `json:"did"`
80
+
Collection string `json:"collection"`
81
+
Rkey string `json:"rkey"`
82
+
Action string `json:"action"`
83
+
Cid string `json:"cid"`
84
+
Record *map[string]any `json:"record,omitempty"`
85
+
}
86
+
87
+
type TapEventIdentity struct {
88
+
Did string `json:"did"`
89
+
Handle string `json:"handle"`
90
+
IsActive bool `json:"isActive"`
91
+
Status string `json:"status"`
92
+
}
93
+
94
+
type TapAck struct {
95
+
Type string `json:"type"`
96
+
Id int64 `json:"id"`
97
+
}
+44
cmd/atkafka/main.go
+44
cmd/atkafka/main.go
···
102
103
return nil
104
},
105
+
Commands: cli.Commands{
106
+
&cli.Command{
107
+
Name: "tap-mode",
108
+
Flags: []cli.Flag{
109
+
&cli.StringFlag{
110
+
Name: "tap-host",
111
+
Usage: "Tap host to subscribe to for events",
112
+
Value: "ws://localhost:2480",
113
+
EnvVars: []string{"ATKAFKA_TAP_HOST"},
114
+
},
115
+
&cli.BoolFlag{
116
+
Name: "disable-acks",
117
+
Usage: "Set to `true` to disable sending of event acks to Tap. May result in the loss of events.",
118
+
EnvVars: []string{"ATKAFKA_DISABLE_ACKS"},
119
+
},
120
+
},
121
+
Action: func(cmd *cli.Context) error {
122
+
123
+
ctx := context.Background()
124
+
125
+
telemetry.StartMetrics(cmd)
126
+
logger := telemetry.StartLogger(cmd)
127
+
128
+
s, err := atkafka.NewServer(&atkafka.ServerArgs{
129
+
TapHost: cmd.String("tap-host"),
130
+
DisableAcks: cmd.Bool("disable-acks"),
131
+
BootstrapServers: cmd.StringSlice("bootstrap-servers"),
132
+
OutputTopic: cmd.String("output-topic"),
133
+
WatchedCollections: cmd.StringSlice("watched-collections"),
134
+
IgnoredCollections: cmd.StringSlice("ignored-collections"),
135
+
Logger: logger,
136
+
})
137
+
if err != nil {
138
+
return fmt.Errorf("failed to create new server: %w", err)
139
+
}
140
+
141
+
if err := s.RunTapMode(ctx); err != nil {
142
+
return fmt.Errorf("error running server: %w", err)
143
+
}
144
+
145
+
return nil
146
+
},
147
+
},
148
+
},
149
}
150
151
if err := app.Run(os.Args); err != nil {
+80
docker-compose.tap.yml
+80
docker-compose.tap.yml
···
···
1
+
services:
2
+
zookeeper:
3
+
image: confluentinc/cp-zookeeper:7.6.0
4
+
hostname: zookeeper
5
+
container_name: zookeeper
6
+
ports:
7
+
- "2181:2181"
8
+
environment:
9
+
ZOOKEEPER_CLIENT_PORT: 2181
10
+
ZOOKEEPER_TICK_TIME: 2000
11
+
volumes:
12
+
- zookeeper-data:/var/lib/zookeeper/data
13
+
- zookeeper-logs:/var/lib/zookeeper/log
14
+
15
+
kafka:
16
+
image: confluentinc/cp-kafka:7.6.0
17
+
hostname: kafka
18
+
container_name: kafka
19
+
depends_on:
20
+
- zookeeper
21
+
ports:
22
+
- "9092:9092"
23
+
- "9101:9101"
24
+
environment:
25
+
KAFKA_BROKER_ID: 1
26
+
KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181'
27
+
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
28
+
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:29092,PLAINTEXT_HOST://localhost:9092
29
+
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
30
+
KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
31
+
KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
32
+
KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0
33
+
KAFKA_JMX_PORT: 9101
34
+
KAFKA_JMX_HOSTNAME: localhost
35
+
KAFKA_AUTO_CREATE_TOPICS_ENABLE: 'true'
36
+
volumes:
37
+
- kafka-data:/var/lib/kafka/data
38
+
39
+
40
+
tap:
41
+
image: ghcr.io/bluesky-social/indigo/tap:latest
42
+
hostname: tap
43
+
container_name: tap
44
+
depends_on:
45
+
- kafka
46
+
ports:
47
+
- "2480:2480"
48
+
environment:
49
+
TAP_BIND: ":2480"
50
+
TAP_FULL_NETWORK: true
51
+
TAP_DISABLE_ACKS: false
52
+
TAP_COLLECTION_FILTERS: "app.bsky.graph.follow"
53
+
volumes:
54
+
- tap-data:/data
55
+
56
+
atkafka:
57
+
build:
58
+
context: .
59
+
dockerfile: Dockerfile
60
+
container_name: atkafka
61
+
depends_on:
62
+
- kafka
63
+
- tap
64
+
ports:
65
+
# metrics port
66
+
- "6010:6009"
67
+
command: ["tap-mode"]
68
+
environment:
69
+
ATKAFKA_TAP_HOST: "ws://tap:2480"
70
+
ATKAFKA_DISABLE_ACKS: false
71
+
ATKAFKA_BOOTSTRAP_SERVERS: "kafka:29092"
72
+
ATKAFKA_OUTPUT_TOPIC: "tap-events"
73
+
74
+
restart: unless-stopped
75
+
76
+
volumes:
77
+
zookeeper-data:
78
+
zookeeper-logs:
79
+
kafka-data:
80
+
tap-data:
+2
-1
go.mod
+2
-1
go.mod
···
3
go 1.25.4
4
5
require (
6
github.com/bluesky-social/go-util v0.0.0-20251012040650-2ebbf57f5934
7
github.com/bluesky-social/indigo v0.0.0-20251125184450-35c1e15d2e5f
8
github.com/gorilla/websocket v1.5.3
···
11
github.com/prometheus/client_golang v1.23.2
12
github.com/twmb/franz-go v1.19.5
13
github.com/urfave/cli/v2 v2.25.7
14
golang.org/x/time v0.6.0
15
)
16
···
95
go.yaml.in/yaml/v2 v2.4.2 // indirect
96
golang.org/x/crypto v0.40.0 // indirect
97
golang.org/x/exp v0.0.0-20240112132812-db7319d0e0e3 // indirect
98
-
golang.org/x/sync v0.16.0 // indirect
99
golang.org/x/sys v0.35.0 // indirect
100
golang.org/x/text v0.28.0 // indirect
101
golang.org/x/xerrors v0.0.0-20231012003039-104605ab7028 // indirect
···
3
go 1.25.4
4
5
require (
6
+
github.com/araddon/dateparse v0.0.0-20210429162001-6b43995a97de
7
github.com/bluesky-social/go-util v0.0.0-20251012040650-2ebbf57f5934
8
github.com/bluesky-social/indigo v0.0.0-20251125184450-35c1e15d2e5f
9
github.com/gorilla/websocket v1.5.3
···
12
github.com/prometheus/client_golang v1.23.2
13
github.com/twmb/franz-go v1.19.5
14
github.com/urfave/cli/v2 v2.25.7
15
+
golang.org/x/sync v0.16.0
16
golang.org/x/time v0.6.0
17
)
18
···
97
go.yaml.in/yaml/v2 v2.4.2 // indirect
98
golang.org/x/crypto v0.40.0 // indirect
99
golang.org/x/exp v0.0.0-20240112132812-db7319d0e0e3 // indirect
100
golang.org/x/sys v0.35.0 // indirect
101
golang.org/x/text v0.28.0 // indirect
102
golang.org/x/xerrors v0.0.0-20231012003039-104605ab7028 // indirect
+5
go.sum
+5
go.sum
···
3
github.com/RussellLuo/slidingwindow v0.0.0-20200528002341-535bb99d338b/go.mod h1:4+EPqMRApwwE/6yo6CxiHoSnBzjRr3jsqer7frxP8y4=
4
github.com/alexbrainman/goissue34681 v0.0.0-20191006012335-3fc7a47baff5 h1:iW0a5ljuFxkLGPNem5Ui+KBjFJzKg4Fv2fnxe4dvzpM=
5
github.com/alexbrainman/goissue34681 v0.0.0-20191006012335-3fc7a47baff5/go.mod h1:Y2QMoi1vgtOIfc+6DhrMOGkLoGzqSV2rKp4Sm+opsyA=
6
github.com/benbjohnson/clock v1.1.0/go.mod h1:J11/hYXuz8f4ySSvYwY0FKfm+ezbsZBKZxNJlLklBHA=
7
github.com/benbjohnson/clock v1.3.0 h1:ip6w0uFQkncKQ979AypyG0ER7mqUSBdKLOgAle/AT8A=
8
github.com/benbjohnson/clock v1.3.0/go.mod h1:J11/hYXuz8f4ySSvYwY0FKfm+ezbsZBKZxNJlLklBHA=
···
200
github.com/mattn/go-isatty v0.0.14/go.mod h1:7GGIvUiUoEMVVmxf/4nioHXj79iQHKdU27kJ6hsGG94=
201
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
202
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
203
github.com/mattn/go-sqlite3 v1.14.22 h1:2gZY6PC6kBnID23Tichd1K+Z0oS6nE/XwU+Vz/5o4kU=
204
github.com/mattn/go-sqlite3 v1.14.22/go.mod h1:Uh1q+B4BYcTPb+yiD3kU8Ct7aC0hY9fxUwlHK0RXw+Y=
205
github.com/miekg/dns v1.1.50 h1:DQUfb9uc6smULcREF09Uc+/Gd46YWqJd5DbpPE9xkcA=
···
249
github.com/prometheus/common v0.66.1/go.mod h1:gcaUsgf3KfRSwHY4dIMXLPV0K/Wg1oZ8+SbZk/HH/dA=
250
github.com/prometheus/procfs v0.16.1 h1:hZ15bTNuirocR6u0JZ6BAHHmwS1p8B4P6MRqxtzMyRg=
251
github.com/prometheus/procfs v0.16.1/go.mod h1:teAbpZRB1iIAJYREa1LsoWUXykVXA1KlTmWl8x/U+Is=
252
github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
253
github.com/rogpeppe/go-internal v1.10.0 h1:TMyTOH3F/DB16zRVcYyreMH6GnZZrwQVAoYjRBZyWFQ=
254
github.com/rogpeppe/go-internal v1.10.0/go.mod h1:UQnix2H7Ngw/k4C5ijL5+65zddjncjaFoBhdsK/akog=
255
github.com/russross/blackfriday/v2 v2.0.1/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
256
github.com/russross/blackfriday/v2 v2.1.0 h1:JIOH55/0cWyOuilr9/qlrm0BSXldqnqwMsf35Ld67mk=
257
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
258
github.com/shurcooL/sanitized_anchor_name v1.0.0/go.mod h1:1NzhyTcUVG4SuEtjjoZeVRXNmyL/1OwPU0+IJeTBvfc=
259
github.com/smartystreets/assertions v1.2.0 h1:42S6lae5dvLc7BrLu/0ugRtcFVjoJNMC/N3yZFZkDFs=
260
github.com/smartystreets/assertions v1.2.0/go.mod h1:tcbTF8ujkAEcZ8TElKY+i30BzYlVhC/LOxJk7iOWnoo=
···
3
github.com/RussellLuo/slidingwindow v0.0.0-20200528002341-535bb99d338b/go.mod h1:4+EPqMRApwwE/6yo6CxiHoSnBzjRr3jsqer7frxP8y4=
4
github.com/alexbrainman/goissue34681 v0.0.0-20191006012335-3fc7a47baff5 h1:iW0a5ljuFxkLGPNem5Ui+KBjFJzKg4Fv2fnxe4dvzpM=
5
github.com/alexbrainman/goissue34681 v0.0.0-20191006012335-3fc7a47baff5/go.mod h1:Y2QMoi1vgtOIfc+6DhrMOGkLoGzqSV2rKp4Sm+opsyA=
6
+
github.com/araddon/dateparse v0.0.0-20210429162001-6b43995a97de h1:FxWPpzIjnTlhPwqqXc4/vE0f7GvRjuAsbW+HOIe8KnA=
7
+
github.com/araddon/dateparse v0.0.0-20210429162001-6b43995a97de/go.mod h1:DCaWoUhZrYW9p1lxo/cm8EmUOOzAPSEZNGF2DK1dJgw=
8
github.com/benbjohnson/clock v1.1.0/go.mod h1:J11/hYXuz8f4ySSvYwY0FKfm+ezbsZBKZxNJlLklBHA=
9
github.com/benbjohnson/clock v1.3.0 h1:ip6w0uFQkncKQ979AypyG0ER7mqUSBdKLOgAle/AT8A=
10
github.com/benbjohnson/clock v1.3.0/go.mod h1:J11/hYXuz8f4ySSvYwY0FKfm+ezbsZBKZxNJlLklBHA=
···
202
github.com/mattn/go-isatty v0.0.14/go.mod h1:7GGIvUiUoEMVVmxf/4nioHXj79iQHKdU27kJ6hsGG94=
203
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
204
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
205
+
github.com/mattn/go-runewidth v0.0.10/go.mod h1:RAqKPSqVFrSLVXbA8x7dzmKdmGzieGRCM46jaSJTDAk=
206
github.com/mattn/go-sqlite3 v1.14.22 h1:2gZY6PC6kBnID23Tichd1K+Z0oS6nE/XwU+Vz/5o4kU=
207
github.com/mattn/go-sqlite3 v1.14.22/go.mod h1:Uh1q+B4BYcTPb+yiD3kU8Ct7aC0hY9fxUwlHK0RXw+Y=
208
github.com/miekg/dns v1.1.50 h1:DQUfb9uc6smULcREF09Uc+/Gd46YWqJd5DbpPE9xkcA=
···
252
github.com/prometheus/common v0.66.1/go.mod h1:gcaUsgf3KfRSwHY4dIMXLPV0K/Wg1oZ8+SbZk/HH/dA=
253
github.com/prometheus/procfs v0.16.1 h1:hZ15bTNuirocR6u0JZ6BAHHmwS1p8B4P6MRqxtzMyRg=
254
github.com/prometheus/procfs v0.16.1/go.mod h1:teAbpZRB1iIAJYREa1LsoWUXykVXA1KlTmWl8x/U+Is=
255
+
github.com/rivo/uniseg v0.1.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
256
github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
257
github.com/rogpeppe/go-internal v1.10.0 h1:TMyTOH3F/DB16zRVcYyreMH6GnZZrwQVAoYjRBZyWFQ=
258
github.com/rogpeppe/go-internal v1.10.0/go.mod h1:UQnix2H7Ngw/k4C5ijL5+65zddjncjaFoBhdsK/akog=
259
github.com/russross/blackfriday/v2 v2.0.1/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
260
github.com/russross/blackfriday/v2 v2.1.0 h1:JIOH55/0cWyOuilr9/qlrm0BSXldqnqwMsf35Ld67mk=
261
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
262
+
github.com/scylladb/termtables v0.0.0-20191203121021-c4c0b6d42ff4/go.mod h1:C1a7PQSMz9NShzorzCiG2fk9+xuCgLkPeCvMHYR2OWg=
263
github.com/shurcooL/sanitized_anchor_name v1.0.0/go.mod h1:1NzhyTcUVG4SuEtjjoZeVRXNmyL/1OwPU0+IJeTBvfc=
264
github.com/smartystreets/assertions v1.2.0 h1:42S6lae5dvLc7BrLu/0ugRtcFVjoJNMC/N3yZFZkDFs=
265
github.com/smartystreets/assertions v1.2.0/go.mod h1:tcbTF8ujkAEcZ8TElKY+i30BzYlVhC/LOxJk7iOWnoo=