aleeve.dev
Scalable and distributed custom feed generator, ott - on that topic
Add cloudflare tunnel to figure out how to expose this from homelab
+9
-3
README.md
+9
-3
README.md
···
4
4
5
5
The flow is as follows:
6
6
7
-
1. Two fluvio connector services consume wss streams, one for raw-posts and one for raw-likes
8
-
2. ott-filter consumes the keyed posts, and likes streams and keeps count on likes and other filters.
7
+
1. Two fluvio connector services consume wss streams, one for raw-posts and one for raw-likes.
8
+
2. ott-filter consumes the keyed posts and likes streams, and keeps count on likes and other filters.
9
9
It sends the passing posts to the fluvio topic posts.
10
10
3. ott-embed consumes the posts topic, embeds them with tei running on host and stores the vectors in a pg cluster
11
-
4. ott-xrpc listens to getFeedSkeleton requests, gets the users last likeed post and gets similar posts from the pg db.
11
+
4. ott-xrpc listens to getFeedSkeleton requests, gets the users last liked post and gets similar posts from the pg db.
12
12
13
13
Still work in progress, especially the ott-xrpc service isn't fleshed out yet. Also I intend to add a VIP stream so that all posts
14
14
liked by a feed user are guaranteed to pass the filter.
···
17
17
This is still the intention.
18
18
19
19
# Setup
20
+
Note that the ott-helm currently assumes that you intend to deploy through a cloudflare zerotrust tunnel, and defaults to starting up a cloudflared daemon
21
+
forwarding to the ott-xrpc service. If your DNS isn't configured like that and you have no tunnel simply disable cloudflare in your helm values.
22
+
23
+
Also note that this should likely not be done at all just yet, this software is in early development.
20
24
21
25
## Install dependencies
22
26
···
46
50
```shell
47
51
skaffold run
48
52
```
53
+
54
+
+38
helm/ott/templates/cloudflared_tunnel.yaml
+38
helm/ott/templates/cloudflared_tunnel.yaml
···
1
+
{{- if .Values.cloudflare.enabled }}
2
+
apiVersion: v1
3
+
kind: Secret
4
+
metadata:
5
+
name: cloudflare-tunnel-token
6
+
namespace: {{ .Values.cloudflared.namespace }}
7
+
type: Opaque
8
+
stringData:
9
+
token: {{ required "cloudflared.token is required" .Values.cloudflared.token }}
10
+
---
11
+
apiVersion: apps/v1
12
+
kind: Deployment
13
+
metadata:
14
+
name: cloudflare-tunnel
15
+
spec:
16
+
replicas: 1
17
+
selector:
18
+
matchLabels:
19
+
app: cloudflare-tunnel
20
+
template:
21
+
metadata:
22
+
labels:
23
+
app: cloudflare-tunnel
24
+
spec:
25
+
containers:
26
+
- name: cloudflared
27
+
image: "cloudflare/cloudflared:latest"
28
+
args:
29
+
- tunnel
30
+
- run
31
+
- --token=$(CLOUDFLARE_TUNNEL_TOKEN)
32
+
env:
33
+
- name: CLOUDFLARE_TUNNEL_TOKEN
34
+
valueFrom:
35
+
secretKeyRef:
36
+
name: cloudflare-tunnel-token
37
+
key: token
38
+
{{- end }}
+16
-1
helm/ott/templates/deployments.yaml
helm/ott/templates/services.yaml
+16
-1
helm/ott/templates/deployments.yaml
helm/ott/templates/services.yaml
···
1
-
{{- range .Values.deployments }}
1
+
{{- range .Values.services }}
2
2
---
3
3
apiVersion: apps/v1
4
4
kind: Deployment
···
22
22
imagePullPolicy: {{ .image.pullPolicy }}
23
23
env:
24
24
{{- toYaml .env | nindent 10 }}
25
+
---
26
+
apiVersion: v1
27
+
kind: Service
28
+
metadata:
29
+
name: {{ .name }}
30
+
labels:
31
+
app: {{ .name }}
32
+
spec:
33
+
selector:
34
+
app: {{ .name }}
35
+
ports:
36
+
- port: 80
37
+
targetPort: 8080
38
+
protocol: TCP
39
+
type: ClusterIP
25
40
{{- end }}
+4
-1
helm/ott/values.yaml
+4
-1
helm/ott/values.yaml
+6
-5
skaffold.yaml
+6
-5
skaffold.yaml
···
85
85
valuesFiles:
86
86
- helm/ott/values.yaml
87
87
setValueTemplates:
88
+
cloudflared.token: "{{.CLOUDFLARE_TUNNEL_TOKEN}}"
88
89
postgresql.migration_image_fqn: "{{.IMAGE_FULLY_QUALIFIED_migration_pg}}"
89
-
deployments[0].image.fqn: "{{.IMAGE_FULLY_QUALIFIED_likes_connector}}"
90
-
deployments[1].image.fqn: "{{.IMAGE_FULLY_QUALIFIED_posts_connector}}"
91
-
deployments[2].image.fqn: "{{.IMAGE_FULLY_QUALIFIED_ott_filter}}"
92
-
deployments[3].image.fqn: "{{.IMAGE_FULLY_QUALIFIED_ott_embed}}"
93
-
deployments[4].image.fqn: "{{.IMAGE_FULLY_QUALIFIED_ott_xrpc}}"
90
+
services[0].image.fqn: "{{.IMAGE_FULLY_QUALIFIED_likes_connector}}"
91
+
services[1].image.fqn: "{{.IMAGE_FULLY_QUALIFIED_posts_connector}}"
92
+
services[2].image.fqn: "{{.IMAGE_FULLY_QUALIFIED_ott_filter}}"
93
+
services[3].image.fqn: "{{.IMAGE_FULLY_QUALIFIED_ott_embed}}"
94
+
services[4].image.fqn: "{{.IMAGE_FULLY_QUALIFIED_ott_xrpc}}"
94
95
version: 0.1.0