ptr.pet
forked from
microcosm.blue/Allegedly
Server tools to backfill, tail, mirror, and verify PLC logs
1# Allegedly
2
3Some [public ledger](https://github.com/did-method-plc/did-method-plc) tools and services for servers
4
5Allegedly can
6
7- Tail PLC ops to stdout: `allegedly tail | jq`
8- Export PLC ops to weekly gzipped bundles: `allegdly bundle --dest ./some-folder`
9- Dump bundled ops to stdout FAST: `allegedly backfill --source-workers 6 | pv -l > /ops-unordered.jsonl`
10- Wrap the reference PLC server and run it as a mirror, copying ops from upstream:
11
12 ```bash
13 allegedly mirror \
14 --wrap "http://127.0.0.1:3000" \
15 --wrap-pg "postgresql://user:pass@pg-host:5432/plc-db"
16 ```
17
18- Run a fully self-contained mirror using an embedded fjall database (no postgres or local plc server needed):
19
20 ```bash
21 # backfill first
22 allegedly backfill --no-bulk --to-fjall ./plc-data
23
24 # then run the mirror
25 allegedly mirror --wrap-fjall ./plc-data
26 ```
27
28- Wrap a plc server, maximalist edition:
29
30 ```bash
31 # put sensitive values in environment so they don't leak via process name.
32 export ALLEGEDLY_WRAP_PG="postgresql://user:pass@pg-host:5432/plc-db"
33
34 # sudo to bind :80 + :443 for acme tls, but it's better to give user net cap.
35 # will try to autoprovision cert for "plc.wtf" from letsencrypt staging.
36 sudo allegedly mirror \
37 --upstream "https://plc.directory" \
38 --wrap "http://127.0.0.1:3000" \
39 --wrap-pg-cert "/opt/allegedly/postgres-cert.pem" \
40 --acme-domain "plc.wtf" \
41 --acme-domain "alt.plc.wtf" \
42 --experimental-acme-domain "experimental.plc.wtf" \
43 --acme-cache-path ./acme-cache \
44 --acme-directory-url "https://acme-staging-v02.api.letsencrypt.org/directory" \
45 --acme-ipv6 \
46 --experimental-write-upstream
47 ```
48
49- Reverse-proxy to any PLC server, terminating TLS and forwarding writes upstream
50
51 ```bash
52 sudo allegedly wrap \
53 --wrap "http://127.0.0.1:3000" \
54 --acme-ipv6 \
55 --acme-cache-path ./acme-cache \
56 --acme-domain "plc.wtf" \
57 --experimental-acme-domain "experimental.plc.wtf" \
58 --experimental-write-upstream \
59 --upstream "https://plc.wtf" \
60 ```
61
62
63add `--help` to any command for more info about it
64
65
66## install
67
68```bash
69cargo install allegedly
70```
71
72the version on crates might be behind while new features are under development.
73to install the latest from source:
74
75- make sure you have rust/rustup set up
76- clone the repo
77- install
78
79 ```bash
80 cargo install --path . --bin allegedly
81 ```
82
83
84## future improvements
85
86### existing stuff
87
88- signals and shutdown handling
89- monitoring of the various tasks
90- health check pings
91- expose metrics/tracing
92- [x] read-only flag for mirror wrapper
93- bundle: write directly to s3-compatible object storage
94- helpers for automating periodic `bundle` runs
95
96
97### new things
98
99- [ ] experimental: websocket version of /export
100- [x] experimental: accept writes by forwarding them upstream
101- [ ] experimental: serve a tlog
102- [x] experimental: embed a log database directly for fast and efficient mirroring
103- [ ] experimental: support multiple upstreams?
104
105- [ ] new command todo: `zip` or `check` or `diff`: compare two plc logs over some time range
106- [ ] new command to consider: `scatter` or something: broadcast plc writes to multiple upstreams
107
108
109if you have an idea for a new command, [open a request](https://tangled.org/@microcosm.blue/Allegedly/issues/new)!
110
111
112## license
113
114This work is dual-licensed under MIT and Apache 2.0. You can choose between one of them if you use this work.
115
116`SPDX-License-Identifier: MIT OR Apache-2.0`