lmao.bsky.social / knot-iroh
Lightweight decentralized “knot” server prototype using Iroh + ATProto.
fork atom
Rust 97.6%
Shell 2.4%
68 1 0

Clone this repository

https://tangled.org/lmao.bsky.social/knot-iroh https://tangled.org/did:plc:iu35o4qokbffmkeor7e7hvuj/knot-iroh
git@tangled.org:lmao.bsky.social/knot-iroh git@tangled.org:did:plc:iu35o4qokbffmkeor7e7hvuj/knot-iroh

For self-hosted knots, clone URLs may differ based on your setup.

Download tar.gz
README.md

knot-iroh#

Lightweight decentralized “knot” server prototype using Iroh + ATProto.

What it is#

knot-iroh is an experiment in decentralizing a Git “knot” server:

  • Iroh stores and distributes pack/manifest blobs.
  • ATProto holds the canonical head pointer and peer hints.
  • A git:// daemon serves repos via standard Git clients.

Single‑user repos, single canonical head, and no force‑push are the current constraints.

Status (2026‑01‑27)#

Working pieces:

  • Iroh blob exchange + manifest persistence.
  • Head → manifest → pack materialization flow.
  • Repo registry (default: ~/.local/share/knot-iroh/registry.json) + git:// daemon (git-upload-pack and git-receive-pack).
  • Fast‑forward enforcement (no force push).
  • Background sync inside the daemon (file watching + periodic refresh).
  • ATProto record fetch (daemon can read a real PDS via com.atproto.repo.getRecord).
  • End‑to‑end git clone / pull / push over the daemon (optional tests).
  • Handle-first register UX:
    • register --handle <handle> lists repos (does not register).
    • register --handle <handle> --repo-name <name> registers a single repo.
    • --repo expects a full DID-style repo id (did:.../name) and cannot be combined with --handle.

Still to do:

  • ATProto write‑back for new heads/manifests (networked putRecord).
  • Better service abstractions and per‑repo locking.

Code map (high‑level)#

  • src/git_daemon.rs: git:// daemon entry point and connection handling.
  • src/git_service.rs: shared “prepare repo” orchestration (resolve → sync → materialize).
  • src/repo_resolver.rs + src/registry.rs: registry loading + path resolution.
  • src/cache.rs: repo cache + bare repo materialization from packs.
  • src/flow.rs: head → manifest → pack materialization from peers.

Quick start#

Build:

cargo build

By default, runtime state lives under ~/.local/share/knot-iroh. For a self-contained demo, set a local data dir once per shell:

export KNOT_DATA_DIR="$PWD/.knot-iroh"

Create a repo and publish the initial commit:

mkdir -p demo-repo
cd demo-repo
git init
echo "hello from demo" > README.md
git add README.md
git commit -m "init"
cd ..

./target/debug/knot-iroh publish \
  --repo-id did:plc:example/repo \
  --repo-path demo-repo \
  --git-path /example.git

Start the git daemon (includes background sync + seeding):

./target/debug/knot-iroh daemon --addr 127.0.0.1:9418

Clone via git://:

git clone git://127.0.0.1:9418/example.git

Register a remote repo by handle + name (follower machine):

./target/debug/knot-iroh register --handle alice.bsky.social
./target/debug/knot-iroh register --handle alice.bsky.social --repo-name example
./target/debug/knot-iroh daemon
git clone git://127.0.0.1:9418/example.git

Real ATProto smoke test (bsky.social)#

This uses a real PDS and reads your app password from a local file so it doesn't end up in the repo.

  1. Put your app password in a safe local file:
mkdir -p ~/.local/share/knot-iroh
printf '%s\n' "xxxx-xxxx-xxxx-xxxx" > ~/.local/share/knot-iroh/password
chmod 600 ~/.local/share/knot-iroh/password
  1. Set your handle:
export KNOT_ATP_IDENTIFIER="your.handle.bsky.social"
  1. Run the smoke script (it will create a repo, publish it locally, push the record to ATProto, and print the head/manifest hashes plus the repo name/path):
scripts/atproto_real_smoke.sh

You can also pass a repo name:

scripts/atproto_real_smoke.sh my-repo-name

On a follower machine, you can discover and register the repo via handle:

./target/debug/knot-iroh register --handle "$KNOT_ATP_IDENTIFIER"
./target/debug/knot-iroh register --handle "$KNOT_ATP_IDENTIFIER" --repo-name my-repo-name
./target/debug/knot-iroh daemon
git clone git://127.0.0.1:9418/my-repo-name.git

Notes:

  • Password location is configurable via KNOT_ATP_PASSWORD_FILE.
  • Host defaults to https://bsky.social but can be overridden with KNOT_ATP_HOST.
  • Runtime state defaults to ~/.local/share/knot-iroh; set KNOT_DATA_DIR to override.
  • The daemon will read ATProto in the background when the registry entry contains ATProto locator info.

Push + pull demo (4 terminals)#

This walkthrough shows: create a local repo → publish initial pack/manifest → seed blobs → push via daemon → pull from another client.

In each terminal, point knot-iroh at a local demo data dir:

export KNOT_DATA_DIR="$PWD/.knot-iroh-demo"
record_path="$KNOT_DATA_DIR/records/did_plc_example_repo.json"

Terminal 1 (create a repo and publish the initial commit):

mkdir -p demo-repo
cd demo-repo
git init
echo "hello from demo" > README.md
git add README.md
git commit -m "init"
cd ..

./target/debug/knot-iroh publish \
  --repo-id did:plc:example/repo \
  --repo-path demo-repo \
  --git-path /example.git

Terminal 2 (seed the published blobs via Iroh; keep this running):

./target/debug/knot-iroh seed --record-path "$record_path"

The seed process is optional now; the daemon can do background seeding. It's still useful if you want a dedicated seeding process.

Terminal 3 (start daemon):

./target/debug/knot-iroh daemon --addr 127.0.0.1:9418

Terminal 4 (clone + push a new commit):

git clone git://127.0.0.1:9418/example.git repo-a
cd repo-a
echo "update from repo-a" >> README.md
git add README.md
git commit -m "update"
git push origin main

Terminal 4 (pull the new head):

git clone git://127.0.0.1:9418/example.git repo-b
cd repo-b
git pull --ff-only

Optional: run all three steps with one script:

KNOT_DATA_DIR="$PWD/data-e2e" example/run_e2e.sh

Notes:

  • The daemon supports git-upload-pack and git-receive-pack. Pushes generate a fresh pack + manifest, upload them to Iroh, and write back the local ATProto record file. (Real networked ATProto write‑back is still TODO.)
  • Registering a repo updates $KNOT_DATA_DIR/registry.json so the daemon can serve it without restarting.
  • Optional test: cargo test --test git_clone_e2e -- --ignored
  • Optional test: cargo test --test git_push_e2e -- --ignored
  • Optional test: cargo test --test git_pull_e2e -- --ignored

HTTP server (optional, debug-only)#

The HTTP server is still available for health/status checks and the manual /sync endpoint, but the preferred UX is the git:// daemon (it will sync on demand).

Run the server:

export KNOT_DATA_DIR="$PWD/.knot-iroh-http"
./target/debug/knot-iroh serve --addr 127.0.0.1:8080

Health + status:

curl -i http://127.0.0.1:8080/health
curl -s http://127.0.0.1:8080/status

Trigger a sync (uses a local ATProto record file):

record_path="$KNOT_DATA_DIR/records/did_plc_example_repo.json"
curl -X POST http://127.0.0.1:8080/sync \
  -H 'Content-Type: application/json' \
  -d "{\"repo_id\":\"did:plc:example/repo\",\"record_path\":\"$record_path\"}"

Example record#

See example/record.json. Replace:

  • head: the hash of the head blob (Iroh hash, hex or base32)
  • peers[0].node_id: the peer’s Iroh node id (base32)
  • peers[0].addrs: the peer’s direct address(es), e.g. 127.0.0.1:12345

Default record paths live under $KNOT_DATA_DIR/records/ (or ~/.local/share/knot-iroh/records/ if unset).

If you generate a head/manifest in code, write the JSON record to disk and call /sync with the path.