lewis.moe
Monorepo for Tangled
1---
2title: Tangled docs
3author: The Tangled Contributors
4date: 21 Sun, Dec 2025
5abstract: |
6 Tangled is a decentralized code hosting and collaboration
7 platform. Every component of Tangled is open-source and
8 self-hostable. [tangled.org](https://tangled.org) also
9 provides hosting and CI services that are free to use.
10
11 There are several models for decentralized code
12 collaboration platforms, ranging from ActivityPub’s
13 (Forgejo) federated model, to Radicle’s entirely P2P model.
14 Our approach attempts to be the best of both worlds by
15 adopting the AT Protocol—a protocol for building decentralized
16 social applications with a central identity
17
18 Our approach to this is the idea of “knots”. Knots are
19 lightweight, headless servers that enable users to host Git
20 repositories with ease. Knots are designed for either single
21 or multi-tenant use which is perfect for self-hosting on a
22 Raspberry Pi at home, or larger “community” servers. By
23 default, Tangled provides managed knots where you can host
24 your repositories for free.
25
26 The appview at tangled.org acts as a consolidated "view"
27 into the whole network, allowing users to access, clone and
28 contribute to repositories hosted across different knots
29 seamlessly.
30---
31
32# Quick start guide
33
34## Login or sign up
35
36You can [login](https://tangled.org) by using your AT Protocol
37account. If you are unclear on what that means, simply head
38to the [signup](https://tangled.org/signup) page and create
39an account. By doing so, you will be choosing Tangled as
40your account provider (you will be granted a handle of the
41form `user.tngl.sh`).
42
43In the AT Protocol network, users are free to choose their account
44provider (known as a "Personal Data Service", or PDS), and
45login to applications that support AT accounts.
46
47You can think of it as "one account for all of the atmosphere"!
48
49If you already have an AT account (you may have one if you
50signed up to Bluesky, for example), you can login with the
51same handle on Tangled (so just use `user.bsky.social` on
52the login page).
53
54## Add an SSH key
55
56Once you are logged in, you can start creating repositories
57and pushing code. Tangled supports pushing git repositories
58over SSH.
59
60First, you'll need to generate an SSH key if you don't
61already have one:
62
63```bash
64ssh-keygen -t ed25519 -C "foo@bar.com"
65```
66
67When prompted, save the key to the default location
68(`~/.ssh/id_ed25519`) and optionally set a passphrase.
69
70Copy your public key to your clipboard:
71
72```bash
73# on X11
74cat ~/.ssh/id_ed25519.pub | xclip -sel c
75
76# on wayland
77cat ~/.ssh/id_ed25519.pub | wl-copy
78
79# on macos
80cat ~/.ssh/id_ed25519.pub | pbcopy
81```
82
83Now, navigate to 'Settings' -> 'Keys' and hit 'Add Key',
84paste your public key, give it a descriptive name, and hit
85save.
86
87## Create a repository
88
89Once your SSH key is added, create your first repository:
90
911. Hit the green `+` icon on the topbar, and select
92 repository
932. Enter a repository name
943. Add a description
954. Choose a knotserver to host this repository on
965. Hit create
97
98Knots are self-hostable, lightweight Git servers that can
99host your repository. Unlike traditional code forges, your
100code can live on any server. Read the [Knots](TODO) section
101for more.
102
103## Configure SSH
104
105To ensure Git uses the correct SSH key and connects smoothly
106to Tangled, add this configuration to your `~/.ssh/config`
107file:
108
109```
110Host tangled.org
111 Hostname tangled.org
112 User git
113 IdentityFile ~/.ssh/id_ed25519
114 AddressFamily inet
115```
116
117This tells SSH to use your specific key when connecting to
118Tangled and prevents authentication issues if you have
119multiple SSH keys.
120
121Note that this configuration only works for knotservers that
122are hosted by tangled.org. If you use a custom knot, refer
123to the [Knots](TODO) section.
124
125## Push your first repository
126
127Initialize a new Git repository:
128
129```bash
130mkdir my-project
131cd my-project
132
133git init
134echo "# My Project" > README.md
135```
136
137Add some content and push!
138
139```bash
140git add README.md
141git commit -m "Initial commit"
142git remote add origin git@tangled.org:user.tngl.sh/my-project
143git push -u origin main
144```
145
146That's it! Your code is now hosted on Tangled.
147
148## Migrating an existing repository
149
150Moving your repositories from GitHub, GitLab, Bitbucket, or
151any other Git forge to Tangled is straightforward. You'll
152simply change your repository's remote URL. At the moment,
153Tangled does not have any tooling to migrate data such as
154GitHub issues or pull requests.
155
156First, create a new repository on tangled.org as described
157in the [Quick Start Guide](#create-a-repository).
158
159Navigate to your existing local repository:
160
161```bash
162cd /path/to/your/existing/repo
163```
164
165You can inspect your existing Git remote like so:
166
167```bash
168git remote -v
169```
170
171You'll see something like:
172
173```
174origin git@github.com:username/my-project (fetch)
175origin git@github.com:username/my-project (push)
176```
177
178Update the remote URL to point to tangled:
179
180```bash
181git remote set-url origin git@tangled.org:user.tngl.sh/my-project
182```
183
184Verify the change:
185
186```bash
187git remote -v
188```
189
190You should now see:
191
192```
193origin git@tangled.org:user.tngl.sh/my-project (fetch)
194origin git@tangled.org:user.tngl.sh/my-project (push)
195```
196
197Push all your branches and tags to Tangled:
198
199```bash
200git push -u origin --all
201git push -u origin --tags
202```
203
204Your repository is now migrated to Tangled! All commit
205history, branches, and tags have been preserved.
206
207## Mirroring a repository to Tangled
208
209If you want to maintain your repository on multiple forges
210simultaneously, for example, keeping your primary repository
211on GitHub while mirroring to Tangled for backup or
212redundancy, you can do so by adding multiple remotes.
213
214You can configure your local repository to push to both
215Tangled and, say, GitHub. You may already have the following
216setup:
217
218```
219$ git remote -v
220origin git@github.com:username/my-project (fetch)
221origin git@github.com:username/my-project (push)
222```
223
224Now add Tangled as an additional push URL to the same
225remote:
226
227```bash
228git remote set-url --add --push origin git@tangled.org:user.tngl.sh/my-project
229```
230
231You also need to re-add the original URL as a push
232destination (Git replaces the push URL when you use `--add`
233the first time):
234
235```bash
236git remote set-url --add --push origin git@github.com:username/my-project
237```
238
239Verify your configuration:
240
241```
242$ git remote -v
243origin git@github.com:username/repo (fetch)
244origin git@tangled.org:username/my-project (push)
245origin git@github.com:username/repo (push)
246```
247
248Notice that there's one fetch URL (the primary remote) and
249two push URLs. Now, whenever you push, Git will
250automatically push to both remotes:
251
252```bash
253git push origin main
254```
255
256This single command pushes your `main` branch to both GitHub
257and Tangled simultaneously.
258
259To push all branches and tags:
260
261```bash
262git push origin --all
263git push origin --tags
264```
265
266If you prefer more control over which remote you push to,
267you can maintain separate remotes:
268
269```bash
270git remote add github git@github.com:username/my-project
271git remote add tangled git@tangled.org:username/my-project
272```
273
274Then push to each explicitly:
275
276```bash
277git push github main
278git push tangled main
279```
280
281# Knot self-hosting guide
282
283So you want to run your own knot server? Great! Here are a few prerequisites:
284
2851. A server of some kind (a VPS, a Raspberry Pi, etc.). Preferably running a Linux distribution of some kind.
2862. A (sub)domain name. People generally use `knot.example.com`.
2873. A valid SSL certificate for your domain.
288
289## NixOS
290
291Refer to the [knot
292module](https://tangled.org/tangled.org/core/blob/master/nix/modules/knot.nix)
293for a full list of options. Sample configurations:
294
295- [The test VM](https://tangled.org/tangled.org/core/blob/master/nix/vm.nix#L85)
296- [@pyrox.dev/nix](https://tangled.org/pyrox.dev/nix/blob/d19571cc1b5fe01035e1e6951ec8cf8a476b4dee/hosts/marvin/services/tangled.nix#L15-25)
297
298## Docker
299
300Refer to
301[@tangled.org/knot-docker](https://tangled.org/@tangled.org/knot-docker).
302Note that this is community maintained.
303
304## Manual setup
305
306First, clone this repository:
307
308```
309git clone https://tangled.org/@tangled.org/core
310```
311
312Then, build the `knot` CLI. This is the knot administration
313and operation tool. For the purpose of this guide, we're
314only concerned with these subcommands:
315
316 * `knot server`: the main knot server process, typically
317 run as a supervised service
318 * `knot guard`: handles role-based access control for git
319 over SSH (you'll never have to run this yourself)
320 * `knot keys`: fetches SSH keys associated with your knot;
321 we'll use this to generate the SSH
322 `AuthorizedKeysCommand`
323
324```
325cd core
326export CGO_ENABLED=1
327go build -o knot ./cmd/knot
328```
329
330Next, move the `knot` binary to a location owned by `root` --
331`/usr/local/bin/` is a good choice. Make sure the binary itself is also owned by `root`:
332
333```
334sudo mv knot /usr/local/bin/knot
335sudo chown root:root /usr/local/bin/knot
336```
337
338This is necessary because SSH `AuthorizedKeysCommand` requires [really
339specific permissions](https://stackoverflow.com/a/27638306). The
340`AuthorizedKeysCommand` specifies a command that is run by `sshd` to
341retrieve a user's public SSH keys dynamically for authentication. Let's
342set that up.
343
344```
345sudo tee /etc/ssh/sshd_config.d/authorized_keys_command.conf <<EOF
346Match User git
347 AuthorizedKeysCommand /usr/local/bin/knot keys -o authorized-keys
348 AuthorizedKeysCommandUser nobody
349EOF
350```
351
352Then, reload `sshd`:
353
354```
355sudo systemctl reload ssh
356```
357
358Next, create the `git` user. We'll use the `git` user's home directory
359to store repositories:
360
361```
362sudo adduser git
363```
364
365Create `/home/git/.knot.env` with the following, updating the values as
366necessary. The `KNOT_SERVER_OWNER` should be set to your
367DID, you can find your DID in the [Settings](https://tangled.sh/settings) page.
368
369```
370KNOT_REPO_SCAN_PATH=/home/git
371KNOT_SERVER_HOSTNAME=knot.example.com
372APPVIEW_ENDPOINT=https://tangled.org
373KNOT_SERVER_OWNER=did:plc:foobar
374KNOT_SERVER_INTERNAL_LISTEN_ADDR=127.0.0.1:5444
375KNOT_SERVER_LISTEN_ADDR=127.0.0.1:5555
376```
377
378If you run a Linux distribution that uses systemd, you can use the provided
379service file to run the server. Copy
380[`knotserver.service`](/systemd/knotserver.service)
381to `/etc/systemd/system/`. Then, run:
382
383```
384systemctl enable knotserver
385systemctl start knotserver
386```
387
388The last step is to configure a reverse proxy like Nginx or Caddy to front your
389knot. Here's an example configuration for Nginx:
390
391```
392server {
393 listen 80;
394 listen [::]:80;
395 server_name knot.example.com;
396
397 location / {
398 proxy_pass http://localhost:5555;
399 proxy_set_header Host $host;
400 proxy_set_header X-Real-IP $remote_addr;
401 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
402 proxy_set_header X-Forwarded-Proto $scheme;
403 }
404
405 # wss endpoint for git events
406 location /events {
407 proxy_set_header X-Forwarded-For $remote_addr;
408 proxy_set_header Host $http_host;
409 proxy_set_header Upgrade websocket;
410 proxy_set_header Connection Upgrade;
411 proxy_pass http://localhost:5555;
412 }
413 # additional config for SSL/TLS go here.
414}
415
416```
417
418Remember to use Let's Encrypt or similar to procure a certificate for your
419knot domain.
420
421You should now have a running knot server! You can finalize
422your registration by hitting the `verify` button on the
423[/settings/knots](https://tangled.org/settings/knots) page. This simply creates
424a record on your PDS to announce the existence of the knot.
425
426### Custom paths
427
428(This section applies to manual setup only. Docker users should edit the mounts
429in `docker-compose.yml` instead.)
430
431Right now, the database and repositories of your knot lives in `/home/git`. You
432can move these paths if you'd like to store them in another folder. Be careful
433when adjusting these paths:
434
435* Stop your knot when moving data (e.g. `systemctl stop knotserver`) to prevent
436any possible side effects. Remember to restart it once you're done.
437* Make backups before moving in case something goes wrong.
438* Make sure the `git` user can read and write from the new paths.
439
440#### Database
441
442As an example, let's say the current database is at `/home/git/knotserver.db`,
443and we want to move it to `/home/git/database/knotserver.db`.
444
445Copy the current database to the new location. Make sure to copy the `.db-shm`
446and `.db-wal` files if they exist.
447
448```
449mkdir /home/git/database
450cp /home/git/knotserver.db* /home/git/database
451```
452
453In the environment (e.g. `/home/git/.knot.env`), set `KNOT_SERVER_DB_PATH` to
454the new file path (_not_ the directory):
455
456```
457KNOT_SERVER_DB_PATH=/home/git/database/knotserver.db
458```
459
460#### Repositories
461
462As an example, let's say the repositories are currently in `/home/git`, and we
463want to move them into `/home/git/repositories`.
464
465Create the new folder, then move the existing repositories (if there are any):
466
467```
468mkdir /home/git/repositories
469# move all DIDs into the new folder; these will vary for you!
470mv /home/git/did:plc:wshs7t2adsemcrrd4snkeqli /home/git/repositories
471```
472
473In the environment (e.g. `/home/git/.knot.env`), update `KNOT_REPO_SCAN_PATH`
474to the new directory:
475
476```
477KNOT_REPO_SCAN_PATH=/home/git/repositories
478```
479
480Similarly, update your `sshd` `AuthorizedKeysCommand` to use the updated
481repository path:
482
483```
484sudo tee /etc/ssh/sshd_config.d/authorized_keys_command.conf <<EOF
485Match User git
486 AuthorizedKeysCommand /usr/local/bin/knot keys -o authorized-keys -git-dir /home/git/repositories
487 AuthorizedKeysCommandUser nobody
488EOF
489```
490
491Make sure to restart your SSH server!
492
493#### MOTD (message of the day)
494
495To configure the MOTD used ("Welcome to this knot!" by default), edit the
496`/home/git/motd` file:
497
498```
499printf "Hi from this knot!\n" > /home/git/motd
500```
501
502Note that you should add a newline at the end if setting a non-empty message
503since the knot won't do this for you.
504
505# Spindles
506
507## Pipelines
508
509Spindle workflows allow you to write CI/CD pipelines in a
510simple format. They're located in the `.tangled/workflows`
511directory at the root of your repository, and are defined
512using YAML.
513
514The fields are:
515
516- [Trigger](#trigger): A **required** field that defines
517 when a workflow should be triggered.
518- [Engine](#engine): A **required** field that defines which
519 engine a workflow should run on.
520- [Clone options](#clone-options): An **optional** field
521 that defines how the repository should be cloned.
522- [Dependencies](#dependencies): An **optional** field that
523 allows you to list dependencies you may need.
524- [Environment](#environment): An **optional** field that
525 allows you to define environment variables.
526- [Steps](#steps): An **optional** field that allows you to
527 define what steps should run in the workflow.
528
529### Trigger
530
531The first thing to add to a workflow is the trigger, which
532defines when a workflow runs. This is defined using a `when`
533field, which takes in a list of conditions. Each condition
534has the following fields:
535
536- `event`: This is a **required** field that defines when
537 your workflow should run. It's a list that can take one or
538 more of the following values:
539 - `push`: The workflow should run every time a commit is
540 pushed to the repository.
541 - `pull_request`: The workflow should run every time a
542 pull request is made or updated.
543 - `manual`: The workflow can be triggered manually.
544- `branch`: Defines which branches the workflow should run
545 for. If used with the `push` event, commits to the
546 branch(es) listed here will trigger the workflow. If used
547 with the `pull_request` event, updates to pull requests
548 targeting the branch(es) listed here will trigger the
549 workflow. This field has no effect with the `manual`
550 event. Supports glob patterns using `*` and `**` (e.g.,
551 `main`, `develop`, `release-*`). Either `branch` or `tag`
552 (or both) must be specified for `push` events.
553- `tag`: Defines which tags the workflow should run for.
554 Only used with the `push` event - when tags matching the
555 pattern(s) listed here are pushed, the workflow will
556 trigger. This field has no effect with `pull_request` or
557 `manual` events. Supports glob patterns using `*` and `**`
558 (e.g., `v*`, `v1.*`, `release-**`). Either `branch` or
559 `tag` (or both) must be specified for `push` events.
560
561For example, if you'd like to define a workflow that runs
562when commits are pushed to the `main` and `develop`
563branches, or when pull requests that target the `main`
564branch are updated, or manually, you can do so with:
565
566```yaml
567when:
568 - event: ["push", "manual"]
569 branch: ["main", "develop"]
570 - event: ["pull_request"]
571 branch: ["main"]
572```
573
574You can also trigger workflows on tag pushes. For instance,
575to run a deployment workflow when tags matching `v*` are
576pushed:
577
578```yaml
579when:
580 - event: ["push"]
581 tag: ["v*"]
582```
583
584You can even combine branch and tag patterns in a single
585constraint (the workflow triggers if either matches):
586
587```yaml
588when:
589 - event: ["push"]
590 branch: ["main", "release-*"]
591 tag: ["v*", "stable"]
592```
593
594### Engine
595
596Next is the engine on which the workflow should run, defined
597using the **required** `engine` field. The currently
598supported engines are:
599
600- `nixery`: This uses an instance of
601 [Nixery](https://nixery.dev) to run steps, which allows
602 you to add [dependencies](#dependencies) from
603 Nixpkgs (https://github.com/NixOS/nixpkgs). You can
604 search for packages on https://search.nixos.org, and
605 there's a pretty good chance the package(s) you're looking
606 for will be there.
607
608Example:
609
610```yaml
611engine: "nixery"
612```
613
614### Clone options
615
616When a workflow starts, the first step is to clone the
617repository. You can customize this behavior using the
618**optional** `clone` field. It has the following fields:
619
620- `skip`: Setting this to `true` will skip cloning the
621 repository. This can be useful if your workflow is doing
622 something that doesn't require anything from the
623 repository itself. This is `false` by default.
624- `depth`: This sets the number of commits, or the "clone
625 depth", to fetch from the repository. For example, if you
626 set this to 2, the last 2 commits will be fetched. By
627 default, the depth is set to 1, meaning only the most
628 recent commit will be fetched, which is the commit that
629 triggered the workflow.
630- `submodules`: If you use Git submodules
631 (https://git-scm.com/book/en/v2/Git-Tools-Submodules)
632 in your repository, setting this field to `true` will
633 recursively fetch all submodules. This is `false` by
634 default.
635
636The default settings are:
637
638```yaml
639clone:
640 skip: false
641 depth: 1
642 submodules: false
643```
644
645### Dependencies
646
647Usually when you're running a workflow, you'll need
648additional dependencies. The `dependencies` field lets you
649define which dependencies to get, and from where. It's a
650key-value map, with the key being the registry to fetch
651dependencies from, and the value being the list of
652dependencies to fetch.
653
654Say you want to fetch Node.js and Go from `nixpkgs`, and a
655package called `my_pkg` you've made from your own registry
656at your repository at
657`https://tangled.org/@example.com/my_pkg`. You can define
658those dependencies like so:
659
660```yaml
661dependencies:
662 # nixpkgs
663 nixpkgs:
664 - nodejs
665 - go
666 # unstable
667 nixpkgs/nixpkgs-unstable:
668 - bun
669 # custom registry
670 git+https://tangled.org/@example.com/my_pkg:
671 - my_pkg
672```
673
674Now these dependencies are available to use in your
675workflow!
676
677### Environment
678
679The `environment` field allows you define environment
680variables that will be available throughout the entire
681workflow. **Do not put secrets here, these environment
682variables are visible to anyone viewing the repository. You
683can add secrets for pipelines in your repository's
684settings.**
685
686Example:
687
688```yaml
689environment:
690 GOOS: "linux"
691 GOARCH: "arm64"
692 NODE_ENV: "production"
693 MY_ENV_VAR: "MY_ENV_VALUE"
694```
695
696### Steps
697
698The `steps` field allows you to define what steps should run
699in the workflow. It's a list of step objects, each with the
700following fields:
701
702- `name`: This field allows you to give your step a name.
703 This name is visible in your workflow runs, and is used to
704 describe what the step is doing.
705- `command`: This field allows you to define a command to
706 run in that step. The step is run in a Bash shell, and the
707 logs from the command will be visible in the pipelines
708 page on the Tangled website. The
709 [dependencies](#dependencies) you added will be available
710 to use here.
711- `environment`: Similar to the global
712 [environment](#environment) config, this **optional**
713 field is a key-value map that allows you to set
714 environment variables for the step. **Do not put secrets
715 here, these environment variables are visible to anyone
716 viewing the repository. You can add secrets for pipelines
717 in your repository's settings.**
718
719Example:
720
721```yaml
722steps:
723 - name: "Build backend"
724 command: "go build"
725 environment:
726 GOOS: "darwin"
727 GOARCH: "arm64"
728 - name: "Build frontend"
729 command: "npm run build"
730 environment:
731 NODE_ENV: "production"
732```
733
734### Complete workflow
735
736```yaml
737# .tangled/workflows/build.yml
738
739when:
740 - event: ["push", "manual"]
741 branch: ["main", "develop"]
742 - event: ["pull_request"]
743 branch: ["main"]
744
745engine: "nixery"
746
747# using the default values
748clone:
749 skip: false
750 depth: 1
751 submodules: false
752
753dependencies:
754 # nixpkgs
755 nixpkgs:
756 - nodejs
757 - go
758 # custom registry
759 git+https://tangled.org/@example.com/my_pkg:
760 - my_pkg
761
762environment:
763 GOOS: "linux"
764 GOARCH: "arm64"
765 NODE_ENV: "production"
766 MY_ENV_VAR: "MY_ENV_VALUE"
767
768steps:
769 - name: "Build backend"
770 command: "go build"
771 environment:
772 GOOS: "darwin"
773 GOARCH: "arm64"
774 - name: "Build frontend"
775 command: "npm run build"
776 environment:
777 NODE_ENV: "production"
778```
779
780If you want another example of a workflow, you can look at
781the one [Tangled uses to build the
782project](https://tangled.org/@tangled.org/core/blob/master/.tangled/workflows/build.yml).
783
784## Self-hosting guide
785
786### Prerequisites
787
788* Go
789* Docker (the only supported backend currently)
790
791### Configuration
792
793Spindle is configured using environment variables. The following environment variables are available:
794
795* `SPINDLE_SERVER_LISTEN_ADDR`: The address the server listens on (default: `"0.0.0.0:6555"`).
796* `SPINDLE_SERVER_DB_PATH`: The path to the SQLite database file (default: `"spindle.db"`).
797* `SPINDLE_SERVER_HOSTNAME`: The hostname of the server (required).
798* `SPINDLE_SERVER_JETSTREAM_ENDPOINT`: The endpoint of the Jetstream server (default: `"wss://jetstream1.us-west.bsky.network/subscribe"`).
799* `SPINDLE_SERVER_DEV`: A boolean indicating whether the server is running in development mode (default: `false`).
800* `SPINDLE_SERVER_OWNER`: The DID of the owner (required).
801* `SPINDLE_PIPELINES_NIXERY`: The Nixery URL (default: `"nixery.tangled.sh"`).
802* `SPINDLE_PIPELINES_WORKFLOW_TIMEOUT`: The default workflow timeout (default: `"5m"`).
803* `SPINDLE_PIPELINES_LOG_DIR`: The directory to store workflow logs (default: `"/var/log/spindle"`).
804
805### Running spindle
806
8071. **Set the environment variables.** For example:
808
809 ```shell
810 export SPINDLE_SERVER_HOSTNAME="your-hostname"
811 export SPINDLE_SERVER_OWNER="your-did"
812 ```
813
8142. **Build the Spindle binary.**
815
816 ```shell
817 cd core
818 go mod download
819 go build -o cmd/spindle/spindle cmd/spindle/main.go
820 ```
821
8223. **Create the log directory.**
823
824 ```shell
825 sudo mkdir -p /var/log/spindle
826 sudo chown $USER:$USER -R /var/log/spindle
827 ```
828
8294. **Run the Spindle binary.**
830
831 ```shell
832 ./cmd/spindle/spindle
833 ```
834
835Spindle will now start, connect to the Jetstream server, and begin processing pipelines.
836
837## Architecture
838
839Spindle is a small CI runner service. Here's a high-level overview of how it operates:
840
841* Listens for [`sh.tangled.spindle.member`](/lexicons/spindle/member.json) and
842[`sh.tangled.repo`](/lexicons/repo.json) records on the Jetstream.
843* When a new repo record comes through (typically when you add a spindle to a
844repo from the settings), spindle then resolves the underlying knot and
845subscribes to repo events (see:
846[`sh.tangled.pipeline`](/lexicons/pipeline.json)).
847* The spindle engine then handles execution of the pipeline, with results and
848logs beamed on the spindle event stream over WebSocket
849
850### The engine
851
852At present, the only supported backend is Docker (and Podman, if Docker
853compatibility is enabled, so that `/run/docker.sock` is created). spindle
854executes each step in the pipeline in a fresh container, with state persisted
855across steps within the `/tangled/workspace` directory.
856
857The base image for the container is constructed on the fly using
858[Nixery](https://nixery.dev), which is handy for caching layers for frequently
859used packages.
860
861The pipeline manifest is [specified here](https://docs.tangled.org/spindles.html#pipelines).
862
863## Secrets with openbao
864
865This document covers setting up spindle to use OpenBao for secrets
866management via OpenBao Proxy instead of the default SQLite backend.
867
868### Overview
869
870Spindle now uses OpenBao Proxy for secrets management. The proxy handles
871authentication automatically using AppRole credentials, while spindle
872connects to the local proxy instead of directly to the OpenBao server.
873
874This approach provides better security, automatic token renewal, and
875simplified application code.
876
877### Installation
878
879Install OpenBao from Nixpkgs:
880
881```bash
882nix shell nixpkgs#openbao # for a local server
883```
884
885### Setup
886
887The setup process can is documented for both local development and production.
888
889#### Local development
890
891Start OpenBao in dev mode:
892
893```bash
894bao server -dev -dev-root-token-id="root" -dev-listen-address=127.0.0.1:8201
895```
896
897This starts OpenBao on `http://localhost:8201` with a root token.
898
899Set up environment for bao CLI:
900
901```bash
902export BAO_ADDR=http://localhost:8200
903export BAO_TOKEN=root
904```
905
906#### Production
907
908You would typically use a systemd service with a
909configuration file. Refer to
910[@tangled.org/infra](https://tangled.org/@tangled.org/infra)
911for how this can be achieved using Nix.
912
913Then, initialize the bao server:
914
915```bash
916bao operator init -key-shares=1 -key-threshold=1
917```
918
919This will print out an unseal key and a root key. Save them
920somewhere (like a password manager). Then unseal the vault
921to begin setting it up:
922
923```bash
924bao operator unseal <unseal_key>
925```
926
927All steps below remain the same across both dev and
928production setups.
929
930#### Configure openbao server
931
932Create the spindle KV mount:
933
934```bash
935bao secrets enable -path=spindle -version=2 kv
936```
937
938Set up AppRole authentication and policy:
939
940Create a policy file `spindle-policy.hcl`:
941
942```hcl
943# Full access to spindle KV v2 data
944path "spindle/data/*" {
945 capabilities = ["create", "read", "update", "delete"]
946}
947
948# Access to metadata for listing and management
949path "spindle/metadata/*" {
950 capabilities = ["list", "read", "delete", "update"]
951}
952
953# Allow listing at root level
954path "spindle/" {
955 capabilities = ["list"]
956}
957
958# Required for connection testing and health checks
959path "auth/token/lookup-self" {
960 capabilities = ["read"]
961}
962```
963
964Apply the policy and create an AppRole:
965
966```bash
967bao policy write spindle-policy spindle-policy.hcl
968bao auth enable approle
969bao write auth/approle/role/spindle \
970 token_policies="spindle-policy" \
971 token_ttl=1h \
972 token_max_ttl=4h \
973 bind_secret_id=true \
974 secret_id_ttl=0 \
975 secret_id_num_uses=0
976```
977
978Get the credentials:
979
980```bash
981# Get role ID (static)
982ROLE_ID=$(bao read -field=role_id auth/approle/role/spindle/role-id)
983
984# Generate secret ID
985SECRET_ID=$(bao write -f -field=secret_id auth/approle/role/spindle/secret-id)
986
987echo "Role ID: $ROLE_ID"
988echo "Secret ID: $SECRET_ID"
989```
990
991#### Create proxy configuration
992
993Create the credential files:
994
995```bash
996# Create directory for OpenBao files
997mkdir -p /tmp/openbao
998
999# Save credentials
1000echo "$ROLE_ID" > /tmp/openbao/role-id
1001echo "$SECRET_ID" > /tmp/openbao/secret-id
1002chmod 600 /tmp/openbao/role-id /tmp/openbao/secret-id
1003```
1004
1005Create a proxy configuration file `/tmp/openbao/proxy.hcl`:
1006
1007```hcl
1008# OpenBao server connection
1009vault {
1010 address = "http://localhost:8200"
1011}
1012
1013# Auto-Auth using AppRole
1014auto_auth {
1015 method "approle" {
1016 mount_path = "auth/approle"
1017 config = {
1018 role_id_file_path = "/tmp/openbao/role-id"
1019 secret_id_file_path = "/tmp/openbao/secret-id"
1020 }
1021 }
1022
1023 # Optional: write token to file for debugging
1024 sink "file" {
1025 config = {
1026 path = "/tmp/openbao/token"
1027 mode = 0640
1028 }
1029 }
1030}
1031
1032# Proxy listener for spindle
1033listener "tcp" {
1034 address = "127.0.0.1:8201"
1035 tls_disable = true
1036}
1037
1038# Enable API proxy with auto-auth token
1039api_proxy {
1040 use_auto_auth_token = true
1041}
1042
1043# Enable response caching
1044cache {
1045 use_auto_auth_token = true
1046}
1047
1048# Logging
1049log_level = "info"
1050```
1051
1052#### Start the proxy
1053
1054Start OpenBao Proxy:
1055
1056```bash
1057bao proxy -config=/tmp/openbao/proxy.hcl
1058```
1059
1060The proxy will authenticate with OpenBao and start listening on
1061`127.0.0.1:8201`.
1062
1063#### Configure spindle
1064
1065Set these environment variables for spindle:
1066
1067```bash
1068export SPINDLE_SERVER_SECRETS_PROVIDER=openbao
1069export SPINDLE_SERVER_SECRETS_OPENBAO_PROXY_ADDR=http://127.0.0.1:8201
1070export SPINDLE_SERVER_SECRETS_OPENBAO_MOUNT=spindle
1071```
1072
1073On startup, spindle will now connect to the local proxy,
1074which handles all authentication automatically.
1075
1076### Production setup for proxy
1077
1078For production, you'll want to run the proxy as a service:
1079
1080Place your production configuration in
1081`/etc/openbao/proxy.hcl` with proper TLS settings for the
1082vault connection.
1083
1084### Verifying setup
1085
1086Test the proxy directly:
1087
1088```bash
1089# Check proxy health
1090curl -H "X-Vault-Request: true" http://127.0.0.1:8201/v1/sys/health
1091
1092# Test token lookup through proxy
1093curl -H "X-Vault-Request: true" http://127.0.0.1:8201/v1/auth/token/lookup-self
1094```
1095
1096Test OpenBao operations through the server:
1097
1098```bash
1099# List all secrets
1100bao kv list spindle/
1101
1102# Add a test secret via the spindle API, then check it exists
1103bao kv list spindle/repos/
1104
1105# Get a specific secret
1106bao kv get spindle/repos/your_repo_path/SECRET_NAME
1107```
1108
1109### How it works
1110
1111- Spindle connects to OpenBao Proxy on localhost (typically
1112 port 8200 or 8201)
1113- The proxy authenticates with OpenBao using AppRole
1114 credentials
1115- All spindle requests go through the proxy, which injects
1116 authentication tokens
1117- Secrets are stored at
1118 `spindle/repos/{sanitized_repo_path}/{secret_key}`
1119- Repository paths like `did:plc:alice/myrepo` become
1120 `did_plc_alice_myrepo`
1121- The proxy handles all token renewal automatically
1122- Spindle no longer manages tokens or authentication
1123 directly
1124
1125### Troubleshooting
1126
1127**Connection refused**: Check that the OpenBao Proxy is
1128running and listening on the configured address.
1129
1130**403 errors**: Verify the AppRole credentials are correct
1131and the policy has the necessary permissions.
1132
1133**404 route errors**: The spindle KV mount probably doesn't
1134exist—run the mount creation step again.
1135
1136**Proxy authentication failures**: Check the proxy logs and
1137verify the role-id and secret-id files are readable and
1138contain valid credentials.
1139
1140**Secret not found after writing**: This can indicate policy
1141permission issues. Verify the policy includes both
1142`spindle/data/*` and `spindle/metadata/*` paths with
1143appropriate capabilities.
1144
1145Check proxy logs:
1146
1147```bash
1148# If running as systemd service
1149journalctl -u openbao-proxy -f
1150
1151# If running directly, check the console output
1152```
1153
1154Test AppRole authentication manually:
1155
1156```bash
1157bao write auth/approle/login \
1158 role_id="$(cat /tmp/openbao/role-id)" \
1159 secret_id="$(cat /tmp/openbao/secret-id)"
1160```
1161
1162# Migrating knots and spindles
1163
1164Sometimes, non-backwards compatible changes are made to the
1165knot/spindle XRPC APIs. If you host a knot or a spindle, you
1166will need to follow this guide to upgrade. Typically, this
1167only requires you to deploy the newest version.
1168
1169This document is laid out in reverse-chronological order.
1170Newer migration guides are listed first, and older guides
1171are further down the page.
1172
1173## Upgrading from v1.8.x
1174
1175After v1.8.2, the HTTP API for knots and spindles has been
1176deprecated and replaced with XRPC. Repositories on outdated
1177knots will not be viewable from the appview. Upgrading is
1178straightforward however.
1179
1180For knots:
1181
1182- Upgrade to the latest tag (v1.9.0 or above)
1183- Head to the [knot dashboard](https://tangled.org/settings/knots) and
1184 hit the "retry" button to verify your knot
1185
1186For spindles:
1187
1188- Upgrade to the latest tag (v1.9.0 or above)
1189- Head to the [spindle
1190 dashboard](https://tangled.org/settings/spindles) and hit the
1191 "retry" button to verify your spindle
1192
1193## Upgrading from v1.7.x
1194
1195After v1.7.0, knot secrets have been deprecated. You no
1196longer need a secret from the appview to run a knot. All
1197authorized commands to knots are managed via [Inter-Service
1198Authentication](https://atproto.com/specs/xrpc#inter-service-authentication-jwt).
1199Knots will be read-only until upgraded.
1200
1201Upgrading is quite easy, in essence:
1202
1203- `KNOT_SERVER_SECRET` is no more, you can remove this
1204 environment variable entirely
1205- `KNOT_SERVER_OWNER` is now required on boot, set this to
1206 your DID. You can find your DID in the
1207 [settings](https://tangled.org/settings) page.
1208- Restart your knot once you have replaced the environment
1209 variable
1210- Head to the [knot dashboard](https://tangled.org/settings/knots) and
1211 hit the "retry" button to verify your knot. This simply
1212 writes a `sh.tangled.knot` record to your PDS.
1213
1214If you use the nix module, simply bump the flake to the
1215latest revision, and change your config block like so:
1216
1217```diff
1218 services.tangled.knot = {
1219 enable = true;
1220 server = {
1221- secretFile = /path/to/secret;
1222+ owner = "did:plc:foo";
1223 };
1224 };
1225```
1226
1227# Hacking on Tangled
1228
1229We highly recommend [installing
1230Nix](https://nixos.org/download/) (the package manager)
1231before working on the codebase. The Nix flake provides a lot
1232of helpers to get started and most importantly, builds and
1233dev shells are entirely deterministic.
1234
1235To set up your dev environment:
1236
1237```bash
1238nix develop
1239```
1240
1241Non-Nix users can look at the `devShell` attribute in the
1242`flake.nix` file to determine necessary dependencies.
1243
1244## Running the appview
1245
1246The Nix flake also exposes a few `app` attributes (run `nix
1247flake show` to see a full list of what the flake provides),
1248one of the apps runs the appview with the `air`
1249live-reloader:
1250
1251```bash
1252TANGLED_DEV=true nix run .#watch-appview
1253
1254# TANGLED_DB_PATH might be of interest to point to
1255# different sqlite DBs
1256
1257# in a separate shell, you can live-reload tailwind
1258nix run .#watch-tailwind
1259```
1260
1261To authenticate with the appview, you will need Redis and
1262OAuth JWKs to be set up:
1263
1264```
1265# OAuth JWKs should already be set up by the Nix devshell:
1266echo $TANGLED_OAUTH_CLIENT_SECRET
1267z42ty4RT1ovnTopY8B8ekz9NuziF2CuMkZ7rbRFpAR9jBqMc
1268
1269echo $TANGLED_OAUTH_CLIENT_KID
12701761667908
1271
1272# if not, you can set it up yourself:
1273goat key generate -t P-256
1274Key Type: P-256 / secp256r1 / ES256 private key
1275Secret Key (Multibase Syntax): save this securely (eg, add to password manager)
1276 z42tuPDKRfM2mz2Kv953ARen2jmrPA8S9LX9tRq4RVcUMwwL
1277Public Key (DID Key Syntax): share or publish this (eg, in DID document)
1278 did:key:zDnaeUBxtG6Xuv3ATJE4GaWeyXM3jyamJsZw3bSPpxx4bNXDR
1279
1280# the secret key from above
1281export TANGLED_OAUTH_CLIENT_SECRET="z42tuP..."
1282
1283# Run Redis in a new shell to store OAuth sessions
1284redis-server
1285```
1286
1287## Running knots and spindles
1288
1289An end-to-end knot setup requires setting up a machine with
1290`sshd`, `AuthorizedKeysCommand`, and a Git user, which is
1291quite cumbersome. So the Nix flake provides a
1292`nixosConfiguration` to do so.
1293
1294<details>
1295 <summary><strong>macOS users will have to set up a Nix Builder first</strong></summary>
1296
1297 In order to build Tangled's dev VM on macOS, you will
1298 first need to set up a Linux Nix builder. The recommended
1299 way to do so is to run a [`darwin.linux-builder`
1300 VM](https://nixos.org/manual/nixpkgs/unstable/#sec-darwin-builder)
1301 and to register it in `nix.conf` as a builder for Linux
1302 with the same architecture as your Mac (`linux-aarch64` if
1303 you are using Apple Silicon).
1304
1305 > IMPORTANT: You must build `darwin.linux-builder` somewhere other than inside
1306 > the Tangled repo so that it doesn't conflict with the other VM. For example,
1307 > you can do
1308 >
1309 > ```shell
1310 > cd $(mktemp -d buildervm.XXXXX) && nix run nixpkgs#darwin.linux-builder
1311 > ```
1312 >
1313 > to store the builder VM in a temporary dir.
1314 >
1315 > You should read and follow [all the other intructions][darwin builder vm] to
1316 > avoid subtle problems.
1317
1318 Alternatively, you can use any other method to set up a
1319 Linux machine with Nix installed that you can `sudo ssh`
1320 into (in other words, root user on your Mac has to be able
1321 to ssh into the Linux machine without entering a password)
1322 and that has the same architecture as your Mac. See
1323 [remote builder
1324 instructions](https://nix.dev/manual/nix/2.28/advanced-topics/distributed-builds.html#requirements)
1325 for how to register such a builder in `nix.conf`.
1326
1327 > WARNING: If you'd like to use
1328 > [`nixos-lima`](https://github.com/nixos-lima/nixos-lima) or
1329 > [Orbstack](https://orbstack.dev/), note that setting them up so that `sudo
1330 > ssh` works can be tricky. It seems to be [possible with
1331 > Orbstack](https://github.com/orgs/orbstack/discussions/1669).
1332
1333</details>
1334
1335To begin, grab your DID from http://localhost:3000/settings.
1336Then, set `TANGLED_VM_KNOT_OWNER` and
1337`TANGLED_VM_SPINDLE_OWNER` to your DID. You can now start a
1338lightweight NixOS VM like so:
1339
1340```bash
1341nix run --impure .#vm
1342
1343# type `poweroff` at the shell to exit the VM
1344```
1345
1346This starts a knot on port 6444, a spindle on port 6555
1347with `ssh` exposed on port 2222.
1348
1349Once the services are running, head to
1350http://localhost:3000/settings/knots and hit "Verify". It should
1351verify the ownership of the services instantly if everything
1352went smoothly.
1353
1354You can push repositories to this VM with this ssh config
1355block on your main machine:
1356
1357```bash
1358Host nixos-shell
1359 Hostname localhost
1360 Port 2222
1361 User git
1362 IdentityFile ~/.ssh/my_tangled_key
1363```
1364
1365Set up a remote called `local-dev` on a git repo:
1366
1367```bash
1368git remote add local-dev git@nixos-shell:user/repo
1369git push local-dev main
1370```
1371
1372The above VM should already be running a spindle on
1373`localhost:6555`. Head to http://localhost:3000/settings/spindles and
1374hit "Verify". You can then configure each repository to use
1375this spindle and run CI jobs.
1376
1377Of interest when debugging spindles:
1378
1379```
1380# Service logs from journald:
1381journalctl -xeu spindle
1382
1383# CI job logs from disk:
1384ls /var/log/spindle
1385
1386# Debugging spindle database:
1387sqlite3 /var/lib/spindle/spindle.db
1388
1389# litecli has a nicer REPL interface:
1390litecli /var/lib/spindle/spindle.db
1391```
1392
1393If for any reason you wish to disable either one of the
1394services in the VM, modify [nix/vm.nix](/nix/vm.nix) and set
1395`services.tangled.spindle.enable` (or
1396`services.tangled.knot.enable`) to `false`.
1397
1398# Contribution guide
1399
1400## Commit guidelines
1401
1402We follow a commit style similar to the Go project. Please keep commits:
1403
1404* **atomic**: each commit should represent one logical change
1405* **descriptive**: the commit message should clearly describe what the
1406change does and why it's needed
1407
1408### Message format
1409
1410```
1411<service/top-level directory>/<affected package/directory>: <short summary of change>
1412
1413Optional longer description can go here, if necessary. Explain what the
1414change does and why, especially if not obvious. Reference relevant
1415issues or PRs when applicable. These can be links for now since we don't
1416auto-link issues/PRs yet.
1417```
1418
1419Here are some examples:
1420
1421```
1422appview/state: fix token expiry check in middleware
1423
1424The previous check did not account for clock drift, leading to premature
1425token invalidation.
1426```
1427
1428```
1429knotserver/git/service: improve error checking in upload-pack
1430```
1431
1432
1433### General notes
1434
1435- PRs get merged "as-is" (fast-forward)—like applying a patch-series
1436using `git am`. At present, there is no squashing—so please author
1437your commits as they would appear on `master`, following the above
1438guidelines.
1439- If there is a lot of nesting, for example "appview:
1440pages/templates/repo/fragments: ...", these can be truncated down to
1441just "appview: repo/fragments: ...". If the change affects a lot of
1442subdirectories, you may abbreviate to just the top-level names, e.g.
1443"appview: ..." or "knotserver: ...".
1444- Keep commits lowercased with no trailing period.
1445- Use the imperative mood in the summary line (e.g., "fix bug" not
1446"fixed bug" or "fixes bug").
1447- Try to keep the summary line under 72 characters, but we aren't too
1448fussed about this.
1449- Follow the same formatting for PR titles if filled manually.
1450- Don't include unrelated changes in the same commit.
1451- Avoid noisy commit messages like "wip" or "final fix"—rewrite history
1452before submitting if necessary.
1453
1454## Code formatting
1455
1456We use a variety of tools to format our code, and multiplex them with
1457[`treefmt`](https://treefmt.com). All you need to do to format your changes
1458is run `nix run .#fmt` (or just `treefmt` if you're in the devshell).
1459
1460## Proposals for bigger changes
1461
1462Small fixes like typos, minor bugs, or trivial refactors can be
1463submitted directly as PRs.
1464
1465For larger changes—especially those introducing new features, significant
1466refactoring, or altering system behavior—please open a proposal first. This
1467helps us evaluate the scope, design, and potential impact before implementation.
1468
1469Create a new issue titled:
1470
1471```
1472proposal: <affected scope>: <summary of change>
1473```
1474
1475In the description, explain:
1476
1477- What the change is
1478- Why it's needed
1479- How you plan to implement it (roughly)
1480- Any open questions or tradeoffs
1481
1482We'll use the issue thread to discuss and refine the idea before moving
1483forward.
1484
1485## Developer Certificate of Origin (DCO)
1486
1487We require all contributors to certify that they have the right to
1488submit the code they're contributing. To do this, we follow the
1489[Developer Certificate of Origin
1490(DCO)](https://developercertificate.org/).
1491
1492By signing your commits, you're stating that the contribution is your
1493own work, or that you have the right to submit it under the project's
1494license. This helps us keep things clean and legally sound.
1495
1496To sign your commit, just add the `-s` flag when committing:
1497
1498```sh
1499git commit -s -m "your commit message"
1500```
1501
1502This appends a line like:
1503
1504```
1505Signed-off-by: Your Name <your.email@example.com>
1506```
1507
1508We won't merge commits if they aren't signed off. If you forget, you can
1509amend the last commit like this:
1510
1511```sh
1512git commit --amend -s
1513```
1514
1515If you're submitting a PR with multiple commits, make sure each one is
1516signed.
1517
1518For [jj](https://jj-vcs.github.io/jj/latest/) users, you can run the following command
1519to make it sign off commits in the tangled repo:
1520
1521```shell
1522# Safety check, should say "No matching config key..."
1523jj config list templates.commit_trailers
1524# The command below may need to be adjusted if the command above returned something.
1525jj config set --repo templates.commit_trailers "format_signed_off_by_trailer(self)"
1526```
1527
1528Refer to the [jujutsu
1529documentation](https://jj-vcs.github.io/jj/latest/config/#commit-trailers)
1530for more information.