this repo has no description
tangled.org
1---
2title: Tangled docs
3author: The Tangled Contributors
4date: 21 Sun, Dec 2025
5---
6
7# Introduction
8
9Tangled is a decentralized code hosting and collaboration
10platform. Every component of Tangled is open-source and
11self-hostable. [tangled.org](https://tangled.org) also
12provides hosting and CI services that are free to use.
13
14There are several models for decentralized code
15collaboration platforms, ranging from ActivityPub’s
16(Forgejo) federated model, to Radicle’s entirely P2P model.
17Our approach attempts to be the best of both worlds by
18adopting the AT Protocol—a protocol for building decentralized
19social applications with a central identity
20
21Our approach to this is the idea of “knots”. Knots are
22lightweight, headless servers that enable users to host Git
23repositories with ease. Knots are designed for either single
24or multi-tenant use which is perfect for self-hosting on a
25Raspberry Pi at home, or larger “community” servers. By
26default, Tangled provides managed knots where you can host
27your repositories for free.
28
29The appview at tangled.org acts as a consolidated "view"
30into the whole network, allowing users to access, clone and
31contribute to repositories hosted across different knots
32seamlessly.
33
34# Quick start guide
35
36## Login or sign up
37
38You can [login](https://tangled.org) by using your AT Protocol
39account. If you are unclear on what that means, simply head
40to the [signup](https://tangled.org/signup) page and create
41an account. By doing so, you will be choosing Tangled as
42your account provider (you will be granted a handle of the
43form `user.tngl.sh`).
44
45In the AT Protocol network, users are free to choose their account
46provider (known as a "Personal Data Service", or PDS), and
47login to applications that support AT accounts.
48
49You can think of it as "one account for all of the atmosphere"!
50
51If you already have an AT account (you may have one if you
52signed up to Bluesky, for example), you can login with the
53same handle on Tangled (so just use `user.bsky.social` on
54the login page).
55
56## Add an SSH key
57
58Once you are logged in, you can start creating repositories
59and pushing code. Tangled supports pushing git repositories
60over SSH.
61
62First, you'll need to generate an SSH key if you don't
63already have one:
64
65```bash
66ssh-keygen -t ed25519 -C "foo@bar.com"
67```
68
69When prompted, save the key to the default location
70(`~/.ssh/id_ed25519`) and optionally set a passphrase.
71
72Copy your public key to your clipboard:
73
74```bash
75# on X11
76cat ~/.ssh/id_ed25519.pub | xclip -sel c
77
78# on wayland
79cat ~/.ssh/id_ed25519.pub | wl-copy
80
81# on macos
82cat ~/.ssh/id_ed25519.pub | pbcopy
83```
84
85Now, navigate to 'Settings' -> 'Keys' and hit 'Add Key',
86paste your public key, give it a descriptive name, and hit
87save.
88
89## Create a repository
90
91Once your SSH key is added, create your first repository:
92
931. Hit the green `+` icon on the topbar, and select
94 repository
952. Enter a repository name
963. Add a description
974. Choose a knotserver to host this repository on
985. Hit create
99
100Knots are self-hostable, lightweight Git servers that can
101host your repository. Unlike traditional code forges, your
102code can live on any server. Read the [Knots](TODO) section
103for more.
104
105## Configure SSH
106
107To ensure Git uses the correct SSH key and connects smoothly
108to Tangled, add this configuration to your `~/.ssh/config`
109file:
110
111```
112Host tangled.org
113 Hostname tangled.org
114 User git
115 IdentityFile ~/.ssh/id_ed25519
116 AddressFamily inet
117```
118
119This tells SSH to use your specific key when connecting to
120Tangled and prevents authentication issues if you have
121multiple SSH keys.
122
123Note that this configuration only works for knotservers that
124are hosted by tangled.org. If you use a custom knot, refer
125to the [Knots](TODO) section.
126
127## Push your first repository
128
129Initialize a new Git repository:
130
131```bash
132mkdir my-project
133cd my-project
134
135git init
136echo "# My Project" > README.md
137```
138
139Add some content and push!
140
141```bash
142git add README.md
143git commit -m "Initial commit"
144git remote add origin git@tangled.org:user.tngl.sh/my-project
145git push -u origin main
146```
147
148That's it! Your code is now hosted on Tangled.
149
150## Migrating an existing repository
151
152Moving your repositories from GitHub, GitLab, Bitbucket, or
153any other Git forge to Tangled is straightforward. You'll
154simply change your repository's remote URL. At the moment,
155Tangled does not have any tooling to migrate data such as
156GitHub issues or pull requests.
157
158First, create a new repository on tangled.org as described
159in the [Quick Start Guide](#create-a-repository).
160
161Navigate to your existing local repository:
162
163```bash
164cd /path/to/your/existing/repo
165```
166
167You can inspect your existing Git remote like so:
168
169```bash
170git remote -v
171```
172
173You'll see something like:
174
175```
176origin git@github.com:username/my-project (fetch)
177origin git@github.com:username/my-project (push)
178```
179
180Update the remote URL to point to tangled:
181
182```bash
183git remote set-url origin git@tangled.org:user.tngl.sh/my-project
184```
185
186Verify the change:
187
188```bash
189git remote -v
190```
191
192You should now see:
193
194```
195origin git@tangled.org:user.tngl.sh/my-project (fetch)
196origin git@tangled.org:user.tngl.sh/my-project (push)
197```
198
199Push all your branches and tags to Tangled:
200
201```bash
202git push -u origin --all
203git push -u origin --tags
204```
205
206Your repository is now migrated to Tangled! All commit
207history, branches, and tags have been preserved.
208
209## Mirroring a repository to Tangled
210
211If you want to maintain your repository on multiple forges
212simultaneously, for example, keeping your primary repository
213on GitHub while mirroring to Tangled for backup or
214redundancy, you can do so by adding multiple remotes.
215
216You can configure your local repository to push to both
217Tangled and, say, GitHub. You may already have the following
218setup:
219
220```
221$ git remote -v
222origin git@github.com:username/my-project (fetch)
223origin git@github.com:username/my-project (push)
224```
225
226Now add Tangled as an additional push URL to the same
227remote:
228
229```bash
230git remote set-url --add --push origin git@tangled.org:user.tngl.sh/my-project
231```
232
233You also need to re-add the original URL as a push
234destination (Git replaces the push URL when you use `--add`
235the first time):
236
237```bash
238git remote set-url --add --push origin git@github.com:username/my-project
239```
240
241Verify your configuration:
242
243```
244$ git remote -v
245origin git@github.com:username/repo (fetch)
246origin git@tangled.org:username/my-project (push)
247origin git@github.com:username/repo (push)
248```
249
250Notice that there's one fetch URL (the primary remote) and
251two push URLs. Now, whenever you push, Git will
252automatically push to both remotes:
253
254```bash
255git push origin main
256```
257
258This single command pushes your `main` branch to both GitHub
259and Tangled simultaneously.
260
261To push all branches and tags:
262
263```bash
264git push origin --all
265git push origin --tags
266```
267
268If you prefer more control over which remote you push to,
269you can maintain separate remotes:
270
271```bash
272git remote add github git@github.com:username/my-project
273git remote add tangled git@tangled.org:username/my-project
274```
275
276Then push to each explicitly:
277
278```bash
279git push github main
280git push tangled main
281```
282
283# Knot self-hosting guide
284
285So you want to run your own knot server? Great! Here are a few prerequisites:
286
2871. A server of some kind (a VPS, a Raspberry Pi, etc.). Preferably running a Linux distribution of some kind.
2882. A (sub)domain name. People generally use `knot.example.com`.
2893. A valid SSL certificate for your domain.
290
291## NixOS
292
293Refer to the [knot
294module](https://tangled.org/tangled.org/core/blob/master/nix/modules/knot.nix)
295for a full list of options. Sample configurations:
296
297- [The test VM](https://tangled.org/tangled.org/core/blob/master/nix/vm.nix#L85)
298- [@pyrox.dev/nix](https://tangled.org/pyrox.dev/nix/blob/d19571cc1b5fe01035e1e6951ec8cf8a476b4dee/hosts/marvin/services/tangled.nix#L15-25)
299
300## Docker
301
302Refer to
303[@tangled.org/knot-docker](https://tangled.org/@tangled.org/knot-docker).
304Note that this is community maintained.
305
306## Manual setup
307
308First, clone this repository:
309
310```
311git clone https://tangled.org/@tangled.org/core
312```
313
314Then, build the `knot` CLI. This is the knot administration
315and operation tool. For the purpose of this guide, we're
316only concerned with these subcommands:
317
318 * `knot server`: the main knot server process, typically
319 run as a supervised service
320 * `knot guard`: handles role-based access control for git
321 over SSH (you'll never have to run this yourself)
322 * `knot keys`: fetches SSH keys associated with your knot;
323 we'll use this to generate the SSH
324 `AuthorizedKeysCommand`
325
326```
327cd core
328export CGO_ENABLED=1
329go build -o knot ./cmd/knot
330```
331
332Next, move the `knot` binary to a location owned by `root` --
333`/usr/local/bin/` is a good choice. Make sure the binary itself is also owned by `root`:
334
335```
336sudo mv knot /usr/local/bin/knot
337sudo chown root:root /usr/local/bin/knot
338```
339
340This is necessary because SSH `AuthorizedKeysCommand` requires [really
341specific permissions](https://stackoverflow.com/a/27638306). The
342`AuthorizedKeysCommand` specifies a command that is run by `sshd` to
343retrieve a user's public SSH keys dynamically for authentication. Let's
344set that up.
345
346```
347sudo tee /etc/ssh/sshd_config.d/authorized_keys_command.conf <<EOF
348Match User git
349 AuthorizedKeysCommand /usr/local/bin/knot keys -o authorized-keys
350 AuthorizedKeysCommandUser nobody
351EOF
352```
353
354Then, reload `sshd`:
355
356```
357sudo systemctl reload ssh
358```
359
360Next, create the `git` user. We'll use the `git` user's home directory
361to store repositories:
362
363```
364sudo adduser git
365```
366
367Create `/home/git/.knot.env` with the following, updating the values as
368necessary. The `KNOT_SERVER_OWNER` should be set to your
369DID, you can find your DID in the [Settings](https://tangled.sh/settings) page.
370
371```
372KNOT_REPO_SCAN_PATH=/home/git
373KNOT_SERVER_HOSTNAME=knot.example.com
374APPVIEW_ENDPOINT=https://tangled.org
375KNOT_SERVER_OWNER=did:plc:foobar
376KNOT_SERVER_INTERNAL_LISTEN_ADDR=127.0.0.1:5444
377KNOT_SERVER_LISTEN_ADDR=127.0.0.1:5555
378```
379
380If you run a Linux distribution that uses systemd, you can use the provided
381service file to run the server. Copy
382[`knotserver.service`](/systemd/knotserver.service)
383to `/etc/systemd/system/`. Then, run:
384
385```
386systemctl enable knotserver
387systemctl start knotserver
388```
389
390The last step is to configure a reverse proxy like Nginx or Caddy to front your
391knot. Here's an example configuration for Nginx:
392
393```
394server {
395 listen 80;
396 listen [::]:80;
397 server_name knot.example.com;
398
399 location / {
400 proxy_pass http://localhost:5555;
401 proxy_set_header Host $host;
402 proxy_set_header X-Real-IP $remote_addr;
403 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
404 proxy_set_header X-Forwarded-Proto $scheme;
405 }
406
407 # wss endpoint for git events
408 location /events {
409 proxy_set_header X-Forwarded-For $remote_addr;
410 proxy_set_header Host $http_host;
411 proxy_set_header Upgrade websocket;
412 proxy_set_header Connection Upgrade;
413 proxy_pass http://localhost:5555;
414 }
415 # additional config for SSL/TLS go here.
416}
417
418```
419
420Remember to use Let's Encrypt or similar to procure a certificate for your
421knot domain.
422
423You should now have a running knot server! You can finalize
424your registration by hitting the `verify` button on the
425[/settings/knots](https://tangled.org/settings/knots) page. This simply creates
426a record on your PDS to announce the existence of the knot.
427
428### Custom paths
429
430(This section applies to manual setup only. Docker users should edit the mounts
431in `docker-compose.yml` instead.)
432
433Right now, the database and repositories of your knot lives in `/home/git`. You
434can move these paths if you'd like to store them in another folder. Be careful
435when adjusting these paths:
436
437* Stop your knot when moving data (e.g. `systemctl stop knotserver`) to prevent
438any possible side effects. Remember to restart it once you're done.
439* Make backups before moving in case something goes wrong.
440* Make sure the `git` user can read and write from the new paths.
441
442#### Database
443
444As an example, let's say the current database is at `/home/git/knotserver.db`,
445and we want to move it to `/home/git/database/knotserver.db`.
446
447Copy the current database to the new location. Make sure to copy the `.db-shm`
448and `.db-wal` files if they exist.
449
450```
451mkdir /home/git/database
452cp /home/git/knotserver.db* /home/git/database
453```
454
455In the environment (e.g. `/home/git/.knot.env`), set `KNOT_SERVER_DB_PATH` to
456the new file path (_not_ the directory):
457
458```
459KNOT_SERVER_DB_PATH=/home/git/database/knotserver.db
460```
461
462#### Repositories
463
464As an example, let's say the repositories are currently in `/home/git`, and we
465want to move them into `/home/git/repositories`.
466
467Create the new folder, then move the existing repositories (if there are any):
468
469```
470mkdir /home/git/repositories
471# move all DIDs into the new folder; these will vary for you!
472mv /home/git/did:plc:wshs7t2adsemcrrd4snkeqli /home/git/repositories
473```
474
475In the environment (e.g. `/home/git/.knot.env`), update `KNOT_REPO_SCAN_PATH`
476to the new directory:
477
478```
479KNOT_REPO_SCAN_PATH=/home/git/repositories
480```
481
482Similarly, update your `sshd` `AuthorizedKeysCommand` to use the updated
483repository path:
484
485```
486sudo tee /etc/ssh/sshd_config.d/authorized_keys_command.conf <<EOF
487Match User git
488 AuthorizedKeysCommand /usr/local/bin/knot keys -o authorized-keys -git-dir /home/git/repositories
489 AuthorizedKeysCommandUser nobody
490EOF
491```
492
493Make sure to restart your SSH server!
494
495#### MOTD (message of the day)
496
497To configure the MOTD used ("Welcome to this knot!" by default), edit the
498`/home/git/motd` file:
499
500```
501printf "Hi from this knot!\n" > /home/git/motd
502```
503
504Note that you should add a newline at the end if setting a non-empty message
505since the knot won't do this for you.
506
507# Spindles
508
509## Pipelines
510
511Spindle workflows allow you to write CI/CD pipelines in a
512simple format. They're located in the `.tangled/workflows`
513directory at the root of your repository, and are defined
514using YAML.
515
516The fields are:
517
518- [Trigger](#trigger): A **required** field that defines
519 when a workflow should be triggered.
520- [Engine](#engine): A **required** field that defines which
521 engine a workflow should run on.
522- [Clone options](#clone-options): An **optional** field
523 that defines how the repository should be cloned.
524- [Dependencies](#dependencies): An **optional** field that
525 allows you to list dependencies you may need.
526- [Environment](#environment): An **optional** field that
527 allows you to define environment variables.
528- [Steps](#steps): An **optional** field that allows you to
529 define what steps should run in the workflow.
530
531### Trigger
532
533The first thing to add to a workflow is the trigger, which
534defines when a workflow runs. This is defined using a `when`
535field, which takes in a list of conditions. Each condition
536has the following fields:
537
538- `event`: This is a **required** field that defines when
539 your workflow should run. It's a list that can take one or
540 more of the following values:
541 - `push`: The workflow should run every time a commit is
542 pushed to the repository.
543 - `pull_request`: The workflow should run every time a
544 pull request is made or updated.
545 - `manual`: The workflow can be triggered manually.
546- `branch`: Defines which branches the workflow should run
547 for. If used with the `push` event, commits to the
548 branch(es) listed here will trigger the workflow. If used
549 with the `pull_request` event, updates to pull requests
550 targeting the branch(es) listed here will trigger the
551 workflow. This field has no effect with the `manual`
552 event. Supports glob patterns using `*` and `**` (e.g.,
553 `main`, `develop`, `release-*`). Either `branch` or `tag`
554 (or both) must be specified for `push` events.
555- `tag`: Defines which tags the workflow should run for.
556 Only used with the `push` event - when tags matching the
557 pattern(s) listed here are pushed, the workflow will
558 trigger. This field has no effect with `pull_request` or
559 `manual` events. Supports glob patterns using `*` and `**`
560 (e.g., `v*`, `v1.*`, `release-**`). Either `branch` or
561 `tag` (or both) must be specified for `push` events.
562
563For example, if you'd like to define a workflow that runs
564when commits are pushed to the `main` and `develop`
565branches, or when pull requests that target the `main`
566branch are updated, or manually, you can do so with:
567
568```yaml
569when:
570 - event: ["push", "manual"]
571 branch: ["main", "develop"]
572 - event: ["pull_request"]
573 branch: ["main"]
574```
575
576You can also trigger workflows on tag pushes. For instance,
577to run a deployment workflow when tags matching `v*` are
578pushed:
579
580```yaml
581when:
582 - event: ["push"]
583 tag: ["v*"]
584```
585
586You can even combine branch and tag patterns in a single
587constraint (the workflow triggers if either matches):
588
589```yaml
590when:
591 - event: ["push"]
592 branch: ["main", "release-*"]
593 tag: ["v*", "stable"]
594```
595
596### Engine
597
598Next is the engine on which the workflow should run, defined
599using the **required** `engine` field. The currently
600supported engines are:
601
602- `nixery`: This uses an instance of
603 [Nixery](https://nixery.dev) to run steps, which allows
604 you to add [dependencies](#dependencies) from
605 Nixpkgs (https://github.com/NixOS/nixpkgs). You can
606 search for packages on https://search.nixos.org, and
607 there's a pretty good chance the package(s) you're looking
608 for will be there.
609
610Example:
611
612```yaml
613engine: "nixery"
614```
615
616### Clone options
617
618When a workflow starts, the first step is to clone the
619repository. You can customize this behavior using the
620**optional** `clone` field. It has the following fields:
621
622- `skip`: Setting this to `true` will skip cloning the
623 repository. This can be useful if your workflow is doing
624 something that doesn't require anything from the
625 repository itself. This is `false` by default.
626- `depth`: This sets the number of commits, or the "clone
627 depth", to fetch from the repository. For example, if you
628 set this to 2, the last 2 commits will be fetched. By
629 default, the depth is set to 1, meaning only the most
630 recent commit will be fetched, which is the commit that
631 triggered the workflow.
632- `submodules`: If you use Git submodules
633 (https://git-scm.com/book/en/v2/Git-Tools-Submodules)
634 in your repository, setting this field to `true` will
635 recursively fetch all submodules. This is `false` by
636 default.
637
638The default settings are:
639
640```yaml
641clone:
642 skip: false
643 depth: 1
644 submodules: false
645```
646
647### Dependencies
648
649Usually when you're running a workflow, you'll need
650additional dependencies. The `dependencies` field lets you
651define which dependencies to get, and from where. It's a
652key-value map, with the key being the registry to fetch
653dependencies from, and the value being the list of
654dependencies to fetch.
655
656Say you want to fetch Node.js and Go from `nixpkgs`, and a
657package called `my_pkg` you've made from your own registry
658at your repository at
659`https://tangled.org/@example.com/my_pkg`. You can define
660those dependencies like so:
661
662```yaml
663dependencies:
664 # nixpkgs
665 nixpkgs:
666 - nodejs
667 - go
668 # custom registry
669 git+https://tangled.org/@example.com/my_pkg:
670 - my_pkg
671```
672
673Now these dependencies are available to use in your
674workflow!
675
676### Environment
677
678The `environment` field allows you define environment
679variables that will be available throughout the entire
680workflow. **Do not put secrets here, these environment
681variables are visible to anyone viewing the repository. You
682can add secrets for pipelines in your repository's
683settings.**
684
685Example:
686
687```yaml
688environment:
689 GOOS: "linux"
690 GOARCH: "arm64"
691 NODE_ENV: "production"
692 MY_ENV_VAR: "MY_ENV_VALUE"
693```
694
695### Steps
696
697The `steps` field allows you to define what steps should run
698in the workflow. It's a list of step objects, each with the
699following fields:
700
701- `name`: This field allows you to give your step a name.
702 This name is visible in your workflow runs, and is used to
703 describe what the step is doing.
704- `command`: This field allows you to define a command to
705 run in that step. The step is run in a Bash shell, and the
706 logs from the command will be visible in the pipelines
707 page on the Tangled website. The
708 [dependencies](#dependencies) you added will be available
709 to use here.
710- `environment`: Similar to the global
711 [environment](#environment) config, this **optional**
712 field is a key-value map that allows you to set
713 environment variables for the step. **Do not put secrets
714 here, these environment variables are visible to anyone
715 viewing the repository. You can add secrets for pipelines
716 in your repository's settings.**
717
718Example:
719
720```yaml
721steps:
722 - name: "Build backend"
723 command: "go build"
724 environment:
725 GOOS: "darwin"
726 GOARCH: "arm64"
727 - name: "Build frontend"
728 command: "npm run build"
729 environment:
730 NODE_ENV: "production"
731```
732
733### Complete workflow
734
735```yaml
736# .tangled/workflows/build.yml
737
738when:
739 - event: ["push", "manual"]
740 branch: ["main", "develop"]
741 - event: ["pull_request"]
742 branch: ["main"]
743
744engine: "nixery"
745
746# using the default values
747clone:
748 skip: false
749 depth: 1
750 submodules: false
751
752dependencies:
753 # nixpkgs
754 nixpkgs:
755 - nodejs
756 - go
757 # custom registry
758 git+https://tangled.org/@example.com/my_pkg:
759 - my_pkg
760
761environment:
762 GOOS: "linux"
763 GOARCH: "arm64"
764 NODE_ENV: "production"
765 MY_ENV_VAR: "MY_ENV_VALUE"
766
767steps:
768 - name: "Build backend"
769 command: "go build"
770 environment:
771 GOOS: "darwin"
772 GOARCH: "arm64"
773 - name: "Build frontend"
774 command: "npm run build"
775 environment:
776 NODE_ENV: "production"
777```
778
779If you want another example of a workflow, you can look at
780the one [Tangled uses to build the
781project](https://tangled.org/@tangled.org/core/blob/master/.tangled/workflows/build.yml).
782
783## Self-hosting guide
784
785### Prerequisites
786
787* Go
788* Docker (the only supported backend currently)
789
790### Configuration
791
792Spindle is configured using environment variables. The following environment variables are available:
793
794* `SPINDLE_SERVER_LISTEN_ADDR`: The address the server listens on (default: `"0.0.0.0:6555"`).
795* `SPINDLE_SERVER_DB_PATH`: The path to the SQLite database file (default: `"spindle.db"`).
796* `SPINDLE_SERVER_HOSTNAME`: The hostname of the server (required).
797* `SPINDLE_SERVER_JETSTREAM_ENDPOINT`: The endpoint of the Jetstream server (default: `"wss://jetstream1.us-west.bsky.network/subscribe"`).
798* `SPINDLE_SERVER_DEV`: A boolean indicating whether the server is running in development mode (default: `false`).
799* `SPINDLE_SERVER_OWNER`: The DID of the owner (required).
800* `SPINDLE_PIPELINES_NIXERY`: The Nixery URL (default: `"nixery.tangled.sh"`).
801* `SPINDLE_PIPELINES_WORKFLOW_TIMEOUT`: The default workflow timeout (default: `"5m"`).
802* `SPINDLE_PIPELINES_LOG_DIR`: The directory to store workflow logs (default: `"/var/log/spindle"`).
803
804### Running spindle
805
8061. **Set the environment variables.** For example:
807
808 ```shell
809 export SPINDLE_SERVER_HOSTNAME="your-hostname"
810 export SPINDLE_SERVER_OWNER="your-did"
811 ```
812
8132. **Build the Spindle binary.**
814
815 ```shell
816 cd core
817 go mod download
818 go build -o cmd/spindle/spindle cmd/spindle/main.go
819 ```
820
8213. **Create the log directory.**
822
823 ```shell
824 sudo mkdir -p /var/log/spindle
825 sudo chown $USER:$USER -R /var/log/spindle
826 ```
827
8284. **Run the Spindle binary.**
829
830 ```shell
831 ./cmd/spindle/spindle
832 ```
833
834Spindle will now start, connect to the Jetstream server, and begin processing pipelines.
835
836## Architecture
837
838Spindle is a small CI runner service. Here's a high-level overview of how it operates:
839
840* Listens for [`sh.tangled.spindle.member`](/lexicons/spindle/member.json) and
841[`sh.tangled.repo`](/lexicons/repo.json) records on the Jetstream.
842* When a new repo record comes through (typically when you add a spindle to a
843repo from the settings), spindle then resolves the underlying knot and
844subscribes to repo events (see:
845[`sh.tangled.pipeline`](/lexicons/pipeline.json)).
846* The spindle engine then handles execution of the pipeline, with results and
847logs beamed on the spindle event stream over WebSocket
848
849### The engine
850
851At present, the only supported backend is Docker (and Podman, if Docker
852compatibility is enabled, so that `/run/docker.sock` is created). spindle
853executes each step in the pipeline in a fresh container, with state persisted
854across steps within the `/tangled/workspace` directory.
855
856The base image for the container is constructed on the fly using
857[Nixery](https://nixery.dev), which is handy for caching layers for frequently
858used packages.
859
860The pipeline manifest is [specified here](https://docs.tangled.org/spindles.html#pipelines).
861
862## Secrets with openbao
863
864This document covers setting up spindle to use OpenBao for secrets
865management via OpenBao Proxy instead of the default SQLite backend.
866
867### Overview
868
869Spindle now uses OpenBao Proxy for secrets management. The proxy handles
870authentication automatically using AppRole credentials, while spindle
871connects to the local proxy instead of directly to the OpenBao server.
872
873This approach provides better security, automatic token renewal, and
874simplified application code.
875
876### Installation
877
878Install OpenBao from Nixpkgs:
879
880```bash
881nix shell nixpkgs#openbao # for a local server
882```
883
884### Setup
885
886The setup process can is documented for both local development and production.
887
888#### Local development
889
890Start OpenBao in dev mode:
891
892```bash
893bao server -dev -dev-root-token-id="root" -dev-listen-address=127.0.0.1:8201
894```
895
896This starts OpenBao on `http://localhost:8201` with a root token.
897
898Set up environment for bao CLI:
899
900```bash
901export BAO_ADDR=http://localhost:8200
902export BAO_TOKEN=root
903```
904
905#### Production
906
907You would typically use a systemd service with a
908configuration file. Refer to
909[@tangled.org/infra](https://tangled.org/@tangled.org/infra)
910for how this can be achieved using Nix.
911
912Then, initialize the bao server:
913
914```bash
915bao operator init -key-shares=1 -key-threshold=1
916```
917
918This will print out an unseal key and a root key. Save them
919somewhere (like a password manager). Then unseal the vault
920to begin setting it up:
921
922```bash
923bao operator unseal <unseal_key>
924```
925
926All steps below remain the same across both dev and
927production setups.
928
929#### Configure openbao server
930
931Create the spindle KV mount:
932
933```bash
934bao secrets enable -path=spindle -version=2 kv
935```
936
937Set up AppRole authentication and policy:
938
939Create a policy file `spindle-policy.hcl`:
940
941```hcl
942# Full access to spindle KV v2 data
943path "spindle/data/*" {
944 capabilities = ["create", "read", "update", "delete"]
945}
946
947# Access to metadata for listing and management
948path "spindle/metadata/*" {
949 capabilities = ["list", "read", "delete", "update"]
950}
951
952# Allow listing at root level
953path "spindle/" {
954 capabilities = ["list"]
955}
956
957# Required for connection testing and health checks
958path "auth/token/lookup-self" {
959 capabilities = ["read"]
960}
961```
962
963Apply the policy and create an AppRole:
964
965```bash
966bao policy write spindle-policy spindle-policy.hcl
967bao auth enable approle
968bao write auth/approle/role/spindle \
969 token_policies="spindle-policy" \
970 token_ttl=1h \
971 token_max_ttl=4h \
972 bind_secret_id=true \
973 secret_id_ttl=0 \
974 secret_id_num_uses=0
975```
976
977Get the credentials:
978
979```bash
980# Get role ID (static)
981ROLE_ID=$(bao read -field=role_id auth/approle/role/spindle/role-id)
982
983# Generate secret ID
984SECRET_ID=$(bao write -f -field=secret_id auth/approle/role/spindle/secret-id)
985
986echo "Role ID: $ROLE_ID"
987echo "Secret ID: $SECRET_ID"
988```
989
990#### Create proxy configuration
991
992Create the credential files:
993
994```bash
995# Create directory for OpenBao files
996mkdir -p /tmp/openbao
997
998# Save credentials
999echo "$ROLE_ID" > /tmp/openbao/role-id
1000echo "$SECRET_ID" > /tmp/openbao/secret-id
1001chmod 600 /tmp/openbao/role-id /tmp/openbao/secret-id
1002```
1003
1004Create a proxy configuration file `/tmp/openbao/proxy.hcl`:
1005
1006```hcl
1007# OpenBao server connection
1008vault {
1009 address = "http://localhost:8200"
1010}
1011
1012# Auto-Auth using AppRole
1013auto_auth {
1014 method "approle" {
1015 mount_path = "auth/approle"
1016 config = {
1017 role_id_file_path = "/tmp/openbao/role-id"
1018 secret_id_file_path = "/tmp/openbao/secret-id"
1019 }
1020 }
1021
1022 # Optional: write token to file for debugging
1023 sink "file" {
1024 config = {
1025 path = "/tmp/openbao/token"
1026 mode = 0640
1027 }
1028 }
1029}
1030
1031# Proxy listener for spindle
1032listener "tcp" {
1033 address = "127.0.0.1:8201"
1034 tls_disable = true
1035}
1036
1037# Enable API proxy with auto-auth token
1038api_proxy {
1039 use_auto_auth_token = true
1040}
1041
1042# Enable response caching
1043cache {
1044 use_auto_auth_token = true
1045}
1046
1047# Logging
1048log_level = "info"
1049```
1050
1051#### Start the proxy
1052
1053Start OpenBao Proxy:
1054
1055```bash
1056bao proxy -config=/tmp/openbao/proxy.hcl
1057```
1058
1059The proxy will authenticate with OpenBao and start listening on
1060`127.0.0.1:8201`.
1061
1062#### Configure spindle
1063
1064Set these environment variables for spindle:
1065
1066```bash
1067export SPINDLE_SERVER_SECRETS_PROVIDER=openbao
1068export SPINDLE_SERVER_SECRETS_OPENBAO_PROXY_ADDR=http://127.0.0.1:8201
1069export SPINDLE_SERVER_SECRETS_OPENBAO_MOUNT=spindle
1070```
1071
1072On startup, spindle will now connect to the local proxy,
1073which handles all authentication automatically.
1074
1075### Production setup for proxy
1076
1077For production, you'll want to run the proxy as a service:
1078
1079Place your production configuration in
1080`/etc/openbao/proxy.hcl` with proper TLS settings for the
1081vault connection.
1082
1083### Verifying setup
1084
1085Test the proxy directly:
1086
1087```bash
1088# Check proxy health
1089curl -H "X-Vault-Request: true" http://127.0.0.1:8201/v1/sys/health
1090
1091# Test token lookup through proxy
1092curl -H "X-Vault-Request: true" http://127.0.0.1:8201/v1/auth/token/lookup-self
1093```
1094
1095Test OpenBao operations through the server:
1096
1097```bash
1098# List all secrets
1099bao kv list spindle/
1100
1101# Add a test secret via the spindle API, then check it exists
1102bao kv list spindle/repos/
1103
1104# Get a specific secret
1105bao kv get spindle/repos/your_repo_path/SECRET_NAME
1106```
1107
1108### How it works
1109
1110- Spindle connects to OpenBao Proxy on localhost (typically
1111 port 8200 or 8201)
1112- The proxy authenticates with OpenBao using AppRole
1113 credentials
1114- All spindle requests go through the proxy, which injects
1115 authentication tokens
1116- Secrets are stored at
1117 `spindle/repos/{sanitized_repo_path}/{secret_key}`
1118- Repository paths like `did:plc:alice/myrepo` become
1119 `did_plc_alice_myrepo`
1120- The proxy handles all token renewal automatically
1121- Spindle no longer manages tokens or authentication
1122 directly
1123
1124### Troubleshooting
1125
1126**Connection refused**: Check that the OpenBao Proxy is
1127running and listening on the configured address.
1128
1129**403 errors**: Verify the AppRole credentials are correct
1130and the policy has the necessary permissions.
1131
1132**404 route errors**: The spindle KV mount probably doesn't
1133exist—run the mount creation step again.
1134
1135**Proxy authentication failures**: Check the proxy logs and
1136verify the role-id and secret-id files are readable and
1137contain valid credentials.
1138
1139**Secret not found after writing**: This can indicate policy
1140permission issues. Verify the policy includes both
1141`spindle/data/*` and `spindle/metadata/*` paths with
1142appropriate capabilities.
1143
1144Check proxy logs:
1145
1146```bash
1147# If running as systemd service
1148journalctl -u openbao-proxy -f
1149
1150# If running directly, check the console output
1151```
1152
1153Test AppRole authentication manually:
1154
1155```bash
1156bao write auth/approle/login \
1157 role_id="$(cat /tmp/openbao/role-id)" \
1158 secret_id="$(cat /tmp/openbao/secret-id)"
1159```
1160
1161# Migrating knots and spindles
1162
1163Sometimes, non-backwards compatible changes are made to the
1164knot/spindle XRPC APIs. If you host a knot or a spindle, you
1165will need to follow this guide to upgrade. Typically, this
1166only requires you to deploy the newest version.
1167
1168This document is laid out in reverse-chronological order.
1169Newer migration guides are listed first, and older guides
1170are further down the page.
1171
1172## Upgrading from v1.8.x
1173
1174After v1.8.2, the HTTP API for knots and spindles has been
1175deprecated and replaced with XRPC. Repositories on outdated
1176knots will not be viewable from the appview. Upgrading is
1177straightforward however.
1178
1179For knots:
1180
1181- Upgrade to the latest tag (v1.9.0 or above)
1182- Head to the [knot dashboard](https://tangled.org/settings/knots) and
1183 hit the "retry" button to verify your knot
1184
1185For spindles:
1186
1187- Upgrade to the latest tag (v1.9.0 or above)
1188- Head to the [spindle
1189 dashboard](https://tangled.org/settings/spindles) and hit the
1190 "retry" button to verify your spindle
1191
1192## Upgrading from v1.7.x
1193
1194After v1.7.0, knot secrets have been deprecated. You no
1195longer need a secret from the appview to run a knot. All
1196authorized commands to knots are managed via [Inter-Service
1197Authentication](https://atproto.com/specs/xrpc#inter-service-authentication-jwt).
1198Knots will be read-only until upgraded.
1199
1200Upgrading is quite easy, in essence:
1201
1202- `KNOT_SERVER_SECRET` is no more, you can remove this
1203 environment variable entirely
1204- `KNOT_SERVER_OWNER` is now required on boot, set this to
1205 your DID. You can find your DID in the
1206 [settings](https://tangled.org/settings) page.
1207- Restart your knot once you have replaced the environment
1208 variable
1209- Head to the [knot dashboard](https://tangled.org/settings/knots) and
1210 hit the "retry" button to verify your knot. This simply
1211 writes a `sh.tangled.knot` record to your PDS.
1212
1213If you use the nix module, simply bump the flake to the
1214latest revision, and change your config block like so:
1215
1216```diff
1217 services.tangled.knot = {
1218 enable = true;
1219 server = {
1220- secretFile = /path/to/secret;
1221+ owner = "did:plc:foo";
1222 };
1223 };
1224```
1225
1226# Hacking on Tangled
1227
1228We highly recommend [installing
1229Nix](https://nixos.org/download/) (the package manager)
1230before working on the codebase. The Nix flake provides a lot
1231of helpers to get started and most importantly, builds and
1232dev shells are entirely deterministic.
1233
1234To set up your dev environment:
1235
1236```bash
1237nix develop
1238```
1239
1240Non-Nix users can look at the `devShell` attribute in the
1241`flake.nix` file to determine necessary dependencies.
1242
1243## Running the appview
1244
1245The Nix flake also exposes a few `app` attributes (run `nix
1246flake show` to see a full list of what the flake provides),
1247one of the apps runs the appview with the `air`
1248live-reloader:
1249
1250```bash
1251TANGLED_DEV=true nix run .#watch-appview
1252
1253# TANGLED_DB_PATH might be of interest to point to
1254# different sqlite DBs
1255
1256# in a separate shell, you can live-reload tailwind
1257nix run .#watch-tailwind
1258```
1259
1260To authenticate with the appview, you will need Redis and
1261OAuth JWKs to be set up:
1262
1263```
1264# OAuth JWKs should already be set up by the Nix devshell:
1265echo $TANGLED_OAUTH_CLIENT_SECRET
1266z42ty4RT1ovnTopY8B8ekz9NuziF2CuMkZ7rbRFpAR9jBqMc
1267
1268echo $TANGLED_OAUTH_CLIENT_KID
12691761667908
1270
1271# if not, you can set it up yourself:
1272goat key generate -t P-256
1273Key Type: P-256 / secp256r1 / ES256 private key
1274Secret Key (Multibase Syntax): save this securely (eg, add to password manager)
1275 z42tuPDKRfM2mz2Kv953ARen2jmrPA8S9LX9tRq4RVcUMwwL
1276Public Key (DID Key Syntax): share or publish this (eg, in DID document)
1277 did:key:zDnaeUBxtG6Xuv3ATJE4GaWeyXM3jyamJsZw3bSPpxx4bNXDR
1278
1279# the secret key from above
1280export TANGLED_OAUTH_CLIENT_SECRET="z42tuP..."
1281
1282# Run Redis in a new shell to store OAuth sessions
1283redis-server
1284```
1285
1286## Running knots and spindles
1287
1288An end-to-end knot setup requires setting up a machine with
1289`sshd`, `AuthorizedKeysCommand`, and a Git user, which is
1290quite cumbersome. So the Nix flake provides a
1291`nixosConfiguration` to do so.
1292
1293<details>
1294 <summary><strong>macOS users will have to set up a Nix Builder first</strong></summary>
1295
1296 In order to build Tangled's dev VM on macOS, you will
1297 first need to set up a Linux Nix builder. The recommended
1298 way to do so is to run a [`darwin.linux-builder`
1299 VM](https://nixos.org/manual/nixpkgs/unstable/#sec-darwin-builder)
1300 and to register it in `nix.conf` as a builder for Linux
1301 with the same architecture as your Mac (`linux-aarch64` if
1302 you are using Apple Silicon).
1303
1304 > IMPORTANT: You must build `darwin.linux-builder` somewhere other than inside
1305 > the Tangled repo so that it doesn't conflict with the other VM. For example,
1306 > you can do
1307 >
1308 > ```shell
1309 > cd $(mktemp -d buildervm.XXXXX) && nix run nixpkgs#darwin.linux-builder
1310 > ```
1311 >
1312 > to store the builder VM in a temporary dir.
1313 >
1314 > You should read and follow [all the other intructions][darwin builder vm] to
1315 > avoid subtle problems.
1316
1317 Alternatively, you can use any other method to set up a
1318 Linux machine with Nix installed that you can `sudo ssh`
1319 into (in other words, root user on your Mac has to be able
1320 to ssh into the Linux machine without entering a password)
1321 and that has the same architecture as your Mac. See
1322 [remote builder
1323 instructions](https://nix.dev/manual/nix/2.28/advanced-topics/distributed-builds.html#requirements)
1324 for how to register such a builder in `nix.conf`.
1325
1326 > WARNING: If you'd like to use
1327 > [`nixos-lima`](https://github.com/nixos-lima/nixos-lima) or
1328 > [Orbstack](https://orbstack.dev/), note that setting them up so that `sudo
1329 > ssh` works can be tricky. It seems to be [possible with
1330 > Orbstack](https://github.com/orgs/orbstack/discussions/1669).
1331
1332</details>
1333
1334To begin, grab your DID from http://localhost:3000/settings.
1335Then, set `TANGLED_VM_KNOT_OWNER` and
1336`TANGLED_VM_SPINDLE_OWNER` to your DID. You can now start a
1337lightweight NixOS VM like so:
1338
1339```bash
1340nix run --impure .#vm
1341
1342# type `poweroff` at the shell to exit the VM
1343```
1344
1345This starts a knot on port 6444, a spindle on port 6555
1346with `ssh` exposed on port 2222.
1347
1348Once the services are running, head to
1349http://localhost:3000/settings/knots and hit "Verify". It should
1350verify the ownership of the services instantly if everything
1351went smoothly.
1352
1353You can push repositories to this VM with this ssh config
1354block on your main machine:
1355
1356```bash
1357Host nixos-shell
1358 Hostname localhost
1359 Port 2222
1360 User git
1361 IdentityFile ~/.ssh/my_tangled_key
1362```
1363
1364Set up a remote called `local-dev` on a git repo:
1365
1366```bash
1367git remote add local-dev git@nixos-shell:user/repo
1368git push local-dev main
1369```
1370
1371The above VM should already be running a spindle on
1372`localhost:6555`. Head to http://localhost:3000/settings/spindles and
1373hit "Verify". You can then configure each repository to use
1374this spindle and run CI jobs.
1375
1376Of interest when debugging spindles:
1377
1378```
1379# Service logs from journald:
1380journalctl -xeu spindle
1381
1382# CI job logs from disk:
1383ls /var/log/spindle
1384
1385# Debugging spindle database:
1386sqlite3 /var/lib/spindle/spindle.db
1387
1388# litecli has a nicer REPL interface:
1389litecli /var/lib/spindle/spindle.db
1390```
1391
1392If for any reason you wish to disable either one of the
1393services in the VM, modify [nix/vm.nix](/nix/vm.nix) and set
1394`services.tangled.spindle.enable` (or
1395`services.tangled.knot.enable`) to `false`.
1396
1397# Contribution guide
1398
1399## Commit guidelines
1400
1401We follow a commit style similar to the Go project. Please keep commits:
1402
1403* **atomic**: each commit should represent one logical change
1404* **descriptive**: the commit message should clearly describe what the
1405change does and why it's needed
1406
1407### Message format
1408
1409```
1410<service/top-level directory>/<affected package/directory>: <short summary of change>
1411
1412Optional longer description can go here, if necessary. Explain what the
1413change does and why, especially if not obvious. Reference relevant
1414issues or PRs when applicable. These can be links for now since we don't
1415auto-link issues/PRs yet.
1416```
1417
1418Here are some examples:
1419
1420```
1421appview/state: fix token expiry check in middleware
1422
1423The previous check did not account for clock drift, leading to premature
1424token invalidation.
1425```
1426
1427```
1428knotserver/git/service: improve error checking in upload-pack
1429```
1430
1431
1432### General notes
1433
1434- PRs get merged "as-is" (fast-forward)—like applying a patch-series
1435using `git am`. At present, there is no squashing—so please author
1436your commits as they would appear on `master`, following the above
1437guidelines.
1438- If there is a lot of nesting, for example "appview:
1439pages/templates/repo/fragments: ...", these can be truncated down to
1440just "appview: repo/fragments: ...". If the change affects a lot of
1441subdirectories, you may abbreviate to just the top-level names, e.g.
1442"appview: ..." or "knotserver: ...".
1443- Keep commits lowercased with no trailing period.
1444- Use the imperative mood in the summary line (e.g., "fix bug" not
1445"fixed bug" or "fixes bug").
1446- Try to keep the summary line under 72 characters, but we aren't too
1447fussed about this.
1448- Follow the same formatting for PR titles if filled manually.
1449- Don't include unrelated changes in the same commit.
1450- Avoid noisy commit messages like "wip" or "final fix"—rewrite history
1451before submitting if necessary.
1452
1453## Code formatting
1454
1455We use a variety of tools to format our code, and multiplex them with
1456[`treefmt`](https://treefmt.com). All you need to do to format your changes
1457is run `nix run .#fmt` (or just `treefmt` if you're in the devshell).
1458
1459## Proposals for bigger changes
1460
1461Small fixes like typos, minor bugs, or trivial refactors can be
1462submitted directly as PRs.
1463
1464For larger changes—especially those introducing new features, significant
1465refactoring, or altering system behavior—please open a proposal first. This
1466helps us evaluate the scope, design, and potential impact before implementation.
1467
1468Create a new issue titled:
1469
1470```
1471proposal: <affected scope>: <summary of change>
1472```
1473
1474In the description, explain:
1475
1476- What the change is
1477- Why it's needed
1478- How you plan to implement it (roughly)
1479- Any open questions or tradeoffs
1480
1481We'll use the issue thread to discuss and refine the idea before moving
1482forward.
1483
1484## Developer Certificate of Origin (DCO)
1485
1486We require all contributors to certify that they have the right to
1487submit the code they're contributing. To do this, we follow the
1488[Developer Certificate of Origin
1489(DCO)](https://developercertificate.org/).
1490
1491By signing your commits, you're stating that the contribution is your
1492own work, or that you have the right to submit it under the project's
1493license. This helps us keep things clean and legally sound.
1494
1495To sign your commit, just add the `-s` flag when committing:
1496
1497```sh
1498git commit -s -m "your commit message"
1499```
1500
1501This appends a line like:
1502
1503```
1504Signed-off-by: Your Name <your.email@example.com>
1505```
1506
1507We won't merge commits if they aren't signed off. If you forget, you can
1508amend the last commit like this:
1509
1510```sh
1511git commit --amend -s
1512```
1513
1514If you're submitting a PR with multiple commits, make sure each one is
1515signed.
1516
1517For [jj](https://jj-vcs.github.io/jj/latest/) users, you can run the following command
1518to make it sign off commits in the tangled repo:
1519
1520```shell
1521# Safety check, should say "No matching config key..."
1522jj config list templates.commit_trailers
1523# The command below may need to be adjusted if the command above returned something.
1524jj config set --repo templates.commit_trailers "format_signed_off_by_trailer(self)"
1525```
1526
1527Refer to the [jujutsu
1528documentation](https://jj-vcs.github.io/jj/latest/config/#commit-trailers)
1529for more information.