regnault.dev
forked from
tangled.org/site
engineering blog at https://blog.tangled.sh
1---
2atroot: true
3template:
4slug: ci
5title: introducing spindle
6subtitle: tangled's new CI runner is now generally available
7date: 2025-08-06
8authors:
9 - name: Anirudh
10 email: anirudh@tangled.sh
11 handle: icyphox.sh
12 - name: Akshay
13 email: akshay@tangled.sh
14 handle: oppi.li
15---
16
17Since launching Tangled, continuous integration has
18consistently topped our feature request list. Today, CI is
19no longer a wishlist item, but a fully-featured reality.
20
21Meet **spindle**: Tangled's new CI runner built atop Nix and
22AT Protocol. In typical Tangled fashion we've been
23dogfooding spindle for a while now; this very blog post
24you're reading was [built and published using
25spindle](https://tangled.sh/@tangled.sh/site/pipelines/452/workflow/deploy.yaml).
26
27Tangled is a new social-enabled Git collaboration platform,
28[read our intro](/intro) for more about the project.
29
30
31
32## how spindle works
33
34Spindle is designed around simplicity and the decentralized
35nature of the AT Protocol. In ingests "pipeline" records and
36emits job status updates.
37
38When you push code or open a pull request, the knot hosting
39your repository emits a pipeline event
40(`sh.tangled.pipeline`). Running as a dedicated service,
41spindle subscribes to these events via websocket connections
42to your knot.
43
44Once triggered, spindle reads your pipeline manifest, spins
45up the necessary execution environment (covered below), and
46runs your defined workflow steps. Throughout execution, it
47streams real-time logs and status updates
48(`sh.tangled.pipeline.status`) back through websockets,
49which the Tangled appview subscribes to for live updates.
50
51Over at the appview, these updates are ingested and stored,
52and logs are streamed live.
53
54## spindle pipelines
55
56The pipeline manifest is defined in YAML, and should be
57relatively familiar to those that have used other CI
58solutions. Here's a minimal example:
59
60```yaml
61# test.yaml
62
63when:
64 - event: ["push", "pull_request"]
65 branch: ["master"]
66
67dependencies:
68 nixpkgs:
69 - go
70
71steps:
72 - name: run all tests
73 environment:
74 CGO_ENABLED: 1
75 command: |
76 go test -v ./...
77```
78
79You can read the [full manifest spec
80here](https://tangled.sh/@tangled.sh/core/blob/master/docs/spindle/pipeline.md),
81but the `dependencies` block is the real interesting bit.
82Dependencies for your workflow, like Go, Node.js, Python
83etc. can be pulled in from nixpkgs.
84[Nixpkgs](https://github.com/nixos/nixpkgs/) -- for the
85uninitiated -- is a vast collection of packages for the Nix
86package manager. Fortunately, you needn't know nor care
87about Nix to use it! Just head to https://search.nixos.org
88to find your package of choice (I'll bet 1€ that it's
89there[^1]), toss it in the list and run your build. The
90Nix-savvy of you lot will be happy to know that you can use
91custom registries too.
92
93[^1]: I mean, if it isn't there, it's nowhere.
94
95Workflow manifests are intentionally simple. We do not want
96to include a "marketplace" of workflows or complex job
97orchestration. The bulk of the work should be offloaded to a
98build system, and CI should be used simply for finishing
99touches. That being said, this is still the first revision
100for CI, there is a lot more on the roadmap!
101
102Let's take a look at how spindle executes workflow steps.
103
104## workflow execution
105
106At present, the spindle "engine" supports just the Docker
107backend[^2]. Podman is known to work with the Docker socket
108feature enabled. Each step is run in a separate container,
109with the `/tangled/workspace` and `/nix` volumes persisted
110across steps.
111
112[^2]: Support for additional backends like Firecracker are
113 planned. Contributions welcome!
114
115The container image is built using
116[Nixery](https://nixery.dev). Nixery is a nifty little tool
117that takes a path-separated set of Nix packages and returns
118an OCI image with each package in a separate layer. Try this
119in your terminal if you've got Docker installed:
120
121```
122docker run nixery.dev/bash/hello-go hello-go
123```
124
125This should output `Hello, world!`. This is running the
126[hello-go](https://search.nixos.org/packages?channel=25.05&show=hello-go)
127package from nixpkgs.
128
129Nixery is super handy since we can construct these images
130for CI environments on the fly, with all dependencies baked
131in, and the best part: caching for commonly used packages is
132free thanks to Docker (pre-existing layers get reused). We
133run a Nixery instance of our own at
134https://nixery.tangled.sh but you may override that if you
135choose to.
136
137## debugging CI
138
139We understand that debugging CI can be the worst. There are
140two parts to this problem:
141
142- CI services often bring their own workflow definition
143 formats and it can sometimes be difficult to know why the
144 workflow won't run or why the workflow definition is
145 incorrect
146- The CI job itself fails, but this has more to do with the
147 build system of choice
148
149To mend the first problem: we are making use of git
150[push-options](https://git-scm.com/docs/git-push#Documentation/git-push.txt--ooption).
151When you push to a repository with an option like so:
152
153```
154git push origin master -o verbose-ci
155```
156
157The server runs a basic set of analysis rules on your
158workflow file, and reports any errors:
159
160```
161λ git push origin main -o verbose-ci
162 .
163 .
164 .
165 .
166remote: error: failed to parse workflow(s):
167remote: - at .tangled/workflows/fmt.yml: yaml: line 14: did not find expected key
168remote:
169remote: warning(s) on pipeline:
170remote: - at build.yml: workflow skipped: did not match trigger push
171```
172
173The analysis performed at the moment is quite basic (expect
174it to get better over time), but it is already quite useful
175to help debug workflows that don't trigger!
176
177## pipeline secrets
178
179Secrets are a bit tricky since atproto has no notion of
180private data. Secrets are instead written directly from the
181appview to the spindle instance using [service
182auth](https://docs.bsky.app/docs/api/com-atproto-server-get-service-auth).
183In essence, the appview makes a signed request using the
184logged-in user's DID key; spindle verifies this signature by
185fetching the public key from the DID document.
186
187
188
189The secrets themselves are stored in a secret manager. By
190default, this is the same sqlite database that spindle uses.
191This is *fine* for self-hosters. The hosted, flagship
192instance at https://spindle.tangled.sh however uses
193[OpenBao](https://openbao.org), an OSS fork of HashiCorp
194Vault.
195
196## get started now
197
198You can run your own spindle instance pretty easily: the
199[spindle self-hosting
200guide](https://tangled.sh/@tangled.sh/core/blob/master/docs/spindle/hosting.md)
201should have you covered. Once done, head to your
202repository's settings tab and set it up! Doesn't work? Feel
203free to pop into [Discord](https://chat.tangled.sh) to get
204help -- we have a nice little crew that's always around to
205help.
206
207All Tangled users have access to our hosted spindle
208instance, free of charge[^3]. You don't have any more
209excuses to not migrate to Tangled now -- [get
210started](https://tangled.sh/login) with your AT Protocol
211account today.
212
213[^3]: We can't promise we won't charge for it at some point
214 but there will always be a free tier.