lewis.moe
Monorepo for Tangled
docs: update spindle pipelines workflow syntax documentation
+130
-54
docs/spindle/pipeline.md
+130
-54
docs/spindle/pipeline.md
···
1
-
# spindle pipeline manifest
1
+
# spindle pipelines
2
+
3
+
Spindle workflows allow you to write CI/CD pipelines in a simple format. They're located in the `.tangled/workflows` directory at the root of your repository, and are defined using YAML.
4
+
5
+
The fields are:
2
6
3
-
Spindle pipelines are defined under the `.tangled/workflows` directory in a
4
-
repo. Generally:
7
+
- [Trigger](#trigger): A **required** field that defines when a workflow should be triggered.
8
+
- [Engine](#engine): A **required** field that defines which engine a workflow should run on.
9
+
- [Clone options](#clone-options): An **optional** field that defines how the repository should be cloned.
10
+
- [Dependencies](#dependencies): An **optional** field that allows you to list dependencies you may need.
11
+
- [Environment](#environment): An **optional** field that allows you to define environment variables.
12
+
- [Steps](#steps): An **optional** field that allows you to define what steps should run in the workflow.
5
13
6
-
* Pipelines are defined in YAML.
7
-
* Workflows can run using different *engines*.
14
+
## Trigger
8
15
9
-
The most barebones workflow looks like this:
16
+
The first thing to add to a workflow is the trigger, which defines when a workflow runs. This is defined using a `when` field, which takes in a list of conditions. Each condition has the following fields:
17
+
18
+
- `event`: This is a **required** field that defines when your workflow should run. It's a list that can take one or more of the following values:
19
+
- `push`: The workflow should run every time a commit is pushed to the repository.
20
+
- `pull_request`: The workflow should run every time a pull request is made or updated.
21
+
- `manual`: The workflow can be triggered manually.
22
+
- `branch`: This is a **required** field that defines which branches the workflow should run for. If used with the `push` event, commits to the branch(es) listed here will trigger the workflow. If used with the `pull_request` event, updates to pull requests targeting the branch(es) listed here will trigger the workflow. This field has no effect with the `manual` event.
23
+
24
+
For example, if you'd like define a workflow that runs when commits are pushed to the `main` and `develop` branches, or when pull requests that target the `main` branch are updated, or manually, you can do so with:
10
25
11
26
```yaml
12
27
when:
13
-
- event: ["push"]
28
+
- event: ["push", "manual"]
29
+
branch: ["main", "develop"]
30
+
- event: ["pull_request"]
14
31
branch: ["main"]
32
+
```
15
33
34
+
## Engine
35
+
36
+
Next is the engine on which the workflow should run, defined using the **required** `engine` field. The currently supported engines are:
37
+
38
+
- `nixery`: This uses an instance of [Nixery](https://nixery.dev) to run steps, which allows you to add [dependencies](#dependencies) from [Nixpkgs](https://github.com/NixOS/nixpkgs). You can search for packages on https://search.nixos.org, and there's a pretty good chance the package(s) you're looking for will be there.
39
+
40
+
Example:
41
+
42
+
```yaml
16
43
engine: "nixery"
44
+
```
45
+
46
+
## Clone options
17
47
18
-
# optional
48
+
When a workflow starts, the first step is to clone the repository. You can customize this behavior using the **optional** `clone` field. It has the following fields:
49
+
50
+
- `skip`: Setting this to `true` will skip cloning the repository. This can be useful if your workflow is doing something that doesn't require anything from the repository itself. This is `false` by default.
51
+
- `depth`: This sets the number of commits, or the "clone depth", to fetch from the repository. For example, if you set this to 2, the last 2 commits will be fetched. By default, the depth is set to 1, meaning only the most recent commit will be fetched, which is the commit that triggered the workflow.
52
+
- `submodules`: If you use [git submodules](https://git-scm.com/book/en/v2/Git-Tools-Submodules) in your repository, setting this field to `true` will recursively fetch all submodules. This is `false` by default.
53
+
54
+
The default settings are:
55
+
56
+
```yaml
19
57
clone:
20
58
skip: false
21
-
depth: 50
22
-
submodules: true
59
+
depth: 1
60
+
submodules: false
23
61
```
24
62
25
-
The `when` and `engine` fields are required, while every other aspect
26
-
of how the definition is parsed is up to the engine. Currently, a spindle
27
-
provides at least one of these built-in engines:
63
+
## Dependencies
28
64
29
-
## `nixery`
65
+
Usually when you're running a workflow, you'll need additional dependencies. The `dependencies` field lets you define which dependencies to get, and from where. It's a key-value map, with the key being the registry to fetch dependencies from, and the value being the list of dependencies to fetch.
30
66
31
-
The Nixery engine uses an instance of [Nixery](https://nixery.dev) to run
32
-
steps that use dependencies from [Nixpkgs](https://github.com/NixOS/nixpkgs).
33
-
34
-
Here's an example that uses all fields:
67
+
Say you want to fetch Node.js and Go from `nixpkgs`, and a package called `my_pkg` you've made from your own registry at your repository at `https://tangled.sh/@example.com/my_pkg`. You can define those dependencies like so:
35
68
36
69
```yaml
37
-
# build_and_test.yaml
38
-
when:
39
-
- event: ["push", "pull_request"]
40
-
branch: ["main", "develop"]
41
-
- event: ["manual"]
42
-
43
70
dependencies:
44
-
## from nixpkgs
71
+
# nixpkgs
45
72
nixpkgs:
46
73
- nodejs
47
-
## custom registry
48
-
git+https://tangled.sh/@oppi.li/statix:
49
-
- statix
74
+
- go
75
+
# custom registry
76
+
git+https://tangled.sh/@example.com/my_pkg:
77
+
- my_pkg
78
+
```
50
79
51
-
steps:
52
-
- name: "Install dependencies"
53
-
command: "npm install"
54
-
environment:
55
-
NODE_ENV: "development"
56
-
CI: "true"
80
+
Now these dependencies are available to use in your workflow!
57
81
58
-
- name: "Run linter"
59
-
command: "npm run lint"
82
+
## Environment
83
+
84
+
The `environment` field allows you define environment variables that will be available throughout the entire workflow. **Do not put secrets here, these environment variables are visible to anyone viewing the repository. You can add secrets for pipelines in your repository's settings.**
85
+
86
+
Example:
87
+
88
+
```yaml
89
+
environment:
90
+
GOOS: "linux"
91
+
GOARCH: "arm64"
92
+
NODE_ENV: "production"
93
+
MY_ENV_VAR: "MY_ENV_VALUE"
94
+
```
60
95
61
-
- name: "Run tests"
62
-
command: "npm test"
63
-
environment:
64
-
NODE_ENV: "test"
65
-
JEST_WORKERS: "2"
96
+
## Steps
66
97
67
-
- name: "Build application"
98
+
The `steps` field allows you to define what steps should run in the workflow. It's a list of step objects, each with the following fields:
99
+
100
+
- `name`: This field allows you to give your step a name. This name is visible in your workflow runs, and is used to describe what the step is doing.
101
+
- `command`: This field allows you to define a command to run in that step. The step is run in a Bash shell, and the logs from the command will be visible in the pipelines page on the Tangled website. The [dependencies](#dependencies) you added will be available to use here.
102
+
- `environment`: Similar to the global [environment](#environment) config, this **optional** field is a key-value map that allows you to set environment variables for the step. **Do not put secrets here, these environment variables are visible to anyone viewing the repository. You can add secrets for pipelines in your repository's settings.**
103
+
104
+
Example:
105
+
106
+
```yaml
107
+
steps:
108
+
- name: "Build backend"
109
+
command: "go build"
110
+
environment:
111
+
GOOS: "darwin"
112
+
GOARCH: "arm64"
113
+
- name: "Build frontend"
68
114
command: "npm run build"
69
115
environment:
70
116
NODE_ENV: "production"
117
+
```
71
118
72
-
environment:
73
-
BUILD_NUMBER: "123"
74
-
GIT_BRANCH: "main"
119
+
## Complete workflow
75
120
76
-
## current repository is cloned and checked out at the target ref
77
-
## by default.
121
+
```yaml
122
+
# .tangled/workflows/build.yml
123
+
124
+
when:
125
+
- event: ["push", "manual"]
126
+
branch: ["main", "develop"]
127
+
- event: ["pull_request"]
128
+
branch: ["main"]
129
+
130
+
engine: "nixery"
131
+
132
+
# using the default values
78
133
clone:
79
134
skip: false
80
-
depth: 50
81
-
submodules: true
82
-
```
135
+
depth: 1
136
+
submodules: false
83
137
84
-
## git push options
138
+
dependencies:
139
+
# nixpkgs
140
+
nixpkgs:
141
+
- nodejs
142
+
- go
143
+
# custom registry
144
+
git+https://tangled.sh/@example.com/my_pkg:
145
+
- my_pkg
85
146
86
-
These are push options that can be used with the `--push-option (-o)` flag of git push:
147
+
environment:
148
+
GOOS: "linux"
149
+
GOARCH: "arm64"
150
+
NODE_ENV: "production"
151
+
MY_ENV_VAR: "MY_ENV_VALUE"
152
+
153
+
steps:
154
+
- name: "Build backend"
155
+
command: "go build"
156
+
environment:
157
+
GOOS: "darwin"
158
+
GOARCH: "arm64"
159
+
- name: "Build frontend"
160
+
command: "npm run build"
161
+
environment:
162
+
NODE_ENV: "production"
163
+
```
87
164
88
-
- `verbose-ci`, `ci-verbose`: enables diagnostics reporting for the CI pipeline, allowing you to see any issues when you push.
89
-
- `skip-ci`, `ci-skip`: skips triggering the CI pipeline.
165
+
If you want another example of a workflow, you can look at the one [Tangled uses to build the project](https://tangled.sh/@tangled.sh/core/blob/master/.tangled/workflows/build.yml).