julien.rbrt.fr
commits
WaitStep and TailStep now run in goroutines, and are tracked with a
stepCtx which has a timeout attached. Once stepCtx expires, the step is
killed with DestroyStep.
The default timeout is set to 5m, and is configureable using
SPINDLE_PIPELINES_STEP_TIMEOUT.
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.sh>
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.sh>
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.sh>
Signed-off-by: oppiliappan <me@oppi.li>
Signed-off-by: oppiliappan <me@oppi.li>
Signed-off-by: oppiliappan <me@oppi.li>
Signed-off-by: oppiliappan <me@oppi.li>
Signed-off-by: oppiliappan <me@oppi.li>
Signed-off-by: oppiliappan <me@oppi.li>
Signed-off-by: oppiliappan <me@oppi.li>
Signed-off-by: oppiliappan <me@oppi.li>
Signed-off-by: oppiliappan <me@oppi.li>
Signed-off-by: oppiliappan <me@oppi.li>
Signed-off-by: oppiliappan <me@oppi.li>
Signed-off-by: oppiliappan <me@oppi.li>
Signed-off-by: oppiliappan <me@oppi.li>
allows the spindle to dynamically configure the knots it is listening
to.
Signed-off-by: oppiliappan <me@oppi.li>
- add spindle to repo
- fix spindle.member lexicon field name
Signed-off-by: oppiliappan <me@oppi.li>
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.sh>
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.sh>
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.sh>
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.sh>
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.sh>
Signed-off-by: oppiliappan <me@oppi.li>
Signed-off-by: oppiliappan <me@oppi.li>
Signed-off-by: oppiliappan <me@oppi.li>
Signed-off-by: oppiliappan <me@oppi.li>
Signed-off-by: oppiliappan <me@oppi.li>
During setup, we register cleanup functions which get executed at the
end of the workflow goroutine (deferred exec of DestroyWorkflow).
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.sh>
As it sees use in both spindle and knotserver.
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.sh>
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.sh>
Logs are streamed for each running pipeline on a websocket at
/logs/{pipelineID}. engine.TailStep demuxes stdout and stderr from the
container's logs and pipes that out to corresponding stdout and stderr
channels.
These channels are maintained inside engine's container
struct, key'd by the pipeline ID, and protected by a read/write mutex.
engine.LogChannels fetches the stdout/stderr chans as recieve-only if
the pipeline is known to exist.
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.sh>
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.sh>
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.sh>
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.sh>
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.sh>
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.sh>
The engine package currently uses the docker client setup the pipeline
and execute steps. The flow is like so:
- setup pipeline network for all steps to join to
- create a volume for the nix store (to persist packages across steps)
- create a volume for the workspace directory
- build a nixery.dev URL with packages we want in the container
- execute each step command in a new container using the same image
It's pretty unfinished still. Things to be done:
- support for other registries; currently only works with nixpkgs
- custom nixery URL
- ... a lot more that I'm forgetting now
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.sh>
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.sh>
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.sh>
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.sh>
Signed-off-by: oppiliappan <me@oppi.li>
WaitStep and TailStep now run in goroutines, and are tracked with a
stepCtx which has a timeout attached. Once stepCtx expires, the step is
killed with DestroyStep.
The default timeout is set to 5m, and is configureable using
SPINDLE_PIPELINES_STEP_TIMEOUT.
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.sh>
Signed-off-by: oppiliappan <me@oppi.li>
Logs are streamed for each running pipeline on a websocket at
/logs/{pipelineID}. engine.TailStep demuxes stdout and stderr from the
container's logs and pipes that out to corresponding stdout and stderr
channels.
These channels are maintained inside engine's container
struct, key'd by the pipeline ID, and protected by a read/write mutex.
engine.LogChannels fetches the stdout/stderr chans as recieve-only if
the pipeline is known to exist.
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.sh>
The engine package currently uses the docker client setup the pipeline
and execute steps. The flow is like so:
- setup pipeline network for all steps to join to
- create a volume for the nix store (to persist packages across steps)
- create a volume for the workspace directory
- build a nixery.dev URL with packages we want in the container
- execute each step command in a new container using the same image
It's pretty unfinished still. Things to be done:
- support for other registries; currently only works with nixpkgs
- custom nixery URL
- ... a lot more that I'm forgetting now
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.sh>