techaro.lol / anubis
Weighs the soul of incoming HTTP requests to stop AI crawlers
fork atom

test: introduce SSH based CI for non-native test hosts (#644)

* feat: ssh based CI

Signed-off-by: Xe Iaso <me@xeiaso.net>

* test: implement SSH ci with caches and github actions

Signed-off-by: Xe Iaso <me@xeiaso.net>

* test(ssh-ci): fix known hosts secret

Signed-off-by: Xe Iaso <me@xeiaso.net>

* test(ssh-ci): clone the repo, that's important

Signed-off-by: Xe Iaso <me@xeiaso.net>

* chore: spelling

Signed-off-by: Xe Iaso <me@xeiaso.net>

* test(ssh-ci): speed up ci by prebaking the SSH CI image

Signed-off-by: Xe Iaso <me@xeiaso.net>

* test(ssh-ci): set -euo

Signed-off-by: Xe Iaso <me@xeiaso.net>

* test(ssh-ci): enable pull_request_target so things work

Signed-off-by: Xe Iaso <me@xeiaso.net>

* chore: spelling

Signed-off-by: Xe Iaso <me@xeiaso.net>

* test(ssh-ci): oh goody it's broken

Signed-off-by: Xe Iaso <me@xeiaso.net>

* test(ssh-ci): add cronjob to rebuild ci runner image

Signed-off-by: Xe Iaso <me@xeiaso.net>

* test(ssh-ci): also run yeet

Signed-off-by: Xe Iaso <me@xeiaso.net>

* test(ssh-ci): force git version for yeet

Signed-off-by: Xe Iaso <me@xeiaso.net>

* test(ssh-ci): run set -x in the container

Signed-off-by: Xe Iaso <me@xeiaso.net>

* test(ssh-ci): fix yeet?

Signed-off-by: Xe Iaso <me@xeiaso.net>

* test(ssh-ci): remove yeet for now

Signed-off-by: Xe Iaso <me@xeiaso.net>

* test(ssh-ci): disable for PRs for now

Signed-off-by: Xe Iaso <me@xeiaso.net>

---------

Signed-off-by: Xe Iaso <me@xeiaso.net>

authored by

Xe Iaso and committed by
GitHub f5140ae5 bbdee34f

+151 -4
unified split
+4
.github/actions/spelling/expect.txt
··· 6 6 anthro 7 7 anubis 8 8 anubistest 9 + apk 9 10 Applebot 10 11 archlinux 11 12 badregexes ··· 68 69 eerror 69 70 ellenjoe 70 71 enbyware 72 + euo 71 73 everyones 72 74 evilbot 73 75 evilsite ··· 117 119 inp 118 120 iss 119 121 isset 122 + itv 120 123 ivh 121 124 Jenomis 122 125 JGit ··· 246 249 uberspace 247 250 unixhttpd 248 251 unmarshal 252 + uuidgen 249 253 uvx 250 254 UXP 251 255 Varis
+3 -4
.github/workflows/docker.yml
··· 3 3 on: 4 4 workflow_dispatch: 5 5 push: 6 - branches: [ "main" ] 7 - tags: [ "v*" ] 6 + branches: ["main"] 7 + tags: ["v*"] 8 8 9 9 env: 10 10 DOCKER_METADATA_SET_OUTPUT_ENV: "true" ··· 55 55 run: | 56 56 brew bundle 57 57 58 - - name: Log into registry 58 + - name: Log into registry 59 59 uses: docker/login-action@74a5d142397b4f367a81961eba4e8cd7edddf772 # v3.4.0 60 60 with: 61 61 registry: ghcr.io ··· 77 77 DOCKER_REPO: ${{ env.IMAGE }} 78 78 SLOG_LEVEL: debug 79 79 80 - 81 80 - name: Generate artifact attestation 82 81 uses: actions/attest-build-provenance@db473fddc028af60658334401dc6fa3ffd8669fd # v2.3.0 83 82 with:
+36
.github/workflows/ssh-ci-runner-cron.yml
··· 1 + name: Regenerate ssh ci runner image 2 + 3 + on: 4 + # pull_request: 5 + # branches: ["main"] 6 + schedule: 7 + - cron: "0 0 1,8,15,22 * *" 8 + workflow_dispatch: 9 + 10 + permissions: 11 + pull-requests: write 12 + contents: write 13 + packages: write 14 + 15 + jobs: 16 + ssh-ci-rebuild: 17 + runs-on: ubuntu-latest 18 + steps: 19 + - name: Checkout code 20 + uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 21 + with: 22 + fetch-tags: true 23 + fetch-depth: 0 24 + persist-credentials: false 25 + - name: Log into registry 26 + uses: docker/login-action@343f7c4344506bcbf9b4de18042ae17996df046d # v3.0.0 27 + with: 28 + registry: ghcr.io 29 + username: ${{ github.repository_owner }} 30 + password: ${{ secrets.GITHUB_TOKEN }} 31 + - name: Set up Docker Buildx 32 + uses: docker/setup-buildx-action@b5ca514318bd6ebac0fb2aedd5d36ec1b5c232a2 # v3.10.0 33 + - name: Build and push 34 + run: | 35 + cd ./test/ssh-ci 36 + docker buildx bake --push
+36
.github/workflows/ssh-ci.yml
··· 1 + name: SSH CI 2 + 3 + on: 4 + push: 5 + branches: ["main"] 6 + # pull_request: 7 + # branches: ["main"] 8 + 9 + permissions: 10 + contents: read 11 + 12 + jobs: 13 + ssh: 14 + runs-on: ubuntu-24.04 15 + strategy: 16 + matrix: 17 + host: 18 + - ubuntu@riscv64.techaro.lol 19 + - ci@ppc64le.techaro.lol 20 + steps: 21 + - name: Checkout code 22 + uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2 23 + with: 24 + fetch-tags: true 25 + fetch-depth: 0 26 + persist-credentials: false 27 + - name: Install CI target SSH key 28 + uses: shimataro/ssh-key-action@d4fffb50872869abe2d9a9098a6d9c5aa7d16be4 # v2.7.0 29 + with: 30 + key: ${{ secrets.CI_SSH_KEY }} 31 + name: id_rsa 32 + known_hosts: ${{ secrets.CI_SSH_KNOWN_HOSTS }} 33 + - name: Run CI 34 + run: bash test/ssh-ci/rigging.sh ${{ matrix.host }} 35 + env: 36 + GITHUB_RUN_ID: ${{ github.run_id }}
+5
test/ssh-ci/Dockerfile
··· 1 + ARG ALPINE_VERSION=3.22 2 + 3 + FROM alpine:${ALPINE_VERSION} 4 + RUN apk add -U go nodejs git build-base git npm bash zstd brotli gzip 5 + LABEL org.opencontainers.image.source="https://github.com/TecharoHQ/anubis"
+26
test/ssh-ci/docker-bake.hcl
··· 1 + variable "ALPINE_VERSION" { default = "3.22" } 2 + 3 + group "default" { 4 + targets = [ 5 + "ci-runner", 6 + ] 7 + } 8 + 9 + target "ci-runner" { 10 + args = { 11 + ALPINE_VERSION = "3.22" 12 + } 13 + context = "." 14 + dockerfile = "./Dockerfile" 15 + platforms = [ 16 + "linux/amd64", 17 + "linux/arm64", 18 + "linux/arm/v7", 19 + "linux/ppc64le", 20 + "linux/riscv64", 21 + ] 22 + pull = true 23 + tags = [ 24 + "ghcr.io/techarohq/anubis/ci-runner:latest" 25 + ] 26 + }
+8
test/ssh-ci/in-container.sh
··· 1 + #!/usr/bin/env sh 2 + 3 + set -euo pipefail 4 + set -x 5 + 6 + npm ci 7 + npm run build 8 + SKIP_INTEGRATION=1 go test ./...
+33
test/ssh-ci/rigging.sh
··· 1 + #!/usr/bin/env bash 2 + 3 + set -euo pipefail 4 + [ ! -z "${DEBUG:-}" ] && set -x 5 + 6 + if [ "$#" -ne 1 ]; then 7 + echo "Usage: rigging.sh <user@host>" 8 + fi 9 + 10 + CIRunnerImage="ghcr.io/techarohq/anubis/ci-runner:latest" 11 + RunID=${GITHUB_RUN_ID:-$(uuidgen)} 12 + RunFolder="anubis/runs/${RunID}" 13 + Target="${1}" 14 + 15 + ssh "${Target}" uname -av 16 + ssh "${Target}" mkdir -p "${RunFolder}" 17 + git archive HEAD | ssh "${Target}" tar xC "${RunFolder}" 18 + 19 + ssh "${Target}" << EOF 20 + set -euo pipefail 21 + set -x 22 + mkdir -p "anubis/cache/{go,go-build,node}" 23 + podman pull ${CIRunnerImage} 24 + podman run --rm -it \ 25 + -v "\$HOME/${RunFolder}:/app/anubis" \ 26 + -v "\$HOME/anubis/cache/go:/root/go" \ 27 + -v "\$HOME/anubis/cache/go-build:/root/.cache/go-build" \ 28 + -v "\$HOME/anubis/cache/node:/root/.npm" \ 29 + -w /app/anubis \ 30 + ${CIRunnerImage} \ 31 + sh /app/anubis/test/ssh-ci/in-container.sh 32 + ssh "${Target}" rm -rf "${RunFolder}" 33 + EOF