Pull-based deploys with rollback

Why the server pulls

The obvious design — CI SSHes in and restarts the site — doesn’t work here, and that’s a good thing: sshd listens only on 127.0.0.1 behind the Cloudflare Tunnel, and CI jobs run in their own networks inside the runner’s Docker daemon.

So the server polls instead. Every 2 minutes it checks Gitea’s registry for a new :main using a read-only token. Nothing outside the server can trigger a deploy.

The token

A personal access token named vps-pull with package: Read only, saved on the server as secrets/registry_pull_token.txt (mode 600). The deploy script logs in with a private DOCKER_CONFIG directory, so this token doesn’t mix with any other Docker credentials.

The script

deploy-site.sh, installed as starter/bin/deploy-jonathandobson:

#!/usr/bin/env bash
set -euo pipefail
STARTER=/opt/vps-infra/starter
IMAGE=git.fromjon.com/<owner>/jonathandobson-com
STATE="$HOME/deploy"
export DOCKER_CONFIG="$HOME/.docker-deploy"
mkdir -p "$STATE" "$DOCKER_CONFIG"; chmod 700 "$DOCKER_CONFIG"
exec 9>"$STATE/deploy-site.lock"; flock -n 9 || exit 0      # one run at a time
log() { echo "$(date -u +%FT%TZ) $*" >> "$STATE/deploy-site.log"; }

cd "$STARTER"
compose() { docker compose -f compose.yml -f compose.jonathandobson.yml "$@"; }
set_image() { sed -i "s|^JONATHANDOBSON_IMAGE=.*|JONATHANDOBSON_IMAGE=$1|" .env; }

docker login git.fromjon.com -u <owner> --password-stdin < secrets/registry_pull_token.txt >/dev/null 2>&1
docker pull -q "$IMAGE:main" >/dev/null

new=$(docker image inspect --format '{{.Id}}' "$IMAGE:main")
cur=$(docker inspect --format '{{.Image}}' jonathandobson 2>/dev/null || true)
[ "$new" = "$cur" ] && exit 0                                          # nothing new
[ "$new" = "$(cat "$STATE/failed-image-id" 2>/dev/null || true)" ] && exit 0   # known bad

rev=$(docker image inspect --format '{{index .Config.Labels "org.opencontainers.image.revision"}}' "$IMAGE:main")
tag="$IMAGE:$rev"; docker tag "$IMAGE:main" "$tag"
prev=$(grep '^JONATHANDOBSON_IMAGE=' .env | cut -d= -f2-)

log "deploying $tag (was $prev)"; set_image "$tag"; compose up -d jonathandobson >/dev/null 2>&1
for _ in $(seq 1 24); do
  [ "$(docker inspect --format '{{.State.Health.Status}}' jonathandobson)" = healthy ] \
    && { rm -f "$STATE/failed-image-id"; log "healthy: $tag"; exit 0; }
  sleep 5
done
log "NOT HEALTHY after 2 min: rolling back to $prev"
echo "$new" > "$STATE/failed-image-id"; set_image "$prev"; compose up -d jonathandobson >/dev/null 2>&1
exit 1

Remembering the failed image ID matters: without it, a broken :main would be deployed and rolled back every 2 minutes until the next push.

Schedule it

In the admin user’s crontab (no root needed — the user is in the docker group):

*/2 * * * * /opt/vps-infra/starter/bin/deploy-jonathandobson >/dev/null 2>&1

Run it once by hand first; a second run should log nothing.

The result

A real change pushed to main:

Stage	Time after push
`verify` passed	~2¼ min
image built and pushed	+51 s
server deployed it, container healthy	228 s total

Rolling back by hand

Set JONATHANDOBSON_IMAGE in .env to an older :<commit> tag and run:

docker compose -f compose.yml -f compose.jonathandobson.yml up -d jonathandobson