Skip to content

Runbooks

Operational procedures for managing the Freeze Design webshop.

Environments

Two environments are live, both on a Hetzner VPS with Docker Compose in /opt/webshop. Staging (https://staging.freezedesign.nl, basic auth) runs docker-compose.staging.yml and deploys automatically from the staging branch. Production (https://www.freezedesign.nl, live since 2026-09-30, v1.60 since 2026-11-06) runs docker-compose.prod.yml and deploys tagged releases (vX.Y) from main with scripts/deploy-production.sh. freezedesign.nl and prod.freezedesign.nl redirect to www. DNS is at Openprovider and points straight at the VPSes; nothing sits in front of the shop.

Available Runbooks

In the order of the site navigation:

  • Deployment - Staging and production deploys, zero-downtime rollout, rollback, migrations
  • Disaster Recovery - Database backups and restore, media backup (rclone), VPS rebuild, SSL renewal
  • Monitoring - Sentry, Celery Beat schedule (Dutch time), healthchecks.io pings, celery-beat healthcheck and auto-restart, health endpoints, PostHog and the weekly snapshot, external uptime monitor (to do)
  • Alerting - Step-by-step response procedures per alert type
  • Discord Webhooks - Which Discord channel gets what, and the webhook env vars and secrets
  • ntfy notifications - The application alerts on the self-hosted ntfy server (next to Discord during the transition)
  • Eefje Feedback Intake - Intake-proces voor bugs en wensen via de feedback-repo
  • agent_api - De alleen-lezen API voor de AI-agent op de Freeze AI-VM: sleutels, scopes en IP-toegang
  • Synthetics - Het testproduct en de bestelcheck die elk uur vanaf de Freeze AI-VM draait
  • Productfeed (Merchant Center) - De productfeed voor Google Merchant Center, Meta en Pinterest
  • Inrichting na de groei-review - Afvinklijst van accounts, sleutels en schakelaars na de groei-review
  • Nieuwsbrief (Resend) - Aanmelden met double opt-in, Resend-audience, versturen, storingen
  • Reviews (Google) - Reviewverzoek na levering en het Google-reviewblok
  • Winkelmail bij een bestelling - Mail "Nieuwe bestelling" aan de winkel bij elke betaalde order
  • Betaalmethoden (Mollie) - Meer betaalmethoden dan iDEAL aanzetten via Mollie en de admin
  • Vindbaarheid (Search Console) - Search Console, Bing Webmaster Tools, sitemap en structured data
  • Dagplanning - De nachtelijke planner, de plan-PR, /dagstart en de bewaker
  • Voor Ivar (de lijst) - Het ene vastgepinde issue met alles wat alleen Ivar kan doen
  • E2E-quarantaine - Welke Playwright-tests geparkeerd staan en hoe ze weer meedraaien
  • Staging-inlog wisselen - Naam en wachtwoord van de basic auth op staging vernieuwen, en het GitHub-secret bijwerken
  • Catalogus van productie naar staging - Producten, prijzen en foto's van productie op staging zetten met scripts/catalogus-prod-naar-staging.sh

Quick Reference

On a VPS, as the deploy user. Neither server has a docker-compose.yml, so select the compose file once per SSH session:

cd /opt/webshop
export COMPOSE_FILE=docker-compose.prod.yml   # staging: docker-compose.staging.yml

Check Application Status

# Containers and their Docker health (celery-beat too, since #798)
docker compose ps

# Backend logs
docker compose logs backend --tail 100 -f

# Frontend logs
docker compose logs frontend --tail 100 -f

Restart Services

docker compose restart backend
docker compose restart frontend
docker compose restart celery celery-beat

A restart keeps the old image and the env file the container was created with. After a change to backend/.env.production (or .env.staging), recreate instead: docker compose up -d backend celery celery-beat.

Database Access

# Credentials come from backend/.env.production (POSTGRES_USER / POSTGRES_DB)
docker compose exec db psql -U "$POSTGRES_USER" -d "$POSTGRES_DB"

Incident Response

Priority Levels

Level Response Time Examples
P1 Immediate Site down, payment failures
P2 1 hour Major feature broken
P3 4 hours Minor bugs, degraded performance
P4 Next business day UI issues, minor improvements

Escalation Path

For a solo-operator setup, "escalation" means increasing your own response urgency. See the Alerting Runbook for detailed response procedures per alert type.