Pin your version
Never run :latest in production. An unpinned tag means an unplanned restart can pull a new version with new migrations at an arbitrary time.
Sim publishes images to GHCR, tagged by release alongside latest:
ghcr.io/simstudioai/simstudio
ghcr.io/simstudioai/realtime
ghcr.io/simstudioai/migrationsPin app, realtime, and migrations to the same tag. They share a database schema. An app newer than its migrations runs against a schema missing columns it expects; an app older than its migrations runs against a schema it does not understand. Mismatched tags is the most common self-inflicted upgrade failure.
app:
image:
tag: "v1.2.3" # a tag from the releases page
realtime:
image:
tag: "v1.2.3" # a tag from the releases page
migrations:
image:
tag: "v1.2.3" # a tag from the releases pageWhen image.tag is unset it defaults to the chart's appVersion, which moves when you upgrade the chart. Setting it explicitly decouples the two. For maximum determinism, pin image.digest instead:
app:
image:
digest: "sha256:..."Images track latest by default. To pin, set SIM_VERSION in .env to a tag from the releases page:
# .env
SIM_VERSION=v1.2.3One variable drives all three schema-coupled images, so they cannot drift apart. The cron service is deliberately excluded — it only makes HTTP calls and shares no schema, so it tracks latest unless you pin SIM_CRON_VERSION.
How migrations run
Migrations are Drizzle SQL files applied by a dedicated image.
- Kubernetes — an init container on the app Deployment. Every app pod waits for migrations to complete before it starts, so a failed migration blocks the rollout instead of starting an app against a mismatched schema. The container is idempotent, so it is a no-op on pods that start after the first.
- Docker Compose — a one-shot
migrationsservice withrestart: nothat runs before the app.
Migrations are forward-only. There are no down-migrations, which is why the pre-upgrade backup below is not optional.
Upgrade procedure
Read the release notes
Check the releases page for new required environment variables and breaking changes. When the chart's minor version moves, also read its README.md upgrade notes — chart upgrades occasionally rename or remove values keys.
Take a backup
Snapshot the database immediately before upgrading. Because migrations are forward-only, this snapshot is your only rollback path for schema changes.
# Managed Postgres — take a manual snapshot
aws rds create-db-snapshot --db-instance-identifier sim-db \
--db-snapshot-identifier "sim-pre-upgrade-$(date +%Y%m%d)"
# Bundled Postgres — the Helm chart's database is named `sim` by default
# (Docker Compose uses `simstudio`; the cloud example values files override to `simstudio`)
kubectl exec -n simstudio statefulset/sim-postgresql -- \
pg_dump -U postgres -Fc sim > "pre-upgrade-$(date +%F).dump"Rehearse against real data
Migration surprises are usually data-shaped rather than schema-shaped, so a staging run against a copy of production data catches far more than a run against an empty database.
Apply
helm upgrade sim ./helm/sim \
--namespace simstudio \
--values my-values.yamlPreview first if the chart version changed:
helm diff upgrade sim ./helm/sim -n simstudio --values my-values.yamlThen watch the rollout:
kubectl rollout status -n simstudio deploy/sim-app --timeout=10m
kubectl logs -n simstudio deploy/sim-app -c migrations --tail=100docker compose -f docker-compose.prod.yml pull
docker compose -f docker-compose.prod.yml up -d
docker compose -f docker-compose.prod.yml logs migrationsThere is a short window where the app is unavailable while containers restart. Compose has no rolling-update mechanism — plan a maintenance window, or run Kubernetes if you need zero-downtime upgrades.
Verify
Run the verification checklist. At minimum: sign in, open a workflow, execute it, upload a file, and confirm the background jobs are still firing.
When a migration fails
The app pods will not become ready — this is by design.
kubectl logs -n simstudio deploy/sim-app -c migrations --tail=200docker compose -f docker-compose.prod.yml logs migrationsCommon causes:
| Symptom | Cause |
|---|---|
permission denied to create extension "vector" | The database user lacks superuser rights. Create the pgvector extension manually as an admin, then re-run. |
| Connection refused / timeout | DATABASE_URL wrong, or the database is not reachable from the pod. Check network policy and credentials. |
| Lock timeout on a large table | A long-running query is blocking DDL. Drain traffic and retry during a quiet window. |
| Constraint violation | Pre-existing data conflicts with a new constraint. Capture the error, restore the pre-upgrade backup, and open an issue with the exact message. |
Do not manually edit the migrations table to skip a failed migration — the schema and Sim's expectations will diverge in ways that surface much later.
Rolling back
Application-only rollback (no migrations ran, or the new migrations are additive):
helm rollback sim -n simstudio# Docker Compose — set the previous tag and restart
docker compose -f docker-compose.prod.yml up -dRollback after a schema change requires restoring the database to the pre-upgrade backup, because migrations are forward-only:
- Scale the app to zero.
- Restore the pre-upgrade database snapshot.
- Redeploy the previous image tag on all three images.
- Verify.
This loses everything written since the snapshot. It is why the pre-upgrade backup and a staging rehearsal matter more here than in most systems.