studio (restore and verify)

Studio is not live as of 2026-11-05 (owner context). This runbook helps an operator confirm that state, restore service safely, and record dated evidence before any doc or status page marks studio live.

Current status rule

• Do not claim studio works until a dated verification supports it (snapshot 2026-11-05).
• Record host, revision, probe command, result, and timestamp for every live claim.
• If a dependency is down (database, Eve, auth, filesystem path), report the dependency state. Do not present studio as healthy.

Deploy

Deployment details are host and release specific. Confirm them from the studio release process and the target host unit before changing anything.

# Confirm the unit/process and revision on the target host (snapshot 2026-11-05).
# Record the host, unit name, revision, and date in the incident or restore note.

Do not deploy from this docs repository. tools/deploy.sh in this repo publishes docs only.

Restore

Restore only after identifying the failure class.

# 1. Check process and loopback listeners (addresses may differ by deployment).
ss -ltnp | rg '9191|9192|9193'

# 2. Check probe endpoints on the host loopback.
curl -fsS http://127.0.0.1:9192/healthz
curl -fsS http://127.0.0.1:9192/readyz

# 3. Check service logs through the host supervisor.
# Use the actual unit name on the target host.

If PostgreSQL is involved, follow the estate database restore procedure. Studio operator records are durable in PostgreSQL when a DSN is configured (studio:README.md operator records section). Do not perform ad hoc writes to production data.

Rollback

Use the release artifact or host rollback procedure that matches the failed deployment. Record:

• previous working revision,
• failed revision,
• rollback command or unit change,
• post rollback probe results with timestamps.

A rollback is complete only after health, readiness, auth, and a minimal API path succeed.

Incident: first three checks

Run these checks in order to separate a bad deploy from a bad dependency.

1. Service process and probes. Confirm studiod (or the target process) has the configured process running and loopback health/readiness respond. A missing listener points to process, bind address, or startup failure.
2. Configuration and dependencies. Check STUDIO_PG_DSN_FILE exists, is 0400 or stricter, and points to a reachable database. Check workspace, deploy, and sandbox roots. Check STUDIO_EVE_URL and token file only if Eve backed runs are required.
3. Auth and API behavior. Confirm Pocket ID sign in and an authenticated API call such as /api/projects or /api/runs. A 401 for unauthenticated calls is expected. A 503 with an honest fallback or unavailable message is a dependency state to record, not a UI bug to hide.

For user visible freshness, also check the status page. For estate wide auth behavior, see auth: Pocket ID.

Verification checklist before marking live

1. Record date, host, revision, and verifier.
2. Capture healthz and readyz results.
3. Capture one authenticated project or runs read.
4. Capture runs source health and SSE behavior, including honest degraded responses when a source is unavailable.
5. If Eve is configured, verify Eve backed run visibility without exposing secrets.
6. Update studio surface docs and status evidence with the dated result.

Rotate a key

Studio reads credentials from files (for example DSN and Eve token files), not from committed values. To rotate:

1. Create the replacement credential file with strict permissions.
2. Update the deployment to point at the new file path if needed.
3. Restart or reload the service using the host procedure.
4. Verify health, readiness, auth, and one authenticated API call.
5. Remove the old credential only after verification.

Never paste credential values into docs, tickets, logs, or commits.

Add or remove an integration

Connection and Eve integrations are configuration driven (STUDIO_CONNECTIONS, STUDIO_EVE_URL, and related file based auth settings in studio:README.md:269). A change takes effect only after the service reloads or restarts and a dated verification confirms the expected availability or honest unavailable state.