Launch ops checklist
Short checklist for release / high-traffic days. Pair with Monitoring for endpoint details.
Before send
Staging auto-deploys from main; production Google connect/reconnect smokes
(WP-00, #3841) run at the next production release, not before staging send.
Staging (2026-09-22, Email-ready v1)
-
GET /api/healthon staging returns200 {"status":"ok"}(verified 2026-09-22) - Google connect smoke on staging (founder or agent with test account)
- Google reconnect smoke on staging (revoke, banner, refresh)
- Production Google connect smoke (deferred, #3841)
- Production Google reconnect smoke (deferred, #3841)
-
sse_connection_degradedPostHog alert uses a trailing 60 minute window (see table below)
All environments
-
GET /api/healthreturns200 {"status":"ok"} - Sync
GET /health/liveandGET /health/readyare healthy; logs showexecution=activewhen Google sync is enabled - Deploy Discord health webhook is configured (see CI/CD workflows)
- PostHog Error Tracking is receiving
$exceptionfrom web (open a staging page and trigger a handled test if needed) - Confirm
sync_health_snapshotevents arrive every ~5 minutes in PostHog - Confirm the Meeting dashboard
is readable: empty server tiles are no-data until
booking_operationships, not 100% success. Runbook: Meeting monitoring
Alerts to create in PostHog (or Discord)
Alert on sync_health_snapshot properties (low cardinality — safe to alert):
| Signal | Suggested threshold |
|---|---|
connections.actionRequired | rising day over day, or absolute > 0 for 10+ minutes under load |
connections.oldestImportingAgeMs | > 3600000 (one hour) while any connection is still importing |
jobs.failed | rising vs baseline |
jobs.oldestDueAgeMs | > 5 minutes while execution=active |
freshness.percentOver30s | sustained spike vs quiet baseline. Sample is live connections only (healthy, delayed, catchingUp), not actionRequired or disconnected. |
Also watch:
- Web
$exceptionrate (Error Tracking)
Alerts that already exist in PostHog
Created by hand in the PostHog UI or via the PostHog MCP; all evaluate hourly and email the founder's PostHog account. Check the alerts page before creating another one.
| Alert | Insight | Fires when |
|---|---|---|
| Sync job terminal failure | hourly count of sync_job_terminal_failure | count above 0 in the current hour |
| Sync reconcile sweep starved (production) | sync_reconcile_sweep completions, trailing 45 minutes, production | count below 1 |
| SSE connection degraded burst (production) | sse_connection_degraded, trailing 60 minutes, production | count above 3 |
The SSE alert samples a trailing 60-minute window once an hour, so a burst
that straddles two checks can be under-counted. Widen the insight's window to
90 minutes if that bites; 15-minute evaluation needs a PostHog add-on. The
web app offers a sidebar-footer Refresh control for the same condition
(SidebarRefreshButton) after the stream has been down for 30 seconds.
Meeting launch checks (exhausted recovery, oldest pending, infrastructure failure rate, missing heartbeat) are defined in Meeting monitoring. They are not armed and must not notify a new recipient until the release owner confirms the channel. Hourly evaluation is coarser than the 5-minute pending SLO.
Welcome email launch checks (provider complaint rate above 0.1%, non-zero
failed_count_24h on email_send_heartbeat) are defined in
Welcome email staging verification. Same
not armed rule until the release owner confirms the channel.
During launch
- Watch Sync health snapshot + Error Tracking side by side
- If calendars look stuck: check SSE (
sse_connection_degraded), then Sync diagnostic routes (see Troubleshoot) - Both
compass-backendandcompass-synclogs are in PostHog Logs. Express errors carrymethod,path,status,userId, andstack— filter by service and status for triage
After
- Resolve or suppress any new Error Tracking noise
- Note any
actionRequired/ delayed cohorts for follow-up reconnect email