> ## Documentation Index > Fetch the complete documentation index at: https://gladlabs.mintlify.site/llms.txt > Use this file to discover all available pages before exploring further. # Secret rotation # Secret Rotation Runbook **Last reviewed:** 2026-05-08 **Audience:** solo operator (Matt) at 2am during an incident, or doing scheduled rotation **Prereqs:** Local PC online, Docker running, gh CLI authed, poindexter CLI installed, `~/.poindexter/bootstrap.toml` accessible, `POINDEXTER_SECRET_KEY` set Every secret in Poindexter has a rotation procedure. Most live in `app_settings` with `is_secret=true` and are encrypted at rest via `pgcrypto` — see [`src/cofounder_agent/plugins/secrets.py`](../../src/cofounder_agent/plugins/secrets.py) for how that works. Two secrets (`database_url` and `POINDEXTER_SECRET_KEY` itself) are bootstrap-only and live in `~/.poindexter/bootstrap.toml`. This runbook lists every known secret, where to obtain a fresh value, how to set the new value, and what to restart to pick it up. *** ## Scenarios this covers * Routine 90-day rotation of API keys * Emergency rotation after suspected compromise * Re-seeding after the encryption key was lost (see [`disaster-recovery.md`](./disaster-recovery) CONFIG-2) * Updating a third-party token that the provider expired / regenerated For initial provisioning, see [`local-development-setup.md`](./local-development-setup). For incident triage, see [`incident-response.md`](./incident-response). *** ## Quick triage flowchart ``` What kind of rotation is this? Routine (calendar / proactive) -> Pick the secret, follow its procedure below. Forced by provider (key expired, vendor rotated) -> Same procedure but you don't get to control timing — the integration is already broken. Suspected compromise (logs show unexpected access, key leaked) -> Rotate ALL of: every OAuth client secret (via `poindexter auth migrate-*`), jwt_secret_key, revalidate_secret, every webhook secret. Run "Re-seed all" below. Lost POINDEXTER_SECRET_KEY (encrypted blobs unreadable) -> See disaster-recovery.md CONFIG-2, then come back here and follow "Re-seeding from scratch" for every secret you actually use. ``` *** ## How rotation works (quick model) Every encrypted secret follows this lifecycle: 1. **Generate** a new value (via the provider's UI / API, or `openssl rand`) 2. **Set** the new value into `app_settings` via `poindexter settings set "" --secret` or the `set_secret(...)` Python helper. Either path encrypts under the current `POINDEXTER_SECRET_KEY`. 3. **Restart** any in-process consumers so they re-fetch from `app_settings`. Most async consumers call `await site_config.get_secret(...)` per-request and don't need a restart; some sync constructors cache at startup and DO. 4. **Verify** the rotation by triggering a real downstream call (test webhook, test alert, etc.). 5. **Revoke** the old value at the provider (when applicable). The `poindexter settings set "" --secret` CLI (or direct DB UPDATE, or the `set_secret` Python helper) is your write surface. The `poindexter settings get ` command shows `******* (encrypted)` for `is_secret=true` rows; pass `poindexter settings get --reveal` to decrypt and print the plaintext (value to stdout, exposure warning to stderr) when you need to read one back. *** ## Inventory — every known secret Pulled from `app_settings WHERE is_secret = TRUE` plus the two bootstrap secrets. Refreshed 2026-05-08. | Key | Where it lives | Used for | Rotate via | | ---------------------------------------------- | --------------------------------------------------- | ------------------------------------- | ------------------------------------------------------------------ | | `database_url` | `bootstrap.toml` | DB connection | [§ database\_url](#database_url) | | `POINDEXTER_SECRET_KEY` | `bootstrap.toml` (env) | Encrypts every other secret | [§ POINDEXTER\_SECRET\_KEY](#poindexter_secret_key) | | `_oauth_client_secret` | `app_settings` (encrypted) | OAuth 2.1 worker auth (per consumer) | [§ OAuth client secrets](#oauth-client-secrets) | | `revalidate_secret` | `app_settings` (encrypted) | Vercel ISR revalidation header | [§ revalidate\_secret](#revalidate_secret) | | `jwt_secret_key` | `app_settings` (encrypted) | JWT signing | [§ jwt\_secret\_key](#jwt_secret_key) | | `secret_key` | `app_settings` (encrypted) | App-wide signing | [§ secret\_key](#secret_key) | | `openclaw_webhook_token` | `app_settings` (encrypted) | OpenClaw → worker auth | [§ openclaw\_webhook\_token](#openclaw_webhook_token) | | `alertmanager_webhook_token` | `app_settings` (encrypted) | Alertmanager → worker auth | [§ alertmanager\_webhook\_token](#alertmanager_webhook_token) | | `telegram_bot_token` | `app_settings` (encrypted) | Telegram bot API | [§ telegram\_bot\_token](#telegram_bot_token) | | `discord_bot_token` | `app_settings` (encrypted) | Discord bot API | [§ discord\_bot\_token](#discord_bot_token) | | `discord_voice_bot_token` | `app_settings` (encrypted) | Discord voice bot | [§ discord\_voice\_bot\_token](#discord_voice_bot_token) | | `discord_*_webhook` | `app_settings` (encrypted) | Per-channel webhooks | [§ discord webhooks](#discord-webhooks) | | `lemon_squeezy_webhook_secret` | `app_settings` (encrypted) | Lemon Squeezy webhook HMAC | [§ lemon\_squeezy\_webhook\_secret](#lemon_squeezy_webhook_secret) | | `resend_api_key` | `app_settings` (encrypted) | Newsletter email send | [§ resend\_api\_key](#resend_api_key) | | `resend_webhook_secret` | `app_settings` (encrypted) | Resend webhook HMAC | [§ resend\_webhook\_secret](#resend_webhook_secret) | | `smtp_password` | `app_settings` (NOT yet encrypted as of 2026-04-30) | SMTP fallback | [§ smtp\_password](#smtp_password) | | `uptime_kuma_api_key` | `app_settings` (encrypted) | Uptime Kuma push | [§ uptime\_kuma\_api\_key](#uptime_kuma_api_key) | | `openai_api_key` | `app_settings` (encrypted) | OpenAI fallback | [§ openai\_api\_key](#openai_api_key) | | `anthropic_api_key` | `app_settings` (encrypted) | Claude Haiku critic | [§ anthropic\_api\_key](#anthropic_api_key) | | `gemini_api_key` | `app_settings` (encrypted) | Gemini fallback | [§ gemini\_api\_key](#gemini_api_key) | | `pexels_api_key` | `app_settings` (encrypted) | Stock image fallback | [§ pexels\_api\_key](#pexels_api_key) | | `cloudinary_api_key` + `cloudinary_api_secret` | `app_settings` (encrypted) | Image CDN | [§ cloudinary keys](#cloudinary-keys) | | `storage_secret_key` + `storage_token` | `app_settings` (encrypted) | S3-compatible object storage (R2) | [§ storage keys](#storage-keys) | | `redis_url` | `app_settings` (encrypted) | Redis connection (with AUTH password) | [§ redis\_url](#redis_url) | | `devto_api_key` | `app_settings` (encrypted) | Dev.to cross-post | [§ devto\_api\_key](#devto_api_key) | | `notion_api_key` | `app_settings` (encrypted) | Notion integration | [§ notion\_api\_key](#notion_api_key) | | `grafana_api_key` + `grafana_api_token` | `app_settings` (encrypted) | Self-hosted Grafana HTTP API | [§ grafana keys](#grafana-keys) | | `google_api_key` | `app_settings` (encrypted) | Google APIs | [§ google\_api\_key](#google_api_key) | | `elevenlabs_api_key` | `app_settings` (encrypted) | TTS provider | [§ elevenlabs\_api\_key](#elevenlabs_api_key) | *** ## Generic procedure (covers \~90% of the keys) Most rotations follow this exact recipe: ```bash theme={null} # 1. Generate at the provider — copy the new value into a secure clipboard # (or generate locally for self-issued tokens) # 2. Save to a 0600 file so we don't echo it NEWVAL_FILE=$(mktemp) chmod 600 "$NEWVAL_FILE" # Paste the new value into $NEWVAL_FILE; save and exit. # (or: echo -n "" > "$NEWVAL_FILE" -- but this exposes it in shell history) # 3. Set the new value via plugins.secrets (encrypts under current key) docker exec -i poindexter-worker python -c " import asyncio, asyncpg, os, sys sys.path.insert(0, '/app/src/cofounder_agent') from plugins.secrets import set_secret async def main(): val = open('/tmp/newval').read().strip() conn = await asyncpg.connect(os.environ['DATABASE_URL']) await set_secret(conn, '', val) await conn.close() asyncio.run(main()) " "" --secret if you accept the value being on the cmdline. # Prefer the helper above for true secrets. # 4. Restart any consumer that caches at startup docker restart poindexter-worker # most secrets — async-fetched per request # (some specific ones need additional restarts — see the per-key sections) # 5. Verify with a real downstream call (see per-key sections) # 6. Revoke the old value at the provider ``` **Why not always `poindexter settings set --secret`?** It works, but it puts the secret value in your shell history and the click argv. `set_secret(...)` via the helper above keeps the secret in a 0600 file that you delete after. *** ## Per-key rotation procedures ### `database_url` **Lives in.** `~/.poindexter/bootstrap.toml`, key `database_url`. **When to rotate.** Postgres password compromised, or you're moving the DB to a new host. **Procedure.** ```bash theme={null} # 1. Change the postgres password docker exec -it poindexter-postgres-local psql -U poindexter -d postgres # In psql: ALTER USER poindexter WITH PASSWORD ''; \q # 2. Edit ~/.poindexter/bootstrap.toml — update the connection string chmod 600 ~/.poindexter/bootstrap.toml # 3. Restart everything that connects to the DB docker restart poindexter-worker poindexter-brain-daemon poindexter-grafana ``` **Verify.** ```bash theme={null} curl -s http://localhost:8002/api/health | python -m json.tool # database.connected should be true ``` *** ### `POINDEXTER_SECRET_KEY` **Lives in.** `~/.poindexter/bootstrap.toml` (env var name `POINDEXTER_SECRET_KEY`). **When to rotate.** Annually, or after a suspected compromise. **THIS IS THE DOOMSDAY KEY.** Losing it without rotating means every encrypted `app_settings` row becomes garbage forever (see [`disaster-recovery.md`](./disaster-recovery) CONFIG-2). **Procedure.** Use the `rotate_key` helper from `plugins.secrets` — it decrypts every secret with the OLD key and re-encrypts with the NEW one in a single transaction. ```bash theme={null} # 1. Generate a new key (save it RIGHT NOW to 1Password) NEW_KEY=$(python -c "import secrets; print(secrets.token_urlsafe(48))") echo "$NEW_KEY" # Paste this into 1Password. DO NOT lose it. # 2. Capture the current key OLD_KEY="$POINDEXTER_SECRET_KEY" # 3. Run the rotation (re-encrypts every is_secret=true row) docker exec -i poindexter-worker python -c " import asyncio, asyncpg, os, sys sys.path.insert(0, '/app/src/cofounder_agent') from plugins.secrets import rotate_key async def main(): conn = await asyncpg.connect(os.environ['DATABASE_URL']) n = await rotate_key(conn, old_key='$OLD_KEY', new_key='$NEW_KEY') print(f'Rotated {n} secret(s)') await conn.close() asyncio.run(main()) " # Expected: "Rotated secret(s)" (matches your is_secret=true count) # 4. Update bootstrap.toml with the new key # Edit POINDEXTER_SECRET_KEY = "" in ~/.poindexter/bootstrap.toml # 5. Restart everything that decrypts secrets docker restart poindexter-worker poindexter-brain-daemon ``` **Verify.** ```bash theme={null} # Trigger a decrypt round-trip curl -s http://localhost:8002/api/health docker logs poindexter-worker 2>&1 | grep -i "SecretsError" # Expected: no SecretsError lines after the restart. ``` **Revoke.** N/A — the old key isn't held anywhere external. Just remove it from your password manager once you've confirmed the rotation took. *** ### OAuth client secrets **Lives in.** `app_settings._oauth_client_id` / `app_settings._oauth_client_secret` (encrypted, `is_secret=true`). One pair per consumer (`cli`, `mcp`, `mcp_gladlabs`, `brain`, `scripts`, `openclaw`). **Used for.** OAuth 2.1 Client Credentials Grant — every consumer mints short-lived JWTs by hitting `POST /token` with its client\_id + client\_secret. The legacy static-Bearer plumbing (`POINDEXTER_KEY` / `GLADLABS_KEY` / `app_settings.api_token` / `app_settings.api_auth_token`) was removed in Glad-Labs/poindexter#249 (2026-05-05). **Procedure.** The migrate commands rotate the client\_secret atomically and push the new value into the consumer's config file (`~/.claude.json`, `~/.openclaw/openclaw.json`, `~/.poindexter/bootstrap.toml`, etc.). ```bash theme={null} # Rotate one consumer poindexter auth migrate-cli # CLI poindexter auth migrate-mcp # public MCP server poindexter auth migrate-mcp-gladlabs # operator MCP server poindexter auth migrate-brain # brain daemon poindexter auth migrate-scripts # ad-hoc scripts client poindexter auth migrate-openclaw # OpenClaw bridge # Or rotate the Grafana token (different shape — used for the brain's # Grafana alert sync loop) poindexter auth mint-grafana-token # Restart consumers so they re-read their configs docker restart poindexter-worker poindexter-brain-daemon # Restart Claude Desktop / OpenClaw gateway so they re-read their JSON configs. ``` **Verify.** ```bash theme={null} # Mint a fresh JWT and hit the worker JWT=$(poindexter auth mint-token --client cli) curl -sH "Authorization: Bearer $JWT" http://localhost:8002/api/health \ -o /dev/null -w "%{http_code}\n" # Expected: 200 # An old (pre-rotation) JWT or an unprovisioned client should 401 ``` **Reference.** Consumer wiring lives in `services/auth/oauth_client.py` and `poindexter/cli/auth.py`. The umbrella OAuth migration is umbrella issue Glad-Labs/poindexter#241. *** ### `revalidate_secret` **Lives in.** `app_settings.revalidate_secret` (encrypted). **Used for.** Worker → Vercel ISR revalidation. Sent as `x-revalidate-secret` header. The Vercel function compares against an env var on the Vercel side. **Procedure.** ```bash theme={null} # 1. Generate NEW=$(python -c "import secrets; print(secrets.token_urlsafe(32))") # 2. Set in DB poindexter settings set revalidate_secret "$NEW" --secret # OR use set_secret helper # 3. Update on Vercel — this MUST happen within seconds of step 2, # otherwise revalidation 401s until you do. vercel env rm REVALIDATE_SECRET production echo "$NEW" | vercel env add REVALIDATE_SECRET production # 4. Trigger a Vercel redeploy so the new env var lands in the runtime vercel --prod # 5. Restart worker so it picks up the new value docker restart poindexter-worker ``` **Verify.** ```bash theme={null} # Trigger a revalidation against a real post slug curl -s -X POST http://localhost:8002/api/revalidate \ -H "Authorization: Bearer $(poindexter auth mint-token --client cli)" \ -d '{"path": "/posts/"}' # Expected: {"revalidated": true} # Check Vercel logs for the revalidation hit vercel logs --follow | grep revalidate ``` *** ### `jwt_secret_key` **Lives in.** `app_settings.jwt_secret_key` (encrypted). **When to rotate.** Suspected compromise, or after major code change to JWT issuance. **Rotation invalidates every existing JWT** — anyone holding one will need to re-auth. **Procedure.** Generic procedure. Restart `poindexter-worker` so the new key is picked up before issuing the next JWT. *** ### `secret_key` **Lives in.** `app_settings.secret_key` (encrypted). **Used for.** General-purpose app signing (CSRF tokens, signed cookies). **TBD — needs operator confirmation:** the exact callsites for this key aren't fully audited; treat as `jwt_secret_key`-equivalent. *** ### `openclaw_webhook_token` **Lives in.** `app_settings.openclaw_webhook_token` (encrypted). **Used for.** Worker → OpenClaw gateway auth (`Authorization` header). **Procedure.** ```bash theme={null} # 1. Generate NEW=$(python -c "import secrets; print(secrets.token_urlsafe(32))") # 2. Set in DB poindexter settings set openclaw_webhook_token "$NEW" --secret # OR use set_secret helper # 3. Update OpenClaw config # Edit ~/.openclaw/openclaw.json — set the gateway's expected token. # 4. Restart both docker restart poindexter-worker # Restart OpenClaw (Windows shortcut or:) start "OpenClaw Gateway" cmd /d /c "%USERPROFILE%\.openclaw\gateway.cmd" ``` *** ### `alertmanager_webhook_token` **Lives in.** `app_settings.alertmanager_webhook_token` (encrypted). **Used for.** Alertmanager → worker webhook auth. **Procedure.** Generic. After updating the DB, also update `infrastructure/prometheus/alertmanager.yml.tmpl` (the matching value lives there; the rendered `alertmanager.yml` is regenerated by `render_alertmanager_config.py`) and restart Alertmanager: ```bash theme={null} docker restart poindexter-alertmanager ``` *** ### `telegram_bot_token` **Lives in.** `app_settings.telegram_bot_token` (encrypted) AND optionally `bootstrap.toml` (operator notification fallback). **When to rotate.** Suspected leak (the token is in the URL path of every Telegram API call — it's surprisingly leaky). Or @BotFather forced rotation. **Procedure.** ```bash theme={null} # 1. Open Telegram, message @BotFather: # /mybots -> select your bot -> API Token -> Revoke current token # BotFather hands you the new token. Save it. # 2. Set in DB poindexter settings set telegram_bot_token "" --secret # 3. ALSO update bootstrap.toml fallback (for operator notifications when DB is down) # Edit ~/.poindexter/bootstrap.toml -> telegram_bot_token = "" chmod 600 ~/.poindexter/bootstrap.toml # 4. Restart consumers docker restart poindexter-worker poindexter-brain-daemon # Telegram bot daemon (if running standalone): # Windows: pythonw scripts/telegram-bot.py ``` **Verify.** ```bash theme={null} # Trigger a test alert to your chat curl -s -X POST http://localhost:8002/api/test/notify-operator \ -H "Authorization: Bearer $(poindexter auth mint-token --client cli)" # Expected: a Telegram message arrives within seconds. ``` *** ### `discord_bot_token` **Lives in.** `app_settings.discord_bot_token` (encrypted). **Procedure.** ```bash theme={null} # 1. Open https://discord.com/developers/applications # -> select app -> Bot -> Reset Token (this revokes the old one immediately) # Copy the new token. # 2. Set in DB poindexter settings set discord_bot_token "" --secret # 3. Restart Discord bot consumer docker restart poindexter-worker # If voice bot is separate: docker restart poindexter-voice-bot ``` *** ### `discord_voice_bot_token` **Lives in.** `app_settings.discord_voice_bot_token` (encrypted). **Procedure.** Same as `discord_bot_token` but for the voice-bot Discord application. Restart `poindexter-voice-bot`. *** ### Discord webhooks Keys: `discord_ops_webhook_url`, `discord_alerts_webhook_url`, etc. (per-channel webhook URLs that include a secret token in the path). **Procedure.** ```bash theme={null} # 1. In Discord: Server Settings -> Integrations -> Webhooks # -> select webhook -> Copy Webhook URL (regenerates if needed) # 2. Set in DB poindexter settings set discord_ops_webhook_url "https://discord.com/api/webhooks/.../...token" --secret # 3. ALSO update bootstrap.toml fallback if this is the ops webhook # Edit ~/.poindexter/bootstrap.toml -> discord_ops_webhook_url # 4. Restart consumers docker restart poindexter-worker poindexter-brain-daemon ``` *** ### `lemon_squeezy_webhook_secret` **Lives in.** `app_settings.lemon_squeezy_webhook_secret` (encrypted as of 2026-04-27). **Used for.** HMAC verification of Lemon Squeezy webhooks (subscription created / cancelled / order paid). **Procedure.** ```bash theme={null} # 1. In Lemon Squeezy dashboard: Settings -> Webhooks -> select your webhook # -> "Reveal Signing Secret" / "Regenerate" # Copy the new value. # 2. Set in DB poindexter settings set lemon_squeezy_webhook_secret "" --secret # 3. Restart worker so the verifier reads the new value on next request # (Actually, since the verifier is async and reads per-request, this # might not strictly require a restart — but do it anyway for safety.) docker restart poindexter-worker ``` **Verify.** Trigger a test webhook from Lemon Squeezy dashboard. Worker logs should show a 200 response, not 401. *** ### `resend_api_key` **Lives in.** `app_settings.resend_api_key` (encrypted). **Used for.** Newsletter emails via Resend. **Procedure.** ```bash theme={null} # 1. https://resend.com/api-keys -> Delete old key, Create new key # 2. poindexter settings set resend_api_key "" --secret # 3. docker restart poindexter-worker ``` **Verify.** Send a test newsletter to one address — see `poindexter newsletter --help`. *** ### `resend_webhook_secret` **Lives in.** `app_settings.resend_webhook_secret` (encrypted as of 2026-04-27). **Used for.** HMAC verification of Resend delivery webhooks. **Procedure.** Same shape as Lemon Squeezy. New secret comes from Resend dashboard → Webhooks → endpoint → Signing Secret. *** ### `smtp_password` **Lives in.** `app_settings.smtp_password`. Read via `await site_config.get_secret(...)` (the newsletter `_cfg` callsite), so set it with `--secret` to store it encrypted at rest. **Procedure.** ```bash theme={null} # 1. Generate / obtain new SMTP password from your email host # 2. poindexter settings set smtp_password "" --secret # encrypts at rest (pgcrypto) # 3. docker restart poindexter-worker ``` *** ### `uptime_kuma_api_key` **Lives in.** `app_settings.uptime_kuma_api_key` (encrypted; seeded in the baseline). **Used for.** Pushing health beacons to Uptime Kuma. **Procedure.** ```bash theme={null} # 1. In Uptime Kuma: Settings -> API Keys -> Create / Regenerate # Copy the new key. # 2. poindexter settings set uptime_kuma_api_key "" --secret # 3. docker restart poindexter-worker poindexter-brain-daemon ``` **Verify.** Watch Uptime Kuma — heartbeats should resume within 60 seconds. *** ### `openai_api_key` **Lives in.** `app_settings.openai_api_key` (encrypted). **Used for.** OpenAI fallback in the model fallback chain (only used when explicitly enabled in `app_settings` AND gated by `cost_guard`). **Procedure.** ```bash theme={null} # 1. https://platform.openai.com/api-keys -> Revoke old, Create new # 2. poindexter settings set openai_api_key "sk-..." --secret # 3. docker restart poindexter-worker ``` **Cost guardrails.** Confirm `cost_guard_enabled=true` and `openai_enabled=true` only if you intend to actually use it. Per Matt's policy, local Ollama is the default — paid providers are opt-in fallbacks ONLY. *** ### `anthropic_api_key` **Lives in.** `app_settings.anthropic_api_key` (encrypted). **Used for.** Claude Haiku adversarial QA review — the multi-model QA rail atoms (`qa.critic` / `qa.deepeval`) that replaced the retired `cross_model_qa` stage (#355). **Procedure.** ```bash theme={null} # 1. https://console.anthropic.com/settings/keys -> Revoke old, Create new # 2. poindexter settings set anthropic_api_key "sk-ant-..." --secret # 3. docker restart poindexter-worker ``` **Verify.** Run a content task end-to-end; check that a QA rail atom logs a Claude Haiku call rather than skipping. *** ### `gemini_api_key` **Lives in.** `app_settings.gemini_api_key` (encrypted). **Used for.** Gemini fallback. **High risk** — Matt previously spent \$300 in one night via Gemini before cost guards. Keep `cost_guard_enabled=true` and `gemini_enabled=false` unless you actively need it. **Procedure.** Generic. Same as OpenAI. *** ### `pexels_api_key` **Lives in.** `app_settings.pexels_api_key` (encrypted). **Used for.** Stock image fallback when image-gen is degraded. **Procedure.** ```bash theme={null} # 1. https://www.pexels.com/api/new -> regenerate / new key # 2. poindexter settings set pexels_api_key "" --secret # 3. docker restart poindexter-worker ``` *** ### Cloudinary keys Keys: `cloudinary_api_key` + `cloudinary_api_secret` (both encrypted). **Procedure.** ```bash theme={null} # 1. https://console.cloudinary.com/settings/api-keys -> regenerate # 2. poindexter settings set cloudinary_api_key "" --secret # poindexter settings set cloudinary_api_secret "" --secret # 3. docker restart poindexter-worker ``` **Verify.** Trigger a Cloudinary-backed image upload and watch worker logs for Cloudinary 200 responses. *** ### Storage keys Keys: `storage_secret_key` + `storage_token` (both encrypted). Provider-agnostic — works with R2, S3, B2, MinIO. Old `cloudflare_r2_*` keys are deprecated fallbacks. **Procedure.** ```bash theme={null} # 1. At your storage provider (R2/S3/etc.) -> create new access key, deactivate old # 2. poindexter settings set storage_secret_key "" --secret # poindexter settings set storage_token "" --secret # 3. docker restart poindexter-worker ``` **Verify.** Trigger a static export rebuild — watch for object writes to land in the bucket. *** ### `redis_url` **Lives in.** `app_settings.redis_url` (encrypted — the URL contains the AUTH password). **Procedure.** ```bash theme={null} # 1. Update Redis password # Edit redis.conf or run: docker exec poindexter-prefect-redis redis-cli CONFIG SET requirepass "" # 2. Build new URL: redis://default:@: # 3. poindexter settings set redis_url "redis://default:@..." --secret # 4. docker restart poindexter-worker ``` **Verify.** `RedisCache` initialization in worker logs should not show "degraded to no-cache mode." *** ### `devto_api_key` **Procedure.** [https://dev.to/settings/extensions](https://dev.to/settings/extensions) → API Keys → Generate new. Generic flow. *** ### `notion_api_key` **Procedure.** [https://www.notion.so/my-integrations](https://www.notion.so/my-integrations) → select integration → Refresh / Reset. Generic flow. *** ### Grafana keys Keys: `grafana_api_key` + `grafana_api_token` (both encrypted; some duplication for legacy reasons). **Procedure.** ```bash theme={null} # 1. http://localhost:3000 (or http://:3000 via tailnet) -> # Administration -> Service accounts -> Add service account -> # "alert-sync" with Editor role -> Add token # 2. poindexter settings set grafana_api_key "" --secret # poindexter settings set grafana_api_token "" --secret # if both rows exist # 3. docker restart poindexter-worker ``` *** ### `google_api_key` **Procedure.** [https://console.cloud.google.com](https://console.cloud.google.com) → APIs & Services → Credentials → Regenerate. Generic flow. Note: this single key may grant access to multiple Google APIs depending on which APIs are enabled — review the API restrictions before rotating. *** ### `elevenlabs_api_key` **Procedure.** [https://elevenlabs.io](https://elevenlabs.io) → Profile → API Keys → Generate / Revoke. Generic flow. *** ## Re-seeding from scratch (after losing `POINDEXTER_SECRET_KEY`) After a CONFIG-2 disaster recovery (lost encryption key), you have an empty `app_settings` for every secret. Walk through this checklist: ```text theme={null} [ ] OAuth client secrets (#critical — re-run `poindexter auth migrate-*` for every consumer) [ ] revalidate_secret (#critical — Vercel ISR) [ ] telegram_bot_token (#critical — alerts) [ ] discord_bot_token (if used) [ ] discord_*_webhook (per channel) [ ] openclaw_webhook_token (if OpenClaw is wired) [ ] alertmanager_webhook_token (if Alertmanager is wired) [ ] lemon_squeezy_webhook_secret (if accepting payments) [ ] resend_api_key + resend_webhook_secret (if sending newsletters) [ ] uptime_kuma_api_key (if pushing to Kuma) [ ] openai_api_key (only if you actively use it) [ ] anthropic_api_key (cross-model QA) [ ] gemini_api_key (only if you actively use it) [ ] pexels_api_key (image fallback) [ ] cloudinary_api_key + secret (image CDN) [ ] storage_secret_key + token (R2/S3 — required for static export) [ ] redis_url (if Redis is wired) [ ] devto (cross-post adapter) [ ] grafana_api_key + token (if calling Grafana API from worker) ``` For each one: go to the section above, generate a new value at the provider, set it in DB, restart the consumer. *** ## Verification — was the rotation actually applied? ```bash theme={null} # Confirm the row is encrypted (starts with enc:v1:) docker exec poindexter-postgres-local psql -U poindexter -d poindexter_brain -c \ "SELECT key, value LIKE 'enc:v1:%%' AS encrypted, length(value) AS len, updated_at FROM app_settings WHERE key = '';" # Confirm the worker can decrypt it (no SecretsError in logs) docker logs poindexter-worker --since=2m 2>&1 | grep -i "SecretsError\|" # Trigger a downstream call that uses the secret. Look for HTTP 2xx, not 401. ``` *** ## Rotation cadence — calendar **TBD — needs operator to confirm cadence preferences:** * `POINDEXTER_SECRET_KEY` — annually (low risk if undisturbed; high blast radius if leaked) * OAuth client secrets, `revalidate_secret`, `openclaw_webhook_token` — every 90 days * Bot tokens (Telegram, Discord) — only on suspected compromise (rotation invalidates active sessions) * Paid API keys (OpenAI, Anthropic, Gemini, Resend, Cloudinary) — every 90 days, or on bill anomaly * Webhook secrets (Lemon Squeezy, Resend) — only when the provider rotates them or on suspected compromise * Cross-post adapter tokens (Dev.to) — yearly A scheduled agent should be set up to remind on this cadence (see `/schedule` skill). *** ## See also * [`disaster-recovery.md`](./disaster-recovery) — recovery from lost key (CONFIG-2) * [`incident-response.md`](./incident-response) — alert routing * `src/cofounder_agent/plugins/secrets.py` — encryption module reference * `src/cofounder_agent/poindexter/cli/auth.py` — implementation of `poindexter auth migrate-*`