Shared Responsibility Model
Running an app on Clue2App is a partnership. Some things you do, some things the platform does on your behalf — automatically via the agent layer, and some things the platform does manually via a human operator (us). This page is the explicit boundary.
If a failure happens in the middle of the night and you can't tell whose problem it is, look here.
The three lanes
| You (developer / app owner) | Platform (automated) — agents heal + detect 24/7 | Platform (manual) — operator action when escalated | |
|---|---|---|---|
| App code | Write it. Push to git. | n/a | n/a |
| App secrets | Create via c2a daari secrets create. Rotate via UI / CLI when you decide. | Detects when secrets are missing on a build (BuildRepairAgent Pattern A/B). | Provides AWS Secrets Manager backing on request (Phase 2). |
| Git credentials | Provide initial PAT. Authorize SAML SSO for the org. | GitCredsHealthAgent scans every 6h, alerts on dead / expiring PATs. | We rotate the platform-owned PAT pool quarterly. |
| Builds (kpack) | Push code. c2a app rebuild. | BuildRepairAgent (15min) classifies wedged builds + emits OTel alerts with the exact kubectl recipe. v2 (active mode) auto-applies the repair after a dry-run window. | We unwedge by hand if you ping us (or wait for v2). |
| Service bindings | c2a daari secrets bind <app> <secret>. Coordinator merges into the C2A_SYSTEM_ENV the runtime reads. | Agents validate the binding lifecycle didn't get clobbered by a re-deploy. | n/a |
| Knative service health | Code reads C2A_SYSTEM_ENV correctly. | SurfaceDriftAgent (15min) probes /health, /version, /tools on every ksvc, alerts on regressions. | Operator-on-call investigates critical alerts when paged. |
| Certificates (TLS) | n/a (cert-manager owns it). | CertHealthAgent (6h) alerts on Ready=False or notAfter within 14d. | We rotate when cert-manager renewal fails. |
| Namespace bootstrap | Create a project. | ProjectBootstrapDriftAgent (6h) catches namespaces missing the system-default ServiceAccount kpack needs. | Coordinator now auto-bootstraps new project namespaces (coordinator#146). Legacy namespaces healed on-detection. |
| AWS / GCP / Groq upstream health | n/a | AwsIamHealthAgent, LlmUpstreamAgent, etc. probe every upstream we depend on. Alerts on key-rejected / rate-limited. | We rotate the platform-owned credentials. |
| GitHub App migration | n/a (Phase 3, not active yet) | Future: GhAppTokenRotator mints installation tokens, replaces per-user PATs. | We register the App in the org and store the private key. |
| OTel pipeline | n/a | Agents emit alerts → otel-collector → coordinator → persisted in realtime_notifications → surfaced on /agents console panel + iOS Alerts tab. | We tune the collector pipeline as alert volume grows. |
| Cost tracking | View on /costs. | CostTrackerAgent snapshots CCU usage hourly; computes monthly trend. | We adjust the platform markup if needed (currently 200% over infra base). |
What each lane means in practice
Your lane (you do this)
These are the things only you can do because they encode your intent:
- Decide what your app is. Code, dependencies, environment variables, secrets values.
- Decide who can touch it. Project membership, RBAC roles.
- Decide when to rotate. A secret rotation is a decision about trust; the platform never auto-rotates your application credentials.
- Decide when to bind. Which apps consume which Daari Secret / LLM Gateway instance / Postgres instance.
Platform (automated, "agentic")
These are things the platform watches continuously. You shouldn't have to think about them in steady state. When something goes wrong, an OTel alert lands on:
- The
/agentspanel in the web console - The Alerts tab in the iOS app
- (Future) Slack when wired
Each alert carries a suggested_fix field that is the exact recipe to resolve. In active mode (rolling out over the next quarter, behind a per-agent feature flag), the agent applies the fix itself and emits a confirmation alert instead.
The agents shipped today:
| Agent | Cadence | What it catches | Repair mode |
|---|---|---|---|
cert-health | 6h | Certs not Ready, expiring within 14d | Alert only |
git-creds-health | 6h | Dead / expiring PATs across all namespaces | Alert only |
buildpack-stack | 6h | Paketo stack drift (would have caught libatomic regression) | Alert only |
github-ratelimit | 1h | GitHub API rate limit < 100 remaining | Alert only |
postgres-health | 6h | Project Postgres instances unreachable / slow | Alert only |
aws-iam-health | 1h | STS + ECR write checks; partial failures = policy drift | Alert only |
llm-upstream | 30m | Groq / OpenAI / Anthropic reachability per gateway | Alert only |
project-bootstrap-drift | 6h | Namespaces missing the kpack system-default SA | Alert only |
build-repair | 15m | Wedged kpack Images that won't auto-retry | Dry-run / alert only — v2 active |
surface-drift | 15m | /health breaking, /version regressing, /tools count dropping | Alert only |
Platform (manual, "operator")
These require human judgement or have a high enough blast radius that we run them ourselves:
- AWS IAM credentials that the platform uses (the
c2s-iam-adminuser). Quarterly rotation, by us. - Cluster-wide control-plane patches (coordinator, core, console, agents redeploys).
- The Paketo
BP_NODE_VERSION=22.*.*pin that goes on every new kpack Image (k8s-core-service auto-applies; we'll remove the pin when Paketo ships fixed run images). - AWS Secrets Manager backing for high-stakes secrets when you tag a Daari Secret
source=aws-secrets-manager(Phase 2 — not yet live). - GitHub App registration in the
clue2solveorg (Phase 3 of the per-user PAT migration). - OTel collector tuning as alert volume changes.
Recent capability inventory
What we shipped between 2026-04 and 2026-06. Each capability is in one or more of the lanes above:
Platform reliability sweeps (agent layer)
Pattern: paired core REST endpoint + Python agent. The agent runs on a cadence, calls the core endpoint, emits OTel alerts. See agentic-architecture.
cert-health+ 7 other detection agents (see table above)OTel pipelineend-to-end: agents → otel-collector → coordinator/api/notifications/ingest→ JSONBrealtime_notificationsrows →/agentspanel + iOS Alerts tab
Build experience
c2a app rebuildverifies by default. After the API call, polls for ~30s to confirm kpack actually scheduled a new Build. When wedged, prints the exactkubectl delete build.kpack.io ...recipe instead of pretending success.- PreBuild Audit endpoint (
POST /api/v1/k8s/code/audit) — pre-build static scan for committed secrets, hardcoded AWS keys, PEM private keys. Surfaced viac2a audit <git-url>and/auditconsole page. - Bootstrap drift detection. Every project namespace now auto-gets the
system-defaultServiceAccount kpack needs at project-create time (coordinator#146).
Daari Secrets (service catalog category)
Project-scoped key/value secrets, bindable to apps via the existing service-binding lifecycle. At runtime, every bound value lands in a single C2A_SYSTEM_ENV env var whose JSON contains three lanes: variables (plain env), secrets (masked env), and binding (decoded service references keyed by display name). Apps parse it once at startup and read by service name.
- Coordinator endpoints:
GET /api/v1/daari/secrets/(list + detail + bindings),POSTcreate,PUT /valuesrotate,DELETE - CLI:
c2a daari secrets list / show / create / rotate / delete / bindings / bind - Console: pinned Secrets tab inside
/daariwith deep links to/daari/secrets/<k8sName> - Values are NEVER echoed back from any surface — only keys + metadata + bindings
- cert-manager-managed TLS secrets surface read-only with a "managed by cert-manager" badge
- Quota: 50 per project (decision #5 of the locked design)
Build wedge repair
BuildRepairAgent(15min) callsGET /api/v1/k8s/builds/wedged. Classifies failures into 4 patterns (missing-sa,git-auth-dead-pat,git-auth-other,transient-or-unknown) with copy-paste fixes that always end with the requiredkubectl delete build.kpack.io ...step (kpack won't auto-retry once latest Build is inSucceeded=False— confirmed during the expense-mcp incident 2026-06-08).
Surface drift
SurfaceDriftAgent(15min) probes well-known endpoints (/health,/version,/tools) on every ksvc cluster-wide. Tracks last snapshot in memory; alerts on regressions:/healthwas 200, now isn't (critical)/versionwent backwards by semver (warning)/toolscount dropped by >20% (warning)- Endpoint was reachable, now unreachable (critical)
- Honest limitation: doesn't catch always-wrong cases (baseline is what was there last scan). v2 will add per-app canonical-source manifests.
Where to look when something breaks
| Symptom | First check |
|---|---|
| App not building | c2a build list <app>, then c2a app rebuild <app> (verifies by default; prints unstick recipe when wedged) |
| Build fails on git auth | c2a creds check -A --only-dead |
c2a heal says 500 | c2a heal endpoint was never implemented; use specific verbs (c2a app rebuild, c2a creds rotate) instead |
| Knative pod won't start | /agents panel — surface drift or build-repair alerts |
| Daari Secret rotate didn't take effect on app | Restart the ksvc (auto-restart on rotate is decision #3 of the spec, ships in v2) |
| You hit a cap (project limit, secrets limit) | The error message names the limit; if it's blocking you, ping us |
Why this matters
Without this boundary, every failure ends up in the same Slack channel labelled "broken." Drawing the line means:
- You spend zero time investigating cluster-internal issues.
- We spend zero time guessing whether a failure is your config or our infra.
- The agent layer takes over the boring repeatable repairs so you (and we) stay on the hard problems.
If you find a failure mode where this table is wrong or silent, tell us — that's the highest-leverage feedback you can give us.