Skip to main content

Shared Responsibility Model

Running an app on Clue2App is a partnership. Some things you do, some things the platform does on your behalf — automatically via the agent layer, and some things the platform does manually via a human operator (us). This page is the explicit boundary.

If a failure happens in the middle of the night and you can't tell whose problem it is, look here.

The three lanes​

You (developer / app owner)Platform (automated) — agents heal + detect 24/7Platform (manual) — operator action when escalated
App codeWrite it. Push to git.n/an/a
App secretsCreate via c2a daari secrets create. Rotate via UI / CLI when you decide.Detects when secrets are missing on a build (BuildRepairAgent Pattern A/B).Provides AWS Secrets Manager backing on request (Phase 2).
Git credentialsProvide initial PAT. Authorize SAML SSO for the org.GitCredsHealthAgent scans every 6h, alerts on dead / expiring PATs.We rotate the platform-owned PAT pool quarterly.
Builds (kpack)Push code. c2a app rebuild.BuildRepairAgent (15min) classifies wedged builds + emits OTel alerts with the exact kubectl recipe. v2 (active mode) auto-applies the repair after a dry-run window.We unwedge by hand if you ping us (or wait for v2).
Service bindingsc2a daari secrets bind <app> <secret>. Coordinator merges into the C2A_SYSTEM_ENV the runtime reads.Agents validate the binding lifecycle didn't get clobbered by a re-deploy.n/a
Knative service healthCode reads C2A_SYSTEM_ENV correctly.SurfaceDriftAgent (15min) probes /health, /version, /tools on every ksvc, alerts on regressions.Operator-on-call investigates critical alerts when paged.
Certificates (TLS)n/a (cert-manager owns it).CertHealthAgent (6h) alerts on Ready=False or notAfter within 14d.We rotate when cert-manager renewal fails.
Namespace bootstrapCreate a project.ProjectBootstrapDriftAgent (6h) catches namespaces missing the system-default ServiceAccount kpack needs.Coordinator now auto-bootstraps new project namespaces (coordinator#146). Legacy namespaces healed on-detection.
AWS / GCP / Groq upstream healthn/aAwsIamHealthAgent, LlmUpstreamAgent, etc. probe every upstream we depend on. Alerts on key-rejected / rate-limited.We rotate the platform-owned credentials.
GitHub App migrationn/a (Phase 3, not active yet)Future: GhAppTokenRotator mints installation tokens, replaces per-user PATs.We register the App in the org and store the private key.
OTel pipelinen/aAgents emit alerts → otel-collector → coordinator → persisted in realtime_notifications → surfaced on /agents console panel + iOS Alerts tab.We tune the collector pipeline as alert volume grows.
Cost trackingView on /costs.CostTrackerAgent snapshots CCU usage hourly; computes monthly trend.We adjust the platform markup if needed (currently 200% over infra base).

What each lane means in practice​

Your lane (you do this)​

These are the things only you can do because they encode your intent:

  • Decide what your app is. Code, dependencies, environment variables, secrets values.
  • Decide who can touch it. Project membership, RBAC roles.
  • Decide when to rotate. A secret rotation is a decision about trust; the platform never auto-rotates your application credentials.
  • Decide when to bind. Which apps consume which Daari Secret / LLM Gateway instance / Postgres instance.

Platform (automated, "agentic")​

These are things the platform watches continuously. You shouldn't have to think about them in steady state. When something goes wrong, an OTel alert lands on:

  • The /agents panel in the web console
  • The Alerts tab in the iOS app
  • (Future) Slack when wired

Each alert carries a suggested_fix field that is the exact recipe to resolve. In active mode (rolling out over the next quarter, behind a per-agent feature flag), the agent applies the fix itself and emits a confirmation alert instead.

The agents shipped today:

AgentCadenceWhat it catchesRepair mode
cert-health6hCerts not Ready, expiring within 14dAlert only
git-creds-health6hDead / expiring PATs across all namespacesAlert only
buildpack-stack6hPaketo stack drift (would have caught libatomic regression)Alert only
github-ratelimit1hGitHub API rate limit < 100 remainingAlert only
postgres-health6hProject Postgres instances unreachable / slowAlert only
aws-iam-health1hSTS + ECR write checks; partial failures = policy driftAlert only
llm-upstream30mGroq / OpenAI / Anthropic reachability per gatewayAlert only
project-bootstrap-drift6hNamespaces missing the kpack system-default SAAlert only
build-repair15mWedged kpack Images that won't auto-retryDry-run / alert only — v2 active
surface-drift15m/health breaking, /version regressing, /tools count droppingAlert only

Platform (manual, "operator")​

These require human judgement or have a high enough blast radius that we run them ourselves:

  • AWS IAM credentials that the platform uses (the c2s-iam-admin user). Quarterly rotation, by us.
  • Cluster-wide control-plane patches (coordinator, core, console, agents redeploys).
  • The Paketo BP_NODE_VERSION=22.*.* pin that goes on every new kpack Image (k8s-core-service auto-applies; we'll remove the pin when Paketo ships fixed run images).
  • AWS Secrets Manager backing for high-stakes secrets when you tag a Daari Secret source=aws-secrets-manager (Phase 2 — not yet live).
  • GitHub App registration in the clue2solve org (Phase 3 of the per-user PAT migration).
  • OTel collector tuning as alert volume changes.

Recent capability inventory​

What we shipped between 2026-04 and 2026-06. Each capability is in one or more of the lanes above:

Platform reliability sweeps (agent layer)​

Pattern: paired core REST endpoint + Python agent. The agent runs on a cadence, calls the core endpoint, emits OTel alerts. See agentic-architecture.

  • cert-health + 7 other detection agents (see table above)
  • OTel pipeline end-to-end: agents → otel-collector → coordinator /api/notifications/ingest → JSONB realtime_notifications rows → /agents panel + iOS Alerts tab

Build experience​

  • c2a app rebuild verifies by default. After the API call, polls for ~30s to confirm kpack actually scheduled a new Build. When wedged, prints the exact kubectl delete build.kpack.io ... recipe instead of pretending success.
  • PreBuild Audit endpoint (POST /api/v1/k8s/code/audit) — pre-build static scan for committed secrets, hardcoded AWS keys, PEM private keys. Surfaced via c2a audit <git-url> and /audit console page.
  • Bootstrap drift detection. Every project namespace now auto-gets the system-default ServiceAccount kpack needs at project-create time (coordinator#146).

Daari Secrets (service catalog category)​

Project-scoped key/value secrets, bindable to apps via the existing service-binding lifecycle. At runtime, every bound value lands in a single C2A_SYSTEM_ENV env var whose JSON contains three lanes: variables (plain env), secrets (masked env), and binding (decoded service references keyed by display name). Apps parse it once at startup and read by service name.

  • Coordinator endpoints: GET /api/v1/daari/secrets/ (list + detail + bindings), POST create, PUT /values rotate, DELETE
  • CLI: c2a daari secrets list / show / create / rotate / delete / bindings / bind
  • Console: pinned Secrets tab inside /daari with deep links to /daari/secrets/<k8sName>
  • Values are NEVER echoed back from any surface — only keys + metadata + bindings
  • cert-manager-managed TLS secrets surface read-only with a "managed by cert-manager" badge
  • Quota: 50 per project (decision #5 of the locked design)

Build wedge repair​

  • BuildRepairAgent (15min) calls GET /api/v1/k8s/builds/wedged. Classifies failures into 4 patterns (missing-sa, git-auth-dead-pat, git-auth-other, transient-or-unknown) with copy-paste fixes that always end with the required kubectl delete build.kpack.io ... step (kpack won't auto-retry once latest Build is in Succeeded=False — confirmed during the expense-mcp incident 2026-06-08).

Surface drift​

  • SurfaceDriftAgent (15min) probes well-known endpoints (/health, /version, /tools) on every ksvc cluster-wide. Tracks last snapshot in memory; alerts on regressions:
    • /health was 200, now isn't (critical)
    • /version went backwards by semver (warning)
    • /tools count dropped by >20% (warning)
    • Endpoint was reachable, now unreachable (critical)
  • Honest limitation: doesn't catch always-wrong cases (baseline is what was there last scan). v2 will add per-app canonical-source manifests.

Where to look when something breaks​

SymptomFirst check
App not buildingc2a build list <app>, then c2a app rebuild <app> (verifies by default; prints unstick recipe when wedged)
Build fails on git authc2a creds check -A --only-dead
c2a heal says 500c2a heal endpoint was never implemented; use specific verbs (c2a app rebuild, c2a creds rotate) instead
Knative pod won't start/agents panel — surface drift or build-repair alerts
Daari Secret rotate didn't take effect on appRestart the ksvc (auto-restart on rotate is decision #3 of the spec, ships in v2)
You hit a cap (project limit, secrets limit)The error message names the limit; if it's blocking you, ping us

Why this matters​

Without this boundary, every failure ends up in the same Slack channel labelled "broken." Drawing the line means:

  • You spend zero time investigating cluster-internal issues.
  • We spend zero time guessing whether a failure is your config or our infra.
  • The agent layer takes over the boring repeatable repairs so you (and we) stay on the hard problems.

If you find a failure mode where this table is wrong or silent, tell us — that's the highest-leverage feedback you can give us.