# Kevo flagging, review & knowledge correction

How the Kevo (ElevenLabs "Live Help") flagging loop works and how to set it up.
Design + verified API details: [PLAN.md](./PLAN.md), [API_NOTES.md](./API_NOTES.md).

## What it does

1. **Flags come in from three sources**, all landing in `kevo_flags`:
   - **In-call**: the agent calls its `flag_issue` server tool →
     `POST /v1/kevo/tools/flag-issue` (shared-secret header, rate-limited).
   - **Post-call**: the ElevenLabs post-call webhook →
     `POST /v1/kevo/webhooks/post-call` (HMAC-verified, 200-fast, processed by
     the `kevo-postcall` BullMQ job which reads the conversation's analysis).
   - **Manual**: a reviewer flags a conversation turn in the Console.
2. **Review in the Console** (`/admin/kevo`): knowledge flags get a verified
   correction written and approved; data-change/escalation flags become
   ClickUp tasks.
3. **Publish**: approved entries render into ONE managed KB document
   ("Kevo – Verified Corrections & Amenity Map"), content-updated in place
   (re-indexes automatically). Human-in-the-loop only — nothing reaches the
   agent without an approve + publish click.

## Environment variables (apps/core/.env)

| Var | What |
|---|---|
| `ELEVENLABS_API_KEY` | Workspace API key (server-side only) |
| `ELEVENLABS_AGENT_ID` | The Kevo agent id |
| `ELEVENLABS_WEBHOOK_SECRET` | HMAC secret from the post-call webhook you create (below) |
| `KEVO_TOOL_SHARED_SECRET` | Any strong random string; sent by ElevenLabs as `X-Kevo-Secret` |
| `KEVO_PUBLIC_BASE_URL` | Public base URL of core, e.g. `https://api.example.com` |
| `CLICKUP_API_TOKEN` / `CLICKUP_LIST_ID` | ClickUp task routing |
| `CONSOLE_BASE_URL` | Console URL used in ClickUp task deep links (default `http://localhost:3002`) |
| `ELEVENLABS_BASE_URL` | Optional; defaults to `https://api.elevenlabs.io` (set for regional residency) |
| `ELEVENLABS_DASHBOARD_URL_TEMPLATE` | Optional; conversation deep-link pattern, default `https://elevenlabs.io/app/agents/history/{conversation_id}` (not documented by ElevenLabs — override if their dashboard URLs change) |

The KB document id and last publish hash live in the `console_settings` table
(key `kevo.kb_document`), not in env.

## Setup

1. **Migrations** run automatically when core boots (tables + RBAC seeds
   `kevo`, `kevo-publish`). Assign those features to roles under
   Settings → Roles. Suggested mapping — Viewer: kevo read; Reviewer: kevo
   read/create/update/delete; Publisher: reviewer + kevo-publish update.
2. **Register the tool + analysis config** (idempotent, re-run after any change):

   ```sh
   pnpm --filter @karma/core exec tsx src/cmd/kevo-setup.ts
   ```

   This upserts the `kevo_tool_shared_secret` workspace secret, the
   `flag_issue` webhook tool (attached via the agent's `tool_ids`), three
   evaluation criteria and three data-collection fields.
3. **Post-call webhook — manual, workspace-level** (not scriptable):
   in ElevenLabs → Agents settings → Webhooks, create a post-call webhook to
   `{KEVO_PUBLIC_BASE_URL}/v1/kevo/webhooks/post-call` with auth type HMAC and
   the `transcript` event; copy the generated secret into
   `ELEVENLABS_WEBHOOK_SECRET`. Enable retries.
4. **System prompt**: paste [system-prompt-addition.md](./system-prompt-addition.md)
   into the agent's system prompt.
5. **App session**: the member app must pass `member_id` in
   `dynamicVariables` when starting the conversation session, or flags arrive
   without a member reference.

## Tests

```sh
pnpm --filter @karma/core test
```

Covers HMAC verification (fresh/stale/tampered), PII redaction, knowledge
markdown rendering + content hashing, and the merge-safe agent attach
(mocked fetch). Flag dedupe is enforced by a unique DB index on
`(conversation_id, category, COALESCE(entity, ''))`.

## Runbook

- **Rotate the tool secret**: change `KEVO_TOOL_SHARED_SECRET`, restart core,
  re-run `kevo-setup.ts` (it updates the workspace secret in place).
- **Rotate the webhook secret**: recreate the webhook in ElevenLabs settings,
  update `ELEVENLABS_WEBHOOK_SECRET`, restart core.
- **Replay a missed conversation**: create a manual flag from the
  conversation id, or re-send the webhook from ElevenLabs (idempotent —
  BullMQ jobId and the conversation upsert dedupe replays).
- **Publish failed**: the attempt is in `kevo_publish_log` with the reason and
  on the Publish page; fix and click Publish again (content hash skips
  no-op publishes).
- **ClickUp failed**: the reviewer sees the error inline; clicking the button
  again is the retry (no background retry by design).
- **Stuck post-call jobs**: failed jobs retry 6× every 2 min, then park in the
  BullMQ failed set (`bull:kevo-postcall:*` in Redis) for 7 days.

## Deliberately out of scope (v1)

- ClickUp status sync-back (task id + link are stored; add a poll later if needed).
- Post-publish smoke test — the documented simulate endpoint was removed by
  ElevenLabs on 31 Oct 2026; its agent-testing replacement can be added when needed.
- Chat history browsing beyond flag review (the ElevenLabs dashboard owns that).
