Skip to content

FEATURE SPEC

Questions or comments?

Post them as a comment on the tracking issue -- requires a GitHub account with access to this repo.

Sensitive Conversation Detection & Routing

Novaworks AI | x_novaw_guardrails | Stage: Experiment | DRAFT

Parent Feature Set: ../concept.md.

History

Version Date Author / Notes
1 July 29, 2026 Eswar Vandanapu — Initial draft, derived from PRD Feature 5.
2 2026-08-11 Eswar Vandanapu — Noted a new dependency: coalition-of-the-willing/transfer-agents provides generic hand-off infrastructure (reason code, optional summary, receiving-agent resolution). Flagged as a candidate owner for a possible declinable, employee-choice private-transfer outcome — distinct from this Feature's existing mandatory "immediate handoff" outcome — since this Feature already makes the per-turn routing decision. See Dependencies and Open Questions.

Problem Statement

There is no reliable way to recognize that a Nova conversation has turned sensitive — harassment, discrimination, self-harm, legal threat, workplace safety — and get a human involved. This is the highest-stakes and least-proven piece of the guardrail framework: false negatives on crisis content are not acceptable, and false positives that over-escalate ordinary conversations erode trust in the assistant.

Stage note

This is an Experiment-stage feature: it needs a spike before the delivery plan is committed. Per the PRD's sequencing, the spike must prove the classifier meets the false-negative bar on crisis content and that routing fits within normal response latency before delivery is committed.

Scope

Classify each turn of a live conversation into a risk category:

  • Harassment or discrimination complaint
  • Self-harm or mental-health crisis
  • Legal or compliance threat
  • Workplace safety

And decide in real time whether the agent may:

  • Continue
  • Must soften and continue
  • Must hand off to a human immediately

Out of Scope

Value

Ensures employees disclosing harassment, self-harm risk, legal threats, or safety issues are reliably handed to a human rather than answered by a bot — the core capability the PRD identifies as most load-bearing for trust in the whole framework, and the dependency Feature 6's escalation handoff is built on.

Spike

Run the classifier against a labelled set of historical and synthetic conversations spanning each risk category, measure false-negative rate on crisis content specifically, and confirm the routing decision can be made within the conversation's normal response latency.

Acceptance Criteria (spike)

  • [ ] Labelled evaluation set assembled spanning all four risk categories (harassment/discrimination, self-harm/mental-health crisis, legal/compliance threat, workplace safety), historical and synthetic
  • [ ] False-negative rate on crisis content (self-harm/mental-health) measured and reported against an agreed bar
  • [ ] False-positive rate on ordinary conversations measured and reported
  • [ ] Routing-decision latency measured and confirmed to fit within the conversation's normal response latency budget
  • [ ] Spike results reviewed against the false-negative bar before the Feature 5 delivery plan is committed

Acceptance Criteria (delivery, post-spike — TBD pending spike outcome)

  • [ ] Each conversation turn is classified in real time into one of the defined risk categories or "none"
  • [ ] Routing decision (continue / soften-and-continue / immediate handoff) is made per turn based on classification
  • [ ] Every classification and routing decision is reason-coded and available to Feature 4 (Observability, Audit & Feedback Loop)
  • [ ] Immediate-handoff decisions route into Feature 6's escalation queue with full conversation context
  • Feature 3 — Guardrail Policy & Configuration Console: which categories route to a human, and at what threshold, is configured there.
  • Feature 4 — Guardrail Observability, Audit & Feedback Loop: classification/routing decisions must be logged per that feature's inventory once defined; false positives/negatives surfaced by humans feed back into classifier improvement.
  • Feature 6 — Human Escalation & Context Handoff: consumes this feature's immediate-handoff decisions.
  • Phase 1 Audit: the sensitive-conversation categories HR, Legal, and Trust & Safety require feed this feature's category taxonomy.
  • coalition-of-the-willing/transfer-agents: new hand-off infrastructure — given a Transfer Reason, a Receiving Agent, and an optional summary, it records the transfer and resolves the Receiving Agent's contact details (currently a placeholder, not a live session). Candidate consumer if this Feature grows a declinable, employee-choice private-transfer outcome (see Open Questions) — that path would call this infrastructure rather than build its own.

Risks

Risk Severity Mitigation
False negative on crisis content (self-harm, mental-health) Critical Not acceptable per PRD; spike must measure and clear an agreed false-negative bar before delivery is committed.
False positives over-escalating ordinary conversations erode trust High Spike measures false-positive rate alongside false-negative rate; policy console (Feature 3) allows threshold tuning post-launch.
Routing decision latency exceeds normal conversation response time Medium Spike explicitly measures this before commitment.

Open Questions

  • What is the agreed false-negative bar on crisis content that the spike must clear?
  • What labelled historical/synthetic conversation set will be used for the spike, and who owns building it?
  • What does "soften and continue" mean concretely — is there a defined behavior/response pattern for this routing outcome, or is it TBD pending the spike?
  • Full sensitive-conversation category taxonomy depends on the Phase 1 audit (HR/Legal/Trust & Safety requirements) — not yet catalogued.
  • Should this Feature's routing outcomes grow a fourth option — a declinable, employee-choice private transfer (nudge, channel picker among human/AI destinations, summary review) — alongside continue/soften/immediate-handoff? This Feature already makes the per-turn routing call, which makes it one candidate owner; human-escalation-context-handoff (Feature 6) is the other, since it already owns the mandatory handoff's mechanics. Not decided — see the matching note on that Feature's own spec.

Status: DRAFT — derived from PRD Feature 5. Experiment stage — delivery plan not committed pending spike results. For internal engineering review only.

Novaworks AI | Confidential