Skip to content

FEATURE SPEC

Questions or comments?

Post them as a comment on the tracking issue -- requires a GitHub account with access to this repo.

Output Guardrail Engine

Novaworks AI | x_novaw_guardrails | Stage: Refine | DRAFT

Parent Feature Set: ../concept.md.

History

Version Date Author / Notes
1 July 29, 2026 Eswar Vandanapu — Initial draft, derived from PRD Feature 2.

Problem Statement

Nova's conversational agents send model output straight to the user with no shared checkpoint. There is no consistent way to catch policy-violating content, ungrounded or hallucinated claims about a system-of-record fact, PII or other regulated data appearing in the response, or tone/brand deviations, before the response is shown.

Scope

Screen every agent-generated response before it reaches the user: - Policy-violating content — including indirect or hypothetically-framed attempts to elicit it (e.g. "asking for a friend"), not just direct requests - Ungrounded or hallucinated claims about a system-of-record fact - PII or other regulated data appearing in the response - Tone or brand deviations

This builds on established content-moderation patterns, so it carries less uncertainty than the sensitive-conversation work in Feature 5.

Out of Scope

Value

A quick win per the PRD's grouping — leans on established content-moderation patterns to give employees confidence that a policy-violating, hallucinated, or data-leaking response is caught before they see it, without needing the higher-uncertainty sensitive-conversation classification work.

Delivery

Stand up the output-checkpoint service, define reason codes per block type, and give each app a fallback response to show when a generated reply is withheld.

Acceptance Criteria

  • [ ] Output-checkpoint service is deployed and reachable by scoped apps as part of x_novaw_guardrails
  • [ ] Every agent-generated response is screened before reaching the user
  • [ ] Policy-violating content is detected and blocked
  • [ ] Ungrounded/hallucinated claims about system-of-record facts are detected
  • [ ] PII or other regulated data appearing in a response is detected
  • [ ] Tone or brand deviations are detected
  • [ ] Policy-circumvention instructions are caught even when phrased indirectly (e.g. hypothetical/third-person framing like "asking for a friend...") — not just direct requests
  • [ ] No insult or demeaning language is ever included in a response, regardless of any persona or personalization settings in effect
  • [ ] Reason codes are defined per block type
  • [ ] Each scoped app has a fallback response to show when a generated reply is withheld
  • [ ] Every checkpoint decision (allow / block / modify) is reason-coded and available to Feature 4 (Observability, Audit & Feedback Loop)
  • Feature 3 — Guardrail Policy & Configuration Console: thresholds and which checks run are configured there, not hardcoded here.
  • Feature 4 — Guardrail Observability, Audit & Feedback Loop: every output-checkpoint decision must be logged per that feature's inventory once it is defined.

Risks

Risk Severity Mitigation
Hallucination detection against system-of-record facts requires a grounding source per app Medium Scope which system-of-record facts are checkable at launch; expand grounding sources incrementally.
Over-blocking (false positives on tone/brand deviation) Low Reason-coded logging (Feature 4) enables tuning; policy console (Feature 3) allows threshold adjustment.

Open Questions

  • What is the grounding source/mechanism for detecting hallucinated system-of-record claims, per app?
  • Are tone/brand deviation checks a shared standard across all scoped apps, or per-app configurable?
  • Exact fallback-response copy per app is not yet defined.

Status: DRAFT — derived from PRD Feature 2. For internal engineering review only.

Novaworks AI | Confidential