FEATURE SPEC¶
Guardrail Observability, Audit & Feedback Loop¶
Novaworks AI | x_novaw_guardrails | Stage: Approach | DRAFT
Parent Feature Set: ../concept.md.
History¶
| Version | Date | Author / Notes |
|---|---|---|
| 1 | July 29, 2026 | Eswar Vandanapu — Initial draft, derived from PRD Feature 4. |
Problem Statement¶
Compliance and Legal have no audit trail of what an agent was asked, what it nearly said, or why a message was blocked or escalated. Without this, compliance sign-off on the whole guardrail framework is not possible — this is flagged in the PRD as the "foundation-first" feature the rest depend on.
Stage note¶
This is a research-and-direction task (Approach), not a build. The deliverable of this stage is an inventory of what must be logged for audit and compliance purposes, and a chosen approach for storage and the feedback loop — not an implementation.
Scope¶
Research and choose an approach for: - Logging every guardrail decision — allowed, blocked, escalated — with a reason code, for compliance and post-incident review - A feedback path so a human's correction of a false positive or false negative can improve the classifiers over time
Frame the options and choose a direction the whole suite follows: - Log to an existing platform data store vs. a dedicated guardrail data store - Human-in-the-loop relabeling vs. periodic offline review
Out of Scope (this stage)¶
- Implementing the chosen storage/logging mechanism
- Implementing the chosen feedback-loop mechanism
Value¶
Gives Compliance/Legal/Trust & Safety a defensible, reason-coded audit trail of every guardrail decision, and gives the guardrail classifiers a path to improve from real human corrections rather than staying static — the prerequisite for compliance sign-off on Features 1, 2, 5, and 6.
Deliverable of this stage¶
- An inventory of what must be logged for audit and compliance purposes (spanning decisions from Features 1, 2, and 5)
- A chosen direction for storage: existing platform data store vs. dedicated guardrail data store
- A chosen direction for the feedback loop: human-in-the-loop relabeling vs. periodic offline review
Acceptance Criteria¶
- [ ] Inventory document lists every field that must be captured per guardrail decision (allow / block / escalate), across Features 1, 2, and 5
- [ ] Storage direction is chosen and documented, with rationale (existing platform data store vs. dedicated guardrail data store)
- [ ] Feedback-loop direction is chosen and documented, with rationale (human-in-the-loop relabeling vs. periodic offline review)
- [ ] Direction is reviewed and accepted by Compliance/Legal/Trust & Safety before Features 1, 2, 5, and 6 rely on it for their own audit logging
- [ ] Correlation id and conversation transaction count (per the platform tracing work) are confirmed as required fields in the inventory
Dependencies / Related Features¶
- Feature 1 — Input Guardrail Engine, Feature 2 — Output Guardrail Engine, Feature 5 — Sensitive Conversation Detection & Routing: each produces guardrail decisions that this feature's inventory must account for.
- Feature 6 — Human Escalation & Context Handoff: escalation records depend on the correlation id and case-record shape this feature's audit trail carries.
- Nova Platform Enhancements PRD v0.4: correlation-id tracing and per-conversation transaction visibility are the tracing foundation this feature's audit log builds on.
Risks¶
| Risk | Severity | Mitigation |
|---|---|---|
| Compliance sign-off on the whole framework blocks on this direction | High | Flagged in the PRD as foundation-first; prioritize resolving this Approach stage before Features 1/2/5/6 reach general availability. |
| Feedback-loop approach chosen without a clear owner for periodic review/relabeling | Medium | Assign an owning team/role as part of the chosen direction, not just a mechanism. |
Open Questions¶
- Existing platform data store vs. dedicated guardrail data store — not yet decided.
- Human-in-the-loop relabeling vs. periodic offline review — not yet decided.
- Who owns ongoing classifier feedback/relabeling once a direction is chosen?
Status: DRAFT — derived from PRD Feature 4. Approach-stage deliverable is a decision, not code. For internal engineering review only.
Novaworks AI | Confidential