Signal, Not Secrets - StarterÂ
Signal, Not Secrets is a publish-ready framework for documenting AI conversation failure modes as observable, testable patterns without making accusations, implying hidden agendas, or relying on “trust me” claims.
This guide helps you turn messy transcripts into clean evidence packets, tag failure patterns consistently, and propose practical mitigations that product, safety, and research teams can actually implement.
What you’ll get
Inside the PDF, you’ll receive:
-
The Signal Lens: a neutral reporting stance that describes outputs as behaviors, not motives
-
A 10-pattern taxonomy of common conversation failure modes (reframe switches, safety script intrusion, contradiction cascades, context drops, and more)
-
Evidence Documentation system: a minimum viable evidence packet, control variables, and repeatability testing guidance
-
A fast scoring rubric to evaluate relevance, framing fidelity, consistency, safety intrusiveness, and repair quality
-
Stress-testing prompt families + metrics so you can reproduce patterns and track thresholds
-
Mitigation playbook: mode locks, frame retention, reconciliation discipline, and “two-lane” responses
-
Copy/paste templates: Pattern Map Template, Evidence Log Template, and a one-page Case Study format
Who this is for
-
Product, safety, and research teams who need defensible documentation
-
Builders and operators who want repeatable QA language
-
Writers, auditors, and analysts who publish findings and need neutral, evidence-forward framing
You do not need perfect prompts. You need a repeatable method.
If it repeats, it’s a behavior. If it’s a behavior, it has triggers. If it has triggers, it can be documented. Start with one transcript, capture it cleanly, run 3 controlled variants, and you’ll have something real to build on.
