Before you begin
Have the following ready:- One human-facing AI workflow to evaluate
- A specific behavior you want the experience to support
- Two product variants or configurations to compare
- Representative situations in which people use the workflow
- A decision you expect the evidence to inform
1
Create or open a notebook
Use a notebook to hold the evaluation question, scenarios, variants, signals, runs, and results for one product workflow.
2
Describe the intended behavior
Write the behavior in terms a reviewer could observe or score. Prefer “The user identifies the correct next step” over “The experience is helpful.”
3
Define signals
Choose the measurable signs that distinguish the intended behavior from its alternatives. Keep each signal focused on one judgment.
4
Add scenarios and variants
Use situations that represent real decisions, including cases where the correct response is uncertain, constrained, or requires escalation.
5
Review readiness and launch
Resolve the readiness checks, confirm the frozen study setup, and launch the sprint. CiCi runs APM tasks in the background and reports progress.
6
Read the Results summary
Start with lifecycle status and evidence limits. Then inspect Behavioral Evidence, outcomes supported by frozen briefs, and any available Signal Gap.
7
Choose the next evidence step
If modeled evidence is sufficient for your immediate decision, create a test-ready refinement brief. If the decision depends on how people actually respond, plan human validation.
Next steps
Plan an evaluation
Turn a product question into scenarios, signals, and comparison variants.
Read your results
Interpret the lifecycle panel, evidence views, and withheld claims.