Protocol detail

A shared review system for turning expert design judgment into clean training signal.
4
protocol families
12
review fields
1
shared rubric
48h
target turnaround
Protocol library Design intelligence Structured feedback

Each protocol follows the same pattern: define the task, capture evidence, write the rationale, and verify the result. That consistency makes the output easier to compare, review, and train on.

Protocol families

The four task families below are the current core of the review system.

Ranking

Preference Ranking

Compare two or more UI variants and choose the strongest one with a concise design justification.

Objective

Teach the model which visual and interaction patterns produce better outcomes for a given task.

Inputs
  • Two or more candidate UI variants
  • Task brief and target device
  • Any accessibility constraints
Output
  • Winner selection
  • One-line rationale
  • Failure notes for rejected options
Quality checks
  • Hierarchy and scan path
  • Spacing and alignment
  • Clarity of affordances
Annotation

UI Annotation

Mark exactly where a layout breaks down and assign the issue to a specific category.

Objective

Convert visual critique into structured labels that can be learned from at scale.

Inputs
  • Wireframe, mock, or screenshot
  • Annotation layer / markup tool
  • Failure taxonomy
Output
  • Bounding notes or callouts
  • Issue category
  • Priority / severity score
Quality checks
  • Evidence is visible in the artifact
  • Labels map to the rubric
  • No duplicate annotations
Writing

Rationale Writing

Explain why a solution is good, not just that it looks polished.

Objective

Train the model on the language of expert judgment and decision making.

Inputs
  • Selected solution or design direction
  • Context about the user and workflow
  • Review rubric
Output
  • Short design rationale
  • Tradeoff summary
  • Next-step recommendation
Quality checks
  • Rationale ties back to user goals
  • Reasoning is specific and testable
  • No generic filler language
Stress test

Red Teaming

Push a model or tool into edge cases and document where it fails.

Objective

Reveal brittle assumptions before the design system or agent reaches users.

Inputs
  • Target tool or model
  • Failure scenarios
  • Expected bad behavior examples
Output
  • Failure report
  • Reproduction steps
  • Recommended fix or guardrail
Quality checks
  • Issue is reproducible
  • Root cause is described clearly
  • Mitigation is concrete

Protocol workflow

The workflow keeps every protocol comparable, even when different designers are reviewing different artifacts.

01

Scope the task

Define the artifact, the user intent, and the review criteria before any judgment is recorded.

02

Capture evidence

Annotate the exact area that supports the decision so the feedback stays grounded in the UI.

03

Write the rationale

Turn visual criticism into concise, reusable reasoning that can train future behavior.

04

Verify quality

Run the output against the rubric to make sure the signal is consistent and actionable.

Data model
Field Value
Artifact Design mock, UI screenshot, or agent-generated interface
Reviewer Senior product designer with domain match
Primary signal Preference, annotation, rationale, or failure mode
Quality gate Specific, reproducible, and rubric-aligned feedback
Delivery format Structured notes, labels, or ranked output

Review rule

If the output cannot be traced back to a visible artifact, a clear rubric item, or a reproducible failure, it should be revised before acceptance.

Ready to review

Use the protocol library as your operating system for design signal.

Share this page with your team when you need a repeatable way to rank UI outputs, annotate issues, and write sharper design rationale.

Talk to Rothr
View related section
Rothr logo Rothr

The design intelligence infrastructure for frontier AI. Mobilizing senior designers to generate high-fidelity training data.

Intelligence

For Designers

For Labs

Company

Legal

© 2026 Rothr