Practical AI starts with work someone can judge

Choose a task with a visible finish

In my work with AI, I keep coming back to the value of judgment. Producing a plausible answer quickly still leaves someone responsible for understanding its assumptions, recognizing what is missing, and deciding whether it can be used. Domain knowledge remains central to that responsibility.

Choose one repeated task that ends in something tangible: a draft, a categorized list, a comparison, or a preparation note. Name the person who receives that output and what they do with it. This gives the pilot a manageable boundary.

Define the review before creating the output

Write a short agreement with the person who performs the task and the person accountable for its result. Settle these questions together:

1. Inputs. What information is available, appropriate for the chosen tool, and necessary for this task?
2. Required output. What must the result contain, and what should it leave for a person to decide?
3. Quality checks. Which errors matter most, and how will the reviewer detect them?
4. Authority. Who can accept the result, revise it, or stop it from moving into the next step?
5. Fallback. How will the work continue if the output is missing, unsuitable, or takes too long to verify?

A review that nobody has time or sufficient knowledge to perform is an unresolved design problem. Address it before expanding the pilot.

Illustrative example: preparing an operating review

Illustrative example: A team wants AI to prepare a draft brief from an approved action register before its weekly review. The brief should organize upcoming commitments and highlight entries that need discussion.

The reviewer checks the draft against the register. Every named owner and due date must be supported by an entry. Missing information should remain visibly unresolved. Suggestions must be distinguishable from recorded commitments.

The draft then becomes preparation for the meeting. The accountable people confirm decisions and changes through the team’s normal process. This keeps the pilot focused on preparation while preserving clear responsibility for what the team commits to do.

Measure the whole task

Time the work from receiving the input to accepting the finished output. Include preparation, prompting, checking, corrections, and recording the result. A faster first draft may still require substantial review.

Compare the pilot with a small baseline from the current workflow. Use consistent definitions for:

• Total effort required to complete the task.
• Corrections needed before the output is usable.
• Important information omitted or represented incorrectly.
• Whether the receiving person can proceed without extra explanation.

Keep the comparison modest and specific. A useful result for one task does not automatically establish that the approach suits every task performed by the team.

Make stopping an ordinary decision

Agree in advance what would cause the team to pause. That might be repeated unsupported statements, an input change the reviewer cannot assess, or checking effort that outweighs the benefit. A stopping rule gives the owner a clear decision when the experiment needs adjustment.

Record why an output was rejected. Look for patterns in the task, the input, and the review instructions. Then decide whether to narrow the task, improve the information, revise the process, or return to the existing approach.

Finish with an operating decision

End the pilot with a documented decision: continue as designed, revise and try again, or stop. If it continues, assign ownership for the instructions, quality review, and future changes. Make the accepted process easy for another team member to follow.

The most useful outcome is a clear understanding of where the approach helps, what it requires from people, and how the team will keep judging its performance as the work changes.