Pachinko pachinko
Pachinko pachinko
Design Notes on Useful AI

scope / note 09

Checking its work takes longer than doing it.

Not every task is a good AI task. The first question is whether you can afford to check the work.

The hidden cost is verification.

A generated image takes a glance to judge. A thousand copied numbers with five quiet errors can take an afternoon. When checking costs more than doing, the automation saved nothing. The hours just came back as review.

Explanation is work too.

A model only knows what you give it: instructions, examples, context. Plenty of jobs run on tacit judgment, office history, and rules nobody wrote down. If writing all of that down is the price of admission, automating the job may cost more than doing it.

What to build.

Design output for inspection. Link claims to sources. Show diffs instead of replacing things wholesale. Keep batches small enough to skim, and let users check a sample - three out of thirty - before trusting the rest. Where a task fails this test, offer a draft to edit instead of a confident final version.

design principle

Scope AI to describable work with visible failure and cheap review.

one implementation

Pachinko's version: runs happen on small, explicit batches, and the output is notes you keep, route, or trash. Card view shows the batch as images and previews, so skimming it takes seconds.