Chapter 4 · Lesson 3
Debug the Workflow, Not Just the Output
Trace weak results back to inputs, instructions, tools, hand-offs, or review rules.
A recurring AI-assisted report is inconsistent, and the team keeps editing the final document without fixing the process.
Find the first point of failure
Inspect the workflow in order. Were the inputs complete and approved? Did extraction preserve the right facts? Did a transformation remove context? The earliest bad artifact usually explains several later problems.
Change one variable at a time
Test a revised input, instruction, model, or review rule separately. If everything changes at once, you cannot tell what improved the result or whether the process will remain reliable on the next run.
Measure the process, not the demo
Evaluate several representative cases, including awkward inputs and known edge cases. Track correction time, failure type, and whether reviewers agree. A workflow is useful when it performs consistently, not when one ideal example looks impressive.
Build the mental model
Workflow debugging starts with the earliest bad artifact and tests one suspected cause at a time.
Trace backward with evidence
Begin at the visible error, then inspect the artifact from each prior stage. Stop at the first point where the expected information became wrong, incomplete, or ambiguous.
Create a regression set
Save representative failures as test cases. A fix is credible only if it corrects the failure without breaking normal and edge-case inputs.
A weekly action report assigns tasks to the wrong people.
Improve the final report prompt so owners are more accurate.
Trace three wrong assignments to their source notes. Compare source → extraction table → action grouping → final report. Identify the first mismatch. Change only that stage, then run five saved cases: two normal, two ambiguous, and the original failure. Record assignment accuracy and manual correction time.
The process locates the first failure, isolates the change, and verifies it against a small regression set rather than judging one polished report.
Keep a simple workflow log
For each run, record input version, template or model version, failed stage, failure type, correction, and review time. You need enough history to see patterns, not a complex observability platform.
If reviewers disagree about whether output is correct, improve the acceptance criterion before changing the prompt. An unmeasurable target cannot be debugged reliably.
Run a first-failure investigation
Take one recurring correction, trace it backward, change one stage, and test the fix on at least three saved cases.
Visible failure: [...] First bad artifact: [...] Suspected cause: [...] One change: [...] Test cases/metric: [...]multiple choice
A weekly report repeatedly assigns actions to the wrong people. What should you inspect first?
Choose the answer that best fits the principle from this lesson.
Fix the earliest failing stage instead of repeatedly polishing its downstream effects.