Plan the cleaning and make the judgement calls
Data cleaning requires making methodological decisions that directly shape the research cohort. In this exercise, you use a structured planning step to evaluate cleaning options proposed by the agent, make necessary domain decisions, and document the resulting protocol in an auditable plan before modifying any data.
Before you start: A data-quality audit of your dataset (Profile the data), or a concise summary of known data anomalies.
Background
Cleaning decisions fall into two distinct methodological categories:
- Deterministic syntactic repairs: Transformations with a single correct solution (e.g., trimming trailing whitespace, standardizing dates to ISO-8601, and eliminating exact duplicate rows). The agent can identify these and propose batch transformations for your approval.
- Substantive methodological choices: Decisions that alter cohort composition or the mathematical meaning of variables (e.g., imputing vs. dropping missing values, converting discordant measurement units, or establishing exclusion thresholds). The agent should propose viable options along with their analytical consequences, leaving the decision to the human researcher.
Enforcing a dedicated planning phase ensures this separation. Many agent harnesses support dedicated plan modes that restrict file edits until you explicitly approve the methodology (Scoping work, Steering).
General methodology
Direct the agent to formulate a comprehensive cleaning protocol based on the profile audit, explicitly separating mechanical repairs from substantive methodological choices:
Do not modify any files yet. Propose an explicit data cleaning protocol for <data file> based on the anomalies identified. For each anomaly, detail viable handling options, downstream analytical implications, and your technical recommendation. Group deterministic syntactic repairs together. Pause and solicit my guidance on substantive methodological choices rather than deciding them autonomously.
Review the proposals, select the preferred remediation rules, and instruct the agent to synthesize the final agreed rules into a concise document (notes/cleaning-decisions.md). Persisting this decision log ensures methodological choices outlive the active conversational session (Memory).
Workshop example · Imaging: chest radiographs
Start from: the imaging kit, out/profile.md and your out/review.csv (or restore the cleaned checkpoint and ignore what it already contains) (how to restore)
Produces: notes/cleaning-decisions.md with your decisions
Prompt
Do not change any files. Using the problems in out/profile.md and my confirmed items in out/review.csv, propose a cleaning plan for the images, labels and scores. For each problem give the options, the consequences of each, and your recommendation. Group the mechanical fixes together, and ask me about any judgement call rather than deciding it yourself. No decision may depend on what a model scored an image; say how your plan guarantees that. When we agree, write the final decisions to notes/cleaning-decisions.md.
Files used: out/profile.md, out/review.csv. Check that out/review.csv exists before you start (ls out/review.csv). If the agent says it cannot find it, the file is probably still in your browser's download folder: move it into out/ (see the previous step). If you skipped the review, download the finished file instead:
Download https://peterchang77.github.io/agent-ai-skills/downloads/imaging-review.csv to out/review.csv (create out/ if needed; if out/review.csv already exists, tell me before replacing it). Fetch only that URL and change nothing else. Then show me the file and how many rows it has.
Mechanical fixes (approve together). Normalize patient IDs (lower case, no spaces, no SITE1- prefix); normalize sex to M/F; fill a blank view position from SeriesDescription; set modality to CR; invert the six
MONOCHROME1 images and set the tag to MONOCHROME2; normalize the label spellings; convert the logit score rows with the sigmoid; match score file names to image file names ignoring case; clear the header patient details.
Judgement calls (the agent should ask you).
| Problem | Options | Why it is your decision |
|---|---|---|
| Inverted images tagged MONOCHROME2 (4, confirmed by you) | Invert the pixels; exclude them | Inverting assumes the picture is otherwise fine. It is likely, but you are changing what the model will see. |
| Mirrored images (4) | Exclude; flip them back | Flipping back looks easy, but laterality matters clinically and you cannot be sure the flip is the only change. Excluding is safer. |
| Patient text burned into the picture (5) | Black out the text; exclude the image | Blacking out removes the information but also edits pixels. Excluding loses a good image. |
| Identical pixels with opposite labels (2 pairs) | Exclude both; keep one | You cannot tell which label is right. Keeping one means guessing. |
| Implausible ages (212Y, 150Y) | Blank the age; exclude | A blank age cannot show the patient is an adult, so the image will drop out at the cohort step anyway. |
| Images of patients under 18 (19 images) | Keep in the cleaned set and exclude at the cohort step; drop now | The protocol covers adults only, but other questions may not. |
| Second study of the same patient (20 images) | Keep and decide at the cohort step | This is not a defect; it matters for the unit of analysis. |
The point of this step. There is no single correct choice for most of these. The reference solution inverts the four confirmed images, excludes the mirrored ones, blacks out the text, excludes both images in each conflicting pair, blanks the implausible ages, and keeps minors and follow-ups until the cohort step. You can choose differently, as long as you decide deliberately and write it down. The rule that is not negotiable: nothing is kept or removed because of a model score.
The reference decisions are in scenarios/imaging/checkpoints/cleaned/notes/cleaning-decisions.md.
Verification criteria
- Clear decision categorization: The proposed plan cleanly separates mechanical formatting fixes from substantive methodological decisions.
- Explicit human steering: The agent pauses to ask for guidance on ambiguous edge cases rather than making arbitrary assumptions.
- Auditable documentation: Every cleaning rule is recorded in
notes/cleaning-decisions.mdwith an unambiguous operational definition.
Common pitfalls
| Symptom | Underlying cause | Resolution |
|---|---|---|
| Agent executes edits during planning | Session was not configured in read-only plan mode, or prompt lacked explicit write prohibitions | Interrupt the run and restate: "Do not modify any files; propose the plan only" |
| Agent resolves substantive ambiguities autonomously | Prompt did not explicitly require options and trade-offs | Instruct the agent: "List alternative methodological options for this issue and pause for my decision" |
Concepts behind this
- Scoping work. Scope requests to sizes that can be independently verified. Require structured plans before executing complex multi-step pipelines. Go deeper →
- Steering. Balance supervisory approval modes with task risk, intervene early when execution drifts, and codify recurring corrections into standing rules. Go deeper →
Builds on: Profile the data and find problems