Skip to content

Plan the cleaning and make the judgement calls

Full step  about 7 min

Data cleaning requires making methodological decisions that directly shape the research cohort. In this exercise, you use a structured planning step to evaluate cleaning options proposed by the agent, make necessary domain decisions, and document the resulting protocol in an auditable plan before modifying any data.

Before you start: A data-quality audit of your dataset (Profile the data), or a concise summary of known data anomalies.

Background

Cleaning decisions fall into two distinct methodological categories:

  • Deterministic syntactic repairs: Transformations with a single correct solution (e.g., trimming trailing whitespace, standardizing dates to ISO-8601, and eliminating exact duplicate rows). The agent can identify these and propose batch transformations for your approval.
  • Substantive methodological choices: Decisions that alter cohort composition or the mathematical meaning of variables (e.g., imputing vs. dropping missing values, converting discordant measurement units, or establishing exclusion thresholds). The agent should propose viable options along with their analytical consequences, leaving the decision to the human researcher.

Enforcing a dedicated planning phase ensures this separation. Many agent harnesses support dedicated plan modes that restrict file edits until you explicitly approve the methodology (Scoping work, Steering).

General methodology

Direct the agent to formulate a comprehensive cleaning protocol based on the profile audit, explicitly separating mechanical repairs from substantive methodological choices:

Do not modify any files yet. Propose an explicit data cleaning protocol for <data file> based on the anomalies identified. For each anomaly, detail viable handling options, downstream analytical implications, and your technical recommendation. Group deterministic syntactic repairs together. Pause and solicit my guidance on substantive methodological choices rather than deciding them autonomously.

Review the proposals, select the preferred remediation rules, and instruct the agent to synthesize the final agreed rules into a concise document (notes/cleaning-decisions.md). Persisting this decision log ensures methodological choices outlive the active conversational session (Memory).

Workshop example · Clinical: clinic network

Start from: the data kit (and, optionally, the profile from the previous step) (how to restore)
Produces: notes/cleaning-decisions.md with your decisions

Prompt

Do not change any files. Using the data problems you found in clinic_visits.csv, propose a cleaning plan. For each problem give the options, the consequences of each, and your recommendation. Group the mechanical fixes together, and ask me about any judgement call rather than deciding it yourself. When we agree, write the final decisions to notes/cleaning-decisions.md.

Files used: clinic_visits.csv, data_dictionary.md.

Mechanical fixes (approve together). Normalise clinic names; convert the six MM/DD/YYYY dates to ISO form; remove the 15 exact duplicate rows; set the four systolic BP values of 0 to missing.

Judgement calls (the agent should ask you).

Problem Options Why it is your decision
HbA1c values of 20 or more (10 rows) Convert from mmol/mol to % using the standard formula; set to missing; exclude the rows Converting assumes they are mmol/mol. That is likely, but it is an assumption about how the data was recorded. Setting to missing is safer, at the cost of 10 values.
Age 212 (1 row) Treat as a typo for the patient's real age (their other visits give one consistent age, 70); set to missing; exclude the row Repairing assumes the other visits are right. Whether you accept that depends on how much you trust the extract.
Two different visits sharing one visit_id (2 rows) Keep both and flag; drop one; assign new IDs Dropping loses a real visit; keeping may break a join on visit_id.
Patients under 18 (8 patients) Keep in the cleaned data and exclude later at the analysis step; drop now The pilot only covers adults, but other questions may not. Excluding later keeps your options open.

The point of this step. There is no single correct choice for the age of 212 or the unit problem; the reference solution converts the units, repairs the age from other visits, keeps both rows with a shared ID (flagged), and keeps minors until the cohort step. You can choose differently, as long as you decide deliberately and write it down. If you set the values to missing instead, results change very slightly, not materially; the lesson is that the decision is recorded, not that it matches ours.

The reference decisions are in scenarios/toy/checkpoints/cleaned/notes/cleaning-decisions.md.

Verification criteria

  • Clear decision categorization: The proposed plan cleanly separates mechanical formatting fixes from substantive methodological decisions.
  • Explicit human steering: The agent pauses to ask for guidance on ambiguous edge cases rather than making arbitrary assumptions.
  • Auditable documentation: Every cleaning rule is recorded in notes/cleaning-decisions.md with an unambiguous operational definition.

Common pitfalls

Symptom Underlying cause Resolution
Agent executes edits during planning Session was not configured in read-only plan mode, or prompt lacked explicit write prohibitions Interrupt the run and restate: "Do not modify any files; propose the plan only"
Agent resolves substantive ambiguities autonomously Prompt did not explicitly require options and trade-offs Instruct the agent: "List alternative methodological options for this issue and pause for my decision"

Concepts behind this

  • Scoping work. Scope requests to sizes that can be independently verified. Require structured plans before executing complex multi-step pipelines. Go deeper →
  • Steering. Balance supervisory approval modes with task risk, intervene early when execution drifts, and codify recurring corrections into standing rules. Go deeper →

Builds on: Profile the data and find problems