Build a viewer and review the flagged images
Some problems in image data cannot be found by code alone. Here you split the work: the agent flags candidates with measurable checks and builds a tool that lets a person review them, and you make the call. Only reviewed items are acted on later.
Before you start: A profile of the images, with the problems code could find (Profile the data).
Background
Code can check metadata exhaustively and can check some pixel properties (size, brightness patterns, exact duplicates). It cannot reliably tell that a picture is mirrored, or that a picture is inverted when the file claims it is not. Those need a person looking. This is a human-in-the-loop pattern: the machine narrows hundreds of images to a handful, a person confirms them, and the decision is recorded in a file the agent reads back (Verification).
The viewer is a good first tool to build with an agent (Tools): small, offline, and reusable. A self-contained HTML page works on any machine with a browser and sends nothing over a network, which also matters for privacy: patient details can sit in the pixels as well as the header (Privacy).
General methodology
Ask for flags with a stated reason for each, then a viewer, then record your decisions and let the agent read them back:
Write a script that scans <folder> and flags images that look wrong, writing <out/flags.csv> with columns file and flag (one row per flag). Use measurable checks and say what each flag means. Then write a second script that builds one self-contained, offline HTML page showing a thumbnail for every image, a filter by flag, a larger view with the header on click, and confirm / reject / unsure buttons that can be exported as <out/review.csv>. Do not embed any labels or model results. Tell me how to open it.
Open the page, review the flagged images, then use the export button to save review.csv. The export is a browser download, so move it into the project's out/ folder yourself (see the example), and tell the agent where it is. Do not let the agent decide for you which flags are real.
Workshop example · Imaging: chest radiographs
Start from: the imaging kit and your out/profile.md (how to restore)
Produces: scripts/flag_images.py, out/flags.csv, scripts/make_viewer.py, out/viewer.html, and your out/review.csv
Prompt 1: Flags
Write scripts/flag_images.py that scans images/ and writes out/flags.csv with columns file and flag. Flag: unreadable files; images that are not chest radiographs; images that are not 1024 x 1024; images whose PhotometricInterpretation is MONOCHROME1; images that look inverted even though they are tagged MONOCHROME2 (use a measurable brightness check and explain it); images with burned-in text in the top-left corner. Print a count per flag, and tell me which problems this cannot find.
Prompt 2: Viewer
Write scripts/make_viewer.py that builds one self-contained, offline HTML page, out/viewer.html, from images/ and out/flags.csv: a thumbnail grid for all images, a filter by flag, a larger view with the DICOM header on click, and confirm / reject / unsure buttons that can be exported as review.csv. Show MONOCHROME1 images the way a viewer would. Do not embed labels or model scores. Tell me how to open it.
Files used: images/*.dcm, out/profile.md.
What you should see (flags). Roughly: 2 unreadable, 3 not chest, 4 at 512 x 512, 6 MONOCHROME1, 5 with burned-in text, and a handful of suspected inversions: the 4 real ones plus about one false alarm (a genuine image with unusual contrast). The mirrored images are not flagged, and the agent should say so.
What you do. Open out/viewer.html and review. Filter to each flag, confirm the real ones and reject the false alarm. Then scan the whole grid for what code could not find: four images are mirrored (look at
the burned-in letters, which read backwards, and at which side the heart is on). Mark them and export review.csv. A reference version is in scenarios/imaging/checkpoints/cleaned/out/review.csv.
Where the exported file goes. The Export button is an ordinary browser download, so review.csv lands in your browser's download folder (usually ~/Downloads), not in the project. Move it into place before the next step:
mv ~/Downloads/review.csv out/review.csv # your own computer
In Cloud Shell or on any remote machine the viewer opens in the browser on your computer, so the file is downloaded there. Download out/viewer.html from the file explorer to open it, then use Upload in the Cloud Shell menu to send review.csv back, and move it into out/. Check with ls out/review.csv before asking the agent to use it; an agent that cannot find the file may search elsewhere or guess. Short on time? Skip the review and download a finished review.csv (the reference review: the 4 inverted and 4 mirrored images confirmed) into out/:
Download https://peterchang77.github.io/agent-ai-skills/downloads/imaging-review.csv to out/review.csv (create out/ if needed; if out/review.csv already exists, tell me before replacing it). Fetch only that URL and change nothing else. Then show me the file and how many rows it has.
Give the agent this prompt. It should fetch one file, report 8 rows (4 inverted, 4 mirrored), and not touch anything else; approve the network request only if the URL is exactly the one shown.
Reference answers
| Issue | In the kit | Found by code | Needs you |
|---|---|---|---|
| Inverted, wrong tag | 4 | 4 flagged (plus about 1 false alarm) | Confirm and reject |
| Mirrored left-right | 4 | 0 | Find by eye |
| Burned-in text | 5 | 5 | Confirm |
Check the tool. Open the HTML source and confirm it contains no web links and no labels or scores. In a headless setting, you can ask the agent to take a screenshot of the page to check that it renders; the agent can then read that picture only if its model supports images.
A reference viewer is in scenarios/imaging/reference/make_viewer.py. Compare yours to it only after checking yours.
Verification criteria
- Every flag has a reason and a count, and the agent reports flags it could not test.
- The page is offline and self-contained (no web requests). Check the file for
httplinks. - No answers in the page. Labels, scores and other results are absent, so your review is not biased by them.
- Decisions come from you. The agent uses
review.csvas given and reports any flag you left unreviewed.
Common pitfalls
| Symptom | Underlying cause | Resolution |
|---|---|---|
| The page is very large or slow | Full-size images were embedded | Ask for downsampled thumbnails and a larger view at moderate size |
| Flags are trusted without review | The agent acted on its own detector | Insist: "Only act on items I confirmed in review.csv" |
| A problem was never flagged | No check can see it | Scan the whole sheet yourself; this step exists because code cannot find everything |
Concepts behind this
- Verification. Language models can be confidently incorrect. Require empirical evidence, verify mathematical invariants, and use clean-context peer review to audit findings. Go deeper →
- Tools. Turning interactive prompts into parameterized scripts creates deterministic, testable tools that run identically on every execution. Go deeper →
- Privacy and data handling. Data read by an agent is transmitted to the model provider as context. Determining whether data is permitted to leave your infrastructure must precede deciding what tasks to delegate. Go deeper →
Builds on: Profile the data and find problems