Skip to content

Demo · Imaging

This track follows one worked example for a radiology and imaging-informatics audience: a department wants to know whether an open-source AI tool that flags pneumothorax (a collapsed lung from air leaking into the chest) on chest radiographs is ready for a pilot. You direct a coding agent through the same workflow used in the clinical demo track: inspect the inputs, audit and clean them, define the study population, analyze, and write up, but the data are DICOM images, the cleaning involves pixels as well as headers, and the analysis is the evaluation of an AI model.

You do not need programming experience to follow along. Each step explains the background, gives a prompt to paste, shows what a good result looks like, and lists what to check. Steps marked as optional or advanced can be skipped.

Case study overview

The exercises follow a small public sample of chest radiographs (real images, with synthetic data problems added) together with model scores, a Word evaluation protocol, a request memo and short radiology reports:

DICOM files → look → profile → review in a viewer → plan and clean → cohort → evaluate → references → abstract
                                      │                                              │
                                      └─→ a person confirms what numbers      ├─→ dashboard, Word/Excel export
                                          cannot show                         └─→ reusable tool and skill → Git

The workflow addresses five questions: how well the primary model separates pneumothorax from none, whether it is really better than three alternatives, whether it works equally well across views, sexes and ages, whether any model meets the pilot bar, and (separately) how to sort 200 radiology reports.

What is distinctive in this example

  • Pixels as well as headers. Header problems are found by code; some pixel problems (a mirrored or inverted image) are not, so a person has to look. You build a small viewer for this.
  • Privacy at the pixel level. Patient details can sit in the header and be burned into the image.
  • Evaluating an AI tool honestly. Intervals, comparing models on the same patients, and treating subgroup differences with care.

This is a toy demonstration with a small public image sample. Nothing here is evidence about any real tool.

Each demo module follows a standardized structure:

  1. Background: Core context and prerequisites for the immediate task.
  2. General methodology: Reusable prompt patterns and computational techniques applicable to any dataset.
  3. Workshop example: A concrete demonstration using this track's benchmark dataset.
  4. Verification criteria & common pitfalls: Targeted guidance for evaluating outputs and troubleshooting discrepancies.
  5. Supporting concepts: Direct references back to foundational principles in the concept track.

Modular progression and checkpoints

You do not need to execute the demo from start to finish in a single pass. Every exercise defines an explicit starting state, consisting of either the initial dataset or a saved checkpoint reflecting earlier results. The Setup guide provides instructions for restoring any checkpoint, allowing you to jump directly into specific exercises.

Exercise categories

Category Description
Full step Core stages of the research pipeline
Quick step Focused, high-leverage administrative or environment checks
Optional detour Deep-dive tasks that enrich the analysis without blocking subsequent steps
Advanced Exercises requiring external API credentials or cloud budget

Adapting to proprietary data

The general methodologies apply universally to empirical datasets. The workshop example demonstrates how these techniques operate on data with known answers. Before adapting these workflows to proprietary or sensitive institutional data, consult Privacy and data handling and Secrets.

Contents

Start

Step Kind Time What you do
Setup Setup Restore the data kit and any starting point you need.
Orient the agent in a folder Quick step 3 min Direct the agent to audit local project files, establishing environmental grounding and identifying files requiring specialized parsers.
Prepare your environment Quick step 5 min Direct the agent to audit local dependencies, initialize an isolated project virtual environment, and document lockfile-pinned setup procedures.

1. Understand the inputs

Step Kind Time What you do
Read and summarize text files Full step 5 min Direct the agent to inspect and synthesize project-local text files, validating source provenance and avoiding copy-paste truncation.
Read spreadsheets, documents, and PDFs Full step 7 min Ingest XLSX, DOCX, and PDF containers, auditing structural extraction compromises such as merged cells, table boundaries, and OCR artifacts.
Look at the images Full step 6 min Have the agent read image headers and render a labelled contact sheet you can open, so that your own eyes check what the numbers say.
A first answer from raw data Full step 6 min Generate an initial exploratory estimate from uncleaned data, requiring explicit definitions and denominators to establish an uncleaned baseline.

2. Check and clean the images

Step Kind Time What you do
Profile the data and find problems Full step 7 min Generate a systematic data-quality audit documenting anomalies, null patterns, and unit mismatches without modifying source files.
Build a viewer and review the flagged images Full step 10 min Have the agent flag suspect images with code, build a small offline viewer, and confirm or reject each flag yourself before anything is changed.
Plan the cleaning and make the judgement calls Full step 7 min Formulate an explicit data cleaning protocol separating deterministic syntactic repairs from substantive domain decisions before modifying data.
Clean the data with a script, and show the flow Full step 7 min Implement approved cleaning rules in a non-destructive script that produces cleaned datasets and a reconciled record-flow table.
Make the agent remember with AGENTS.md Full step 8 min Persist critical project rules and analytical conventions in AGENTS.md, ensuring new sessions maintain project memory without redundant steering.

3. Evaluate the AI tool

Step Kind Time What you do
Build a study cohort with a flow table Full step 8 min Translate formal study eligibility criteria into a reproducible cohort dataset, accompanied by an attrition flow diagram reconciling excluded populations.
Evaluate an AI tool, with intervals Full step 12 min Measure how well a model separates positive from negative cases, compare models on the same patients, check subgroups, and report every estimate with its uncertainty.
Run an AI model yourself, on a CPU Optional detour 10 min Reproduce the scores you were given by running the pretrained model on your own machine, and explain every score that does not match.
Review the analysis with a fresh second agent Optional detour 6 min Commission an independent peer review of scripts and findings using a secondary agent session with a clean context window.

4. References and writing

Step Kind Time What you do
Find references from a bibliographic database Full step 9 min Build a verified BibTeX bibliography via programmatic API retrieval (PubMed E-utilities), establishing a reproducible search log and eliminating hallucinated citations.
Build references from links you provide Optional detour 6 min Construct an auditable BibTeX bibliography from researcher-curated URLs, using DOI content negotiation and PMID lookups while barring model speculation.
Write a research abstract as LaTeX, with citations Full step 9 min Author a structured scientific abstract in LaTeX from empirical results files and a verified bibliography, compiling a publication PDF and tracing numerical claims to their source.

5. Outputs for others

Step Kind Time What you do
Build a dashboard as a single web page Full step 7 min Compile an interactive, self-contained HTML research dashboard displaying aggregated metrics and audit counts without external runtime dependencies.
Export to Word and Excel for collaborators Optional detour 6 min Convert plain-text source files into DOCX and XLSX deliverables for external collaborators, auditing formatting trade-offs and preserving master text sources.

6. Make it repeatable

Step Kind Time What you do
Turn the analysis into a tool and a skill Full step 10 min Package proven analytical pipelines into tested, parameterized command-line tools and modular agent skills for persistent reproducibility.
Review and commit what the agent produced Quick step 5 min Audit repository state, configure .gitignore to exclude raw data and secrets, and record an immutable Git commit snapshot.
Check usage and compact the conversation Quick step 5 min Monitor cumulative token consumption, synthesize session decisions into a persistent summary file, and initialize a clean working context.

Advanced

Step Kind Time What you do
Use a secret safely Advanced 7 min Configure API credentials securely via restricted .env environment files and .gitignore, preventing secret leakage into transcripts and version control.
Classify many records with a language-model API Advanced 12 min Execute batch record classification via programmatic API calls with dry-run estimation, incremental state checkpointing, and validation against a benchmark sample.
Retrieve images from a DICOM server Advanced 12 min Direct the agent to read from an image server through its web interface, read-only and with credentials kept out of the transcript, and reconcile what it finds with the files you already have.

Track navigation