Skip to content

Tutorial · Imaging

This guided tutorial provides a structured, linear path through the workshop on a radiology example: you direct a coding agent to audit and clean a set of chest radiographs, define a study population, and evaluate open-source AI tools that flag pneumothorax. By sequencing conceptual background alongside practical exercises, it ensures you have the necessary context immediately before each hands-on task, with direct links back to the reference pages.

You can complete it in one session or across several. Work from each stage is preserved as a restorable checkpoint, so you can pause, resume, or skip forward at any point (Setup).

This is a toy demonstration on a small public image sample with planted data problems. Nothing here is evidence about any real tool.

Learning outcomes

By completing this tutorial, you will produce:

  • A configured local agent environment with safe data-handling and credential practices, including for images that carry patient information.
  • A reproducible image-cleaning pipeline with a small offline viewer, a human review of what code cannot find, and an explicit audit trail of decisions.
  • An evaluation of four AI models with intervals, paired comparisons and cautious subgroup analysis, independently reviewed by a second agent.
  • A verified bibliography, an abstract with citations, and a self-contained results dashboard.
  • A reusable command-line tool and agent skill tracked in Git, alongside a bulk report-classification workflow using an external API.

Prerequisites

  • A computer with terminal access where you can install tools (or where an agent CLI is already configured). The initial step walks through setup.
  • A dedicated working directory for workshop files.
  • For the advanced section only: an active API key with a modest usage budget.
  • Review the concept page first, then proceed to the demo exercise. Concept summaries are intentionally concise, providing the methodological grounding before you run commands.
  • Work through the benchmark dataset before using your own data. The provided scenario includes known answers, making it straightforward to evaluate whether the agent's calculations are accurate before applying the patterns to novel data.
  • Adapt to your schedule. Optional detours and advanced exercises are clearly labelled. If you have limited time, focus on the core "Full step" exercises listed in the demo track.

Contents

1. Get started

Step Type What it covers
Setup Setup Restore the data kit and any starting point you need.

2. Meet your agent

Step Type What it covers
Agent vs. chat Concept A standard chat model answers from training memory. An agent interacts with a live computing environment to inspect files, execute code, and produce verifiable artifacts.
Orient the agent in a folder Demo Direct the agent to audit local project files, establishing environmental grounding and identifying files requiring specialized parsers.
What code makes possible Concept Coding agents use executable code as a universal interface for research tasks, making text-based source formats and programmatic scripts far more reliable than manual GUI interactions.
Harnesses and models Concept An agent system separates the inference engine (the model), the orchestration software (the harness), and the execution target (the environment). Understanding this separation clarifies system capabilities and boundaries.

3. Work safely

Step Type What it covers
Privacy and data handling Concept Data read by an agent is transmitted to the model provider as context. Determining whether data is permitted to leave your infrastructure must precede deciding what tasks to delegate.
Environment Concept Computational research requires isolated execution environments to prevent dependency conflicts and guarantee that findings can be reproduced on other systems.
Prepare your environment Demo Direct the agent to audit local dependencies, initialize an isolated project virtual environment, and document lockfile-pinned setup procedures.

4. Understand the inputs

Step Type What it covers
Inputs Concept Language models process information as text. Selecting appropriate ingestion pathways and source formats preserves data structure and prevents extraction errors.
Read and summarize text files Demo Direct the agent to inspect and synthesize project-local text files, validating source provenance and avoiding copy-paste truncation.
Read spreadsheets, documents, and PDFs Demo Ingest XLSX, DOCX, and PDF containers, auditing structural extraction compromises such as merged cells, table boundaries, and OCR artifacts.
Look at the images Demo Have the agent read image headers and render a labelled contact sheet you can open, so that your own eyes check what the numbers say.
A first answer from raw data Demo Generate an initial exploratory estimate from uncleaned data, requiring explicit definitions and denominators to establish an uncleaned baseline.

5. Check and clean the images

Step Type What it covers
Scoping work Concept Scope requests to sizes that can be independently verified. Require structured plans before executing complex multi-step pipelines.
Verification Concept Language models can be confidently incorrect. Require empirical evidence, verify mathematical invariants, and use clean-context peer review to audit findings.
Profile the data and find problems Demo Generate a systematic data-quality audit documenting anomalies, null patterns, and unit mismatches without modifying source files.
Build a viewer and review the flagged images Demo Have the agent flag suspect images with code, build a small offline viewer, and confirm or reject each flag yourself before anything is changed.
Steering Concept Balance supervisory approval modes with task risk, intervene early when execution drifts, and codify recurring corrections into standing rules.
Plan the cleaning and make the judgement calls Demo Formulate an explicit data cleaning protocol separating deterministic syntactic repairs from substantive domain decisions before modifying data.
Clean the data with a script, and show the flow Demo Implement approved cleaning rules in a non-destructive script that produces cleaned datasets and a reconciled record-flow table.
Memory Concept Language models do not retain memory across sessions. Preserving knowledge requires externalizing guidelines, documentation, and tools into persistent files.
Make the agent remember with AGENTS.md Demo Persist critical project rules and analytical conventions in AGENTS.md, ensuring new sessions maintain project memory without redundant steering.

6. Evaluate the AI tool

Step Type What it covers
Context Concept Context represents the model's active working memory. Because every turn re-transmits the complete conversation, long sessions increase latency, cost, and attention drift.
Build a study cohort with a flow table Demo Translate formal study eligibility criteria into a reproducible cohort dataset, accompanied by an attrition flow diagram reconciling excluded populations.
Evaluate an AI tool, with intervals Demo Measure how well a model separates positive from negative cases, compare models on the same patients, check subgroups, and report every estimate with its uncertainty.
Run an AI model yourself, on a CPU Demo Reproduce the scores you were given by running the pretrained model on your own machine, and explain every score that does not match.
Review the analysis with a fresh second agent Demo Commission an independent peer review of scripts and findings using a secondary agent session with a clean context window.

7. Write it up

Step Type What it covers
Outputs Concept Request text-based source files and compile them to presentation formats. Source files are inspectable, version-controlled, and reproducible.
Find references from a bibliographic database Demo Build a verified BibTeX bibliography via programmatic API retrieval (PubMed E-utilities), establishing a reproducible search log and eliminating hallucinated citations.
Build references from links you provide Demo Construct an auditable BibTeX bibliography from researcher-curated URLs, using DOI content negotiation and PMID lookups while barring model speculation.
Write a research abstract as LaTeX, with citations Demo Author a structured scientific abstract in LaTeX from empirical results files and a verified bibliography, compiling a publication PDF and tracing numerical claims to their source.
Build a dashboard as a single web page Demo Compile an interactive, self-contained HTML research dashboard displaying aggregated metrics and audit counts without external runtime dependencies.
Export to Word and Excel for collaborators Demo Convert plain-text source files into DOCX and XLSX deliverables for external collaborators, auditing formatting trade-offs and preserving master text sources.

8. Make it repeatable

Step Type What it covers
Tools Concept Turning interactive prompts into parameterized scripts creates deterministic, testable tools that run identically on every execution.
Turn the analysis into a tool and a skill Demo Package proven analytical pipelines into tested, parameterized command-line tools and modular agent skills for persistent reproducibility.
Version control Concept Version control provides an immutable audit trail and safety net, allowing researchers to inspect every line changed by an agent and roll back errors instantly.
Review and commit what the agent produced Demo Audit repository state, configure .gitignore to exclude raw data and secrets, and record an immutable Git commit snapshot.
Costs Concept Language model pricing is determined by token volume. Because conversation history re-transmits on every turn, session length compounds costs quadratically.
Check usage and compact the conversation Demo Monitor cumulative token consumption, synthesize session decisions into a persistent summary file, and initialize a clean working context.

Advanced

Step Type What it covers
Secrets Concept Credentials must never enter prompts, session logs, or version control. Use isolated environment variables and enforce immediate revocation protocols if exposed.
Sandboxing and isolation Concept Coding agents execute commands with your user permissions. Enforcing appropriate isolation boundaries protects local files and prevents indirect injection attacks.
Use a secret safely Demo Configure API credentials securely via restricted .env environment files and .gitignore, preventing secret leakage into transcripts and version control.
Classify many records with a language-model API Demo Execute batch record classification via programmatic API calls with dry-run estimation, incremental state checkpointing, and validation against a benchmark sample.
Retrieve images from a DICOM server Demo Direct the agent to read from an image server through its web interface, read-only and with credentials kept out of the transcript, and reconcile what it finds with the files you already have.

Track navigation