Portfolio · Project 14

Alexander CHERNOV

Principal Engineer · Agentic AI & scientific data systems

← the whole portfolio
14

adam: AI-written ADaM derivations, accepted only on an exact match

2026 – present · Independent work, unaffiliated with any employer.

A model writes a clinical derivation in seconds. It counts only when its column reproduces the reference, row for row, in a verdict anyone can re-run.

A check for AI-written statistical programs: a model drafts each ADaM column with admiral, the draft runs in isolation on public SDTM data, and the column counts only if it equals the published reference exactly. Programs and verdicts stay on the record with their reasons, and a browser can reproduce every verdict.

The demo as recorded: 1 minute 57 seconds of the deployed demo, captions on screen and no sound. An accepted derivation, the screen-failure date gpt-5 derived from the disposition records, with the 7B model’s program for the same variable stopping with an error; then AEACN, which the reference leaves empty and gpt-5 filled for all 1,191 adverse events; and that verdict re-run in the browser with webR, reaching the same 1,191 differing rows. Every number in a caption was read off the page as it was recorded. The file is the one archived on Zenodo.

adam asks a model to write the {admiral} program for one ADaM variable at a time. The model gets the variable’s name, label and required class, and every input column’s name, class, label and a few example values; never the reference values of the variable, and never the template code. The program runs in a fresh R process with no network, a two-minute limit and a memory limit, and {diffdf} compares the column it produces with the published reference on the dataset keys. Any differing row, a wrong class, lost rows or an error is a rejection, recorded with its reason. It is the check a statistical programming team runs between a first-line program and its second-line QC, with a model in the first-line seat.

The data are public: the CDISC pilot study as packaged by pharmaverse. pharmaversesdtm 1.4.0 and admiral 1.4.1 are the exact set that produced pharmaverseadam 1.3.0, and running admiral’s own templates on them reproduces both reference datasets with zero differences. The 159 targets are every column of ADSL and ADAE except the keys: 109 copied unchanged from one input column and 50 derived. The protocol was committed before either model was asked anything, and the check passed its own self-test first, accepting 2 known-correct programs and rejecting all 8 planted faults for the planted reason, from a date one day late and a population flag missing placebo to a network call and an endless loop.

gpt-5 had 118 of 159 columns accepted: 89 of 109 copied and 29 of 50 derived. qwen2.5-coder 7B, running on a laptop GPU, had 35 of 159. On the same variables, 92 were accepted only for gpt-5 and 9 only for the local model (exact McNemar, p < 0.001). The record also shows what a miss looks like. Nine of gpt-5’s 38 wrong-value rejections are columns the reference leaves empty, which it filled anyway, with invented dictionary codes, consent dates, and an action taken for every one of the 1,191 adverse events, inferred from the dosing records. An exact comparison catches each of them.

Any verdict can be re-run in the browser. The demo carries the same data files and the same R comparison code and runs them with webR, so a verdict does not depend on trusting the machine that recorded it. All 318 recorded verdicts reproduced in headless Chrome through the demo’s own button. Here the reference is the output of the public admiral templates; in production it would be an independent second-line QC program, and the check is agreement with that reference rather than validation in the GxP sense.

Transferable principle: let a model write the first-line program, and accept its output only where an independent check reproduces it exactly.

References

  1. Straub et al. (2026). admiral: ADaM in R Asset Library. R package version 1.4.1, CRAN. doi:10.32614/CRAN.package.admiral
  2. Gautier et al. (2026). pharmaverseadam: ADaM Test Data for the ‘Pharmaverse’ Family of Packages. R package version 1.3.0, CRAN. doi:10.32614/CRAN.package.pharmaverseadam
  3. Patil et al. (2026). pharmaversesdtm: SDTM Test Data for the ‘Pharmaverse’ Family of Packages. R package version 1.4.0, CRAN. doi:10.32614/CRAN.package.pharmaversesdtm
  4. Gower-Page and Martin (2025). diffdf: Dataframe Difference Tool. R package version 1.1.2, CRAN. doi:10.32614/CRAN.package.diffdf
  5. McNemar (1947). Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika 12(2), 153–157. doi:10.1007/BF02295996

adam.chernov.cademo.adam.chernov.caResults and limitsDemo video: Zenodo 10.5281/zenodo.23361650