Multimodal clinical research for medical AI

Independent, physician-benchmarked safety evaluations for multimodal medical AI.

Evaluations Physician benchmarks

Why this work

Accuracy on text is not the same as clinical safety

A transcript can preserve every spoken word while leaving out labored breathing, an unsteady gait, a companion who answers for the patient, or a monitor reading that contradicts the handoff. Multimodal evaluation asks whether a system registers those observations, whether it understands their significance, and whether the decision it recommends is safe. A cue a model misses may reflect what it was given as input, not only how it reasoned, and the two call for different fixes.

Research

The Same Encounter. Different Evidence.

What a system can read, hear, and see changes what it can observe, interpret, and act on.

All research

  • Ip et al.npj Digital Medicine2024

    Video added information beyond triage data

    In 723 emergency department patients, adding short video clips to triage data modestly improved prediction of hospitalization.

  • Kang et al.medRxiv2026preprint

    The same examination, different sensory access

    Grading the same recorded student examinations, a model agreed best with physician scores from native audio-video, less from audio or transcript.

  • Nagda et al.arXiv2026preprint

    Real-time video scored higher than text chat

    In simulated interactive consultations with actors, video raised overall scores, yet subtle cues such as tremor remained hard to perceive.

  • Sibbald et al.Medical Education2017

    Fast visual impressions were not reliably correct

    Emergency physicians judged patient videos in seconds, but their disposition predictions were imperfect in this small study.

Multimodal Clinical Collections

Controlled clinical simulations developed for training and evaluating how AI interprets text, speech, visual findings, and clinical context.

Case packages may include synchronized audiovisual recordings, transcripts, timestamped observations, controlled variants, physician reference assessments, and scoring criteria. All cases use standardized patients in staged simulations, and collections are developed and produced by arrangement.

Working with Greyoak

Physician-Benchmarked Datasets

Commissioned or rights-cleared simulated case packages for model development, training and evaluation, with physician reference assessments. Evaluation-only use, training rights and exclusive use are agreed separately.

Physician benchmarks Inquire about datasets

Independent AI Safety Evaluations

Physician-referenced testing of how a system handles text, audio and video, with controlled comparisons and case-level findings on clinically consequential failures. Findings do not depend on who commissions the work.

Evaluations Discuss an evaluation

To talk through a project that may need both, contact Greyoak Research.