MIS 752 · Lab 12 · LienJ

The Physical Exam, voice: an audited ambient-scribe draft of a synthetic visit

One synthetic clinic visit, spoken by a local voice model, transcribed by Whisper, written up by a local language model, and audited for what was misheard and what was invented. $0, no PHI. These numbers are this run's.

The 30-second read

THE VISIT 21 turns, 237 words, 90 s of audio written for this lab (synthetic, invented patient), spoken by speecht5 LISTEN AND TRANSCRIBE (openai/whisper-base.en, cuda) word error rate, clean audio : 2.1% critical terms heard : 18 of 20 missed: a1c, lisinopril ORGANISE (the note writer: qwen2.5:3b-instruct) drafted from : Whisper's transcript invented : none carried mishearing : none lucky fill-in : lisinopril small talk that leaked : baseball

Listen to the visit

Synthetic audio made from a written script by speecht5. No real patient, no real recording.

The draft note stayed inside what was said

The draft, exactly as the model wrote it from Whisper's transcript. Highlighted: numbers and clinical terms the visit never contained. It is unsigned.

SUBJECTIVE - feels tired all the time - feels shaky - has a blood sugar of 62 - has a blood pressure of 148/92 - IAC of 8.1 - has reduced sensation in both feet - has a monofilament test - has a rash to sulfa drugs - has been on her feet a lot due to grandson's baseball team win OBJECTIVE - blood pressure 148 over 92 - IAC 8.1 - feet with reduced sensation - normal foot pulses ASSESSMENT - Diabetic neuropathy - Blood pressure not at goal - Hyperglycemia (62) - Sulfa drug allergy PLAN - Increase Lisinopril to 20 mg daily - Stop Glipizide - Order kidney function panel and urine albumin test - Refer to Podiatry - Check blood sugar every morning and record anything under 70 - Follow-up in 4 weeks

The sign-off checklist

Automated heuristics on words. The real sign-off is a clinician reading the note.

check result how_sure chest pain recorded as ABSENT not mentioned heuristic glipizide recorded as STOPPED ok heuristic sulfa allergy recorded ok exact word clinic measurements under OBJECTIVE, not SUBJECTIVE MIS-FILED: 148, 92, 8.1 under Subjective exact number lisinopril 20 mg in the PLAN ok exact number small talk left out LEAKED: baseball exact word follow-up in 4 weeks ok exact words

What background noise does to the ear

White noise added at each signal-to-noise ratio; two sizes of Whisper. Critical terms are the words whose mishearing changes care.

model noise wer subs dels ins critical_missed missed_terms seconds whisper-base.en clean 2.1% 5 0 0 2 a1c, lisinopril 0.0 whisper-base.en 20 dB 2.1% 4 0 1 2 a1c, lisinopril 3.8 whisper-base.en 10 dB 5.1% 8 0 4 4 a1c, monofilament, lisinopril, podiatry 3.8 whisper-base.en 5 dB 8.9% 14 1 6 7 metformin, glipizide, sulfa, a1c, monofilament, lisinopril, podiatry 3.8 whisper-base.en 0 dB 17.3% 25 1 15 10 metformin, glipizide, 62, sulfa, a1c, monofilament, lisinopril, kidney, albumin, podiatry 3.8 whisper-small.en clean 3.0% 5 0 2 2 a1c, lisinopril 11.7 whisper-small.en 20 dB 1.7% 4 0 0 1 a1c 11.8 whisper-small.en 10 dB 3.0% 6 0 1 3 glipizide, a1c, lisinopril 12.1 whisper-small.en 5 dB 3.4% 7 0 1 3 glipizide, a1c, lisinopril 11.8 whisper-small.en 0 dB 11.0% 17 1 8 6 metformin, glipizide, sulfa, a1c, lisinopril, podiatry 11.5

The note, re-drafted from what was heard at 5 dB

SUBJECTIVE - feels tired all the time - feels shaky - has tingling in feet for 3 weeks - stopped Metformin due to shakiness - last Metformin dose 500 mg twice daily - ISG 8.1 - blood pressure 148/92 - reduced sensation in both feet - normal foot pulses - congratulations on grandson's baseball team win - has allergy to rash from drugs OBJECTIVE - blood pressure 148/92 - ISG 8.1 - reduced sensation in both feet - normal foot pulses ASSESSMENT - Diabetic neuropathy in feet - Uncontrolled blood pressure - ISG 8.1 - Normal foot pulses PLAN - Increase Lisinopril to 20 mg daily - Continue monitoring blood sugar - Order kidney function panel and urine albumin test - Refer to Prodigy - Follow up in 4 weeks invented: none (in neither the transcript nor the visit: written from habit) carried mishearing: none (heard but never said: Whisper's mistake, copied faithfully) lucky fill-in: metformin, lisinopril (said but never heard: a right guess with nothing to go on)

How this was measured

A 21-turn clinic visit was written for this lab: an invented patient modelled on the course's representative patient (68, type 2 diabetes, hypertension, chronic kidney disease). A local text-to-speech model (speecht5) spoke each turn; Whisper (openai/whisper-base.en) transcribed each turn separately, so who said what is known rather than inferred. Word error rate is (substitutions + deletions + insertions) / words said, after both sides are normalised (lower case, no punctuation, number words as digits). A local language model (qwen2.5:3b-instruct, temperature 0) drafted a SOAP note in JSON with instructions to use only what was said. A fact is unsupported when a number or a listed clinical term in the note does not appear in the transcript. Noise is white noise at a fixed seed. Everything ran on free Google Colab hardware at $0. This is a teaching pilot on one synthetic visit, not a clinical evaluation, and no note here was, or should be, signed.

Credits

Speech synthesis: SpeechT5 and its HiFi-GAN vocoder by Microsoft (MIT), with CMU ARCTIC voice prints (MIT), or espeak-ng. Speech recognition: OpenAI Whisper (open weights). Note drafting: qwen2.5:3b-instruct via ollama. See each model card for its licence.