AI Medical Scribe Evaluation · Specialist Domains · 2026
Specialist Clinical Documentation Assessment
Comparative evaluation of MedTalk AI, Heidi, and Lyrebird across five high-complexity specialist scenarios. 12 standardised criteria. Independent clinical safety assessment per domain.
Specialist-domain scenarios designed to assess performance in fields where documentation accuracy has elevated clinical implications.
High-Complexity Specialist Scenarios
Five scenarios across psychiatry, oncology, and geriatric medicine — fields selected for the elevated clinical risk associated with documentation errors in medication dosing, psychiatric safety planning, and polypharmacy management.
12 Standardised Evaluation Criteria
Each document scored against hallucination safety, drug name accuracy, Australian English compliance, completeness, STT error correction, safety netting, conciseness, and medico-legal appropriateness.
DeepSeek AI Evaluation Engine
Scoring conducted using DeepSeek AI against each criterion independently per domain. Clinical safety classification performed separately, not derived from the aggregate numeric score.
Independent Safety Classification
Each tool received a per-domain safety classification: Safe for Clinical Use, Use With Caution, or Not Safe for Clinical Use — based on the presence and type of errors recorded in that domain.
MedTalk AI overall score
92%
Recommended for clinical use
Heidi overall score
81%
Use with caution
Lyrebird overall score
79%
Use with caution
Full scoring matrix
All three platforms scored across psychiatry, oncology, and geriatric documentation scenarios. MedTalk averaged 92%, Heidi 81%, Lyrebird 79%.
Clinical Domain
MedTalk AI
Heidi
Lyrebird
MedTalk AI advantage
Psychiatric SOAP — Major Depressive Disorder, Suicidal Ideation
12-criteria scoring with clinical safety classification and tool-by-tool notes per domain.
🧠
Psychiatric SOAP Note
Major Depressive Disorder — Suicidal Ideation Risk Assessment
93%
MedTalk
82%
Heidi
79%
Lyrebird
MedTalk AISafe for clinical use
PHQ-9 score, risk stratification, protective factors, and safety plan captured in full. Suicidal ideation documented with appropriate clinical language. Minor verbosity deduction in the Plan section.
HeidiSafe for clinical use
Psychiatric content accurate. Stated protective factors — family support and future orientation — not recorded in this evaluation. Scored under completeness criterion.
LyrebirdUse with caution
Safety plan partially transcribed with two action steps not captured. Scored under completeness and safety netting criteria. Higher use of bullet subheadings noted.
Ann Arbor staging, LDH levels, ECOG performance status, and proposed R-CHOP regimen documented with full precision. Consent discussion and fertility preservation counselling both captured. Minor extra inference noted under criterion 6.
HeidiSafe for clinical use
Staging documentation accurate. Patient's expressed preference to delay treatment pending a second opinion not recorded in this evaluation. Scored under completeness criterion.
LyrebirdSafe for clinical use
Possible Hodgkin lymphoma included as an alternative consideration; pathology had already confirmed non-Hodgkin subtype. Scored under criterion 6. Higher verbosity across sections noted.
Timed Up and Go result, Beers Criteria medications flagged, and deprescribing recommendations captured. MMSE score and carer input accurately documented. Minor drug name abbreviation deduction.
HeidiNot safe for clinical use
Metoprolol transcribed as metolazone in this evaluation. Affected scores under criteria 2, 8, and 9, resulting in a Not Safe for Clinical Use classification for this domain.
LyrebirdUse with caution
Clinician's recommendation to cease mirtazapine not recorded in this evaluation. Scored under completeness criterion (criterion 7).
#
Criterion
MedTalk AI
Heidi
Lyrebird
01
No Hallucinations
100%
100%
100%
02
Exact Drug Names
90%
0%
90%
03
Generic Lowercase Names
100%
100%
100%
04
Australian Spelling
100%
100%
100%
05
Assessment Accuracy
100%
100%
100%
06
No Extra Differentials
100%
100%
100%
07
Completeness
100%
80%
70%
08
STT Error Correction
80%
0%
90%
09
Safety Netting
90%
0%
90%
10
Conciseness
90%
90%
90%
11
Format Compliance
100%
100%
100%
12
Severity Appropriateness
100%
80%
90%
🩺
Oncology GP Consult Note
Breast Cancer — Post-Chemotherapy Follow-Up Review
92%
MedTalk
83%
Heidi
78%
Lyrebird
MedTalk AISafe for clinical use
Peripheral neuropathy grading, PICC line status, and surveillance imaging plan documented accurately. Hormone receptor status and adjuvant endocrine therapy recorded precisely. Minor conciseness note only.
HeidiSafe for clinical use
Clinical content accurate and well-structured. CTCAE Grade 2 fatigue not documented in this evaluation. Scored under completeness criterion.
LyrebirdUse with caution
Anastrozole 1 mg daily transcribed as 10 mg daily in this evaluation. Scored under STT error correction criterion (criterion 8).
#
Criterion
MedTalk AI
Heidi
Lyrebird
01
No Hallucinations
100%
100%
100%
02
Exact Drug Names
100%
100%
100%
03
Generic Lowercase Names
100%
100%
100%
04
Australian Spelling
100%
100%
100%
05
Assessment Accuracy
100%
100%
100%
06
No Extra Differentials
100%
100%
100%
07
Completeness
100%
80%
100%
08
STT Error Correction
100%
100%
20%
09
Safety Netting
100%
100%
100%
10
Conciseness
80%
90%
50%
11
Format Compliance
100%
100%
100%
12
Severity Appropriateness
100%
90%
80%
🏥
Geriatric Psychiatry Note
Behavioural & Psychological Symptoms of Dementia (BPSD)
90%
MedTalk
81%
Heidi
77%
Lyrebird
MedTalk AISafe for clinical use
Antipsychotic risk discussion, carer burden assessment, and non-pharmacological strategies captured accurately. Capacity assessment outcome and substitute decision-maker documented.
HeidiSafe for clinical use
Documentation accurate overall. Carer's concern about the patient's nighttime wandering behaviour not recorded in this evaluation. Scored under completeness criterion.
LyrebirdUse with caution
Possible Lewy body dementia included as a consideration; not raised during the consultation. Scored under criterion 6. Lower conciseness scores across subsections.
Based on aggregate scores and per-domain safety classifications across all five specialist scenarios.
MedTalk AI
92%
Recommended
Highest scores across all five specialist domains (90–94%). Strong performance on psychiatric safety plan documentation, oncology staging detail, and medication accuracy. Minor deductions on conciseness and one instance of minor clinical inference across the evaluation.
Heidi
81%
Use with caution
Scores ranged 78–83% across five domains. A medication name transcription error (metoprolol/metolazone) was recorded in the geriatric polypharmacy scenario, resulting in a Not Safe classification for that domain. Completeness deductions recorded in three further scenarios.
Lyrebird
79%
Use with caution
Scores ranged 77–81% across five domains. Criterion 6 (no extra differentials) deductions recorded in the oncology and geriatric psychiatry scenarios. A dose transcription error on anastrozole was recorded in the oncology follow-up domain. Lower conciseness scores across all five scenarios.
Domain-by-domain performance
MedTalk AIHeidiLyrebird
Psychiatric SOAP — Major Depressive Disorder, Suicidal Ideation