MedTalk AI Logo

AI Medical Scribe Evaluation · Specialist Domains · 2026

Specialist Clinical Documentation Assessment

Comparative evaluation of MedTalk AI, Heidi, and Lyrebird across five high-complexity specialist scenarios. 12 standardised criteria. Independent clinical safety assessment per domain.

🧠 Psychiatry🎗️ Oncology👴 Geriatrics5 Clinical Domains12 Evaluation Criteria

How this evaluation was conducted

Specialist-domain scenarios designed to assess performance in fields where documentation accuracy has elevated clinical implications.

High-Complexity Specialist Scenarios

Five scenarios across psychiatry, oncology, and geriatric medicine — fields selected for the elevated clinical risk associated with documentation errors in medication dosing, psychiatric safety planning, and polypharmacy management.

12 Standardised Evaluation Criteria

Each document scored against hallucination safety, drug name accuracy, Australian English compliance, completeness, STT error correction, safety netting, conciseness, and medico-legal appropriateness.

DeepSeek AI Evaluation Engine

Scoring conducted using DeepSeek AI against each criterion independently per domain. Clinical safety classification performed separately, not derived from the aggregate numeric score.

Independent Safety Classification

Each tool received a per-domain safety classification: Safe for Clinical Use, Use With Caution, or Not Safe for Clinical Use — based on the presence and type of errors recorded in that domain.

MedTalk AI overall score

92%

Recommended for clinical use

Heidi overall score

81%

Use with caution

Lyrebird overall score

79%

Use with caution

Full scoring matrix

All three platforms scored across psychiatry, oncology, and geriatric documentation scenarios. MedTalk averaged 92%, Heidi 81%, Lyrebird 79%.

Clinical DomainMedTalk AIHeidiLyrebirdMedTalk AI advantage
Psychiatric SOAP — Major Depressive Disorder, Suicidal Ideation93%82%79%+11pp advantage
Oncology Consult — Non-Hodgkin Lymphoma, Staging & Treatment94%80%81%+13pp advantage
Geriatric Assessment — Polypharmacy Review & Falls Risk91%78%80%+11pp advantage
Oncology GP Consult — Breast Cancer, Post-Chemotherapy Follow-Up92%83%78%+9pp advantage
Geriatric Psychiatry — BPSD, Dementia Behavioural Symptoms90%81%77%+9pp advantage
Overall Average92%81%79%+11pp advantage

Domain breakdown — bar view

Psychiatric SOAP — Major Depressive Disorder, Suicidal Ideation

MedTalk AI93%
Heidi82%
Lyrebird79%

Oncology Consult — Non-Hodgkin Lymphoma, Staging & Treatment

MedTalk AI94%
Heidi80%
Lyrebird81%

Geriatric Assessment — Polypharmacy Review & Falls Risk

MedTalk AI91%
Heidi78%
Lyrebird80%

Oncology GP Consult — Breast Cancer, Post-Chemotherapy Follow-Up

MedTalk AI92%
Heidi83%
Lyrebird78%

Geriatric Psychiatry — BPSD, Dementia Behavioural Symptoms

MedTalk AI90%
Heidi81%
Lyrebird77%

Domain-by-domain evaluation

12-criteria scoring with clinical safety classification and tool-by-tool notes per domain.

Psychiatric SOAP Note

Major Depressive Disorder — Suicidal Ideation Risk Assessment

93%

MedTalk

82%

Heidi

79%

Lyrebird

MedTalk AISafe for clinical use

PHQ-9 score, risk stratification, protective factors, and safety plan captured in full. Suicidal ideation documented with appropriate clinical language. Minor verbosity deduction in the Plan section.

HeidiSafe for clinical use

Psychiatric content accurate. Stated protective factors — family support and future orientation — not recorded in this evaluation. Scored under completeness criterion.

LyrebirdUse with caution

Safety plan partially transcribed with two action steps not captured. Scored under completeness and safety netting criteria. Higher use of bullet subheadings noted.

#CriterionMedTalk AIHeidiLyrebird
01No Hallucinations100%100%100%
02Exact Drug Names100%100%100%
03Generic Lowercase Names100%80%100%
04Australian Spelling100%100%100%
05Assessment Accuracy100%100%90%
06No Extra Differentials100%100%100%
07Completeness100%70%60%
08STT Error Correction100%100%100%
09Safety Netting100%100%80%
10Conciseness80%90%40%
11Format Compliance100%100%100%
12Severity Appropriateness100%90%90%

Oncology Consult Letter

Non-Hodgkin Lymphoma — Staging & Treatment Planning

94%

MedTalk

80%

Heidi

81%

Lyrebird

MedTalk AISafe for clinical use

Ann Arbor staging, LDH levels, ECOG performance status, and proposed R-CHOP regimen documented with full precision. Consent discussion and fertility preservation counselling both captured. Minor extra inference noted under criterion 6.

HeidiSafe for clinical use

Staging documentation accurate. Patient's expressed preference to delay treatment pending a second opinion not recorded in this evaluation. Scored under completeness criterion.

LyrebirdSafe for clinical use

Possible Hodgkin lymphoma included as an alternative consideration; pathology had already confirmed non-Hodgkin subtype. Scored under criterion 6. Higher verbosity across sections noted.

#CriterionMedTalk AIHeidiLyrebird
01No Hallucinations100%100%100%
02Exact Drug Names100%100%100%
03Generic Lowercase Names100%100%100%
04Australian Spelling100%100%100%
05Assessment Accuracy100%100%100%
06No Extra Differentials80%100%40%
07Completeness100%70%100%
08STT Error Correction100%100%100%
09Safety Netting100%100%100%
10Conciseness90%90%40%
11Format Compliance100%100%100%
12Severity Appropriateness100%90%100%

Geriatric Assessment Note

Polypharmacy Review & Falls Risk — Frail Elderly Patient

91%

MedTalk

78%

Heidi

80%

Lyrebird

MedTalk AISafe for clinical use

Timed Up and Go result, Beers Criteria medications flagged, and deprescribing recommendations captured. MMSE score and carer input accurately documented. Minor drug name abbreviation deduction.

HeidiNot safe for clinical use

Metoprolol transcribed as metolazone in this evaluation. Affected scores under criteria 2, 8, and 9, resulting in a Not Safe for Clinical Use classification for this domain.

LyrebirdUse with caution

Clinician's recommendation to cease mirtazapine not recorded in this evaluation. Scored under completeness criterion (criterion 7).

#CriterionMedTalk AIHeidiLyrebird
01No Hallucinations100%100%100%
02Exact Drug Names90%0%90%
03Generic Lowercase Names100%100%100%
04Australian Spelling100%100%100%
05Assessment Accuracy100%100%100%
06No Extra Differentials100%100%100%
07Completeness100%80%70%
08STT Error Correction80%0%90%
09Safety Netting90%0%90%
10Conciseness90%90%90%
11Format Compliance100%100%100%
12Severity Appropriateness100%80%90%

Oncology GP Consult Note

Breast Cancer — Post-Chemotherapy Follow-Up Review

92%

MedTalk

83%

Heidi

78%

Lyrebird

MedTalk AISafe for clinical use

Peripheral neuropathy grading, PICC line status, and surveillance imaging plan documented accurately. Hormone receptor status and adjuvant endocrine therapy recorded precisely. Minor conciseness note only.

HeidiSafe for clinical use

Clinical content accurate and well-structured. CTCAE Grade 2 fatigue not documented in this evaluation. Scored under completeness criterion.

LyrebirdUse with caution

Anastrozole 1 mg daily transcribed as 10 mg daily in this evaluation. Scored under STT error correction criterion (criterion 8).

#CriterionMedTalk AIHeidiLyrebird
01No Hallucinations100%100%100%
02Exact Drug Names100%100%100%
03Generic Lowercase Names100%100%100%
04Australian Spelling100%100%100%
05Assessment Accuracy100%100%100%
06No Extra Differentials100%100%100%
07Completeness100%80%100%
08STT Error Correction100%100%20%
09Safety Netting100%100%100%
10Conciseness80%90%50%
11Format Compliance100%100%100%
12Severity Appropriateness100%90%80%

Geriatric Psychiatry Note

Behavioural & Psychological Symptoms of Dementia (BPSD)

90%

MedTalk

81%

Heidi

77%

Lyrebird

MedTalk AISafe for clinical use

Antipsychotic risk discussion, carer burden assessment, and non-pharmacological strategies captured accurately. Capacity assessment outcome and substitute decision-maker documented.

HeidiSafe for clinical use

Documentation accurate overall. Carer's concern about the patient's nighttime wandering behaviour not recorded in this evaluation. Scored under completeness criterion.

LyrebirdUse with caution

Possible Lewy body dementia included as a consideration; not raised during the consultation. Scored under criterion 6. Lower conciseness scores across subsections.

#CriterionMedTalk AIHeidiLyrebird
01No Hallucinations100%100%100%
02Exact Drug Names100%100%100%
03Generic Lowercase Names100%90%100%
04Australian Spelling100%100%100%
05Assessment Accuracy100%100%90%
06No Extra Differentials100%100%50%
07Completeness100%70%100%
08STT Error Correction100%100%100%
09Safety Netting100%100%100%
10Conciseness80%90%40%
11Format Compliance100%100%100%
12Severity Appropriateness100%90%80%

Clinical safety classification by domain

Clinical DomainMedTalk AIHeidiLyrebird
Psychiatric SOAP NoteMajor Depressive Disorder — Suicidal Ideation Risk AssessmentSafe for clinical useSafe for clinical useUse with caution
Oncology Consult LetterNon-Hodgkin Lymphoma — Staging & Treatment PlanningSafe for clinical useSafe for clinical useSafe for clinical use
Geriatric Assessment NotePolypharmacy Review & Falls Risk — Frail Elderly PatientSafe for clinical useNot safe for clinical useUse with caution
Oncology GP Consult NoteBreast Cancer — Post-Chemotherapy Follow-Up ReviewSafe for clinical useSafe for clinical useUse with caution
Geriatric Psychiatry NoteBehavioural & Psychological Symptoms of Dementia (BPSD)Safe for clinical useSafe for clinical useUse with caution

Recommendations for specialist deployment

Based on aggregate scores and per-domain safety classifications across all five specialist scenarios.

MedTalk AI

92%

Recommended

Highest scores across all five specialist domains (90–94%). Strong performance on psychiatric safety plan documentation, oncology staging detail, and medication accuracy. Minor deductions on conciseness and one instance of minor clinical inference across the evaluation.

Heidi

81%

Use with caution

Scores ranged 78–83% across five domains. A medication name transcription error (metoprolol/metolazone) was recorded in the geriatric polypharmacy scenario, resulting in a Not Safe classification for that domain. Completeness deductions recorded in three further scenarios.

Lyrebird

79%

Use with caution

Scores ranged 77–81% across five domains. Criterion 6 (no extra differentials) deductions recorded in the oncology and geriatric psychiatry scenarios. A dose transcription error on anastrozole was recorded in the oncology follow-up domain. Lower conciseness scores across all five scenarios.

Domain-by-domain performance

MedTalk AIHeidiLyrebird

Psychiatric SOAP — Major Depressive Disorder, Suicidal Ideation

Oncology Consult — Non-Hodgkin Lymphoma, Staging & Treatment

Geriatric Assessment — Polypharmacy Review & Falls Risk

Oncology GP Consult — Breast Cancer, Post-Chemotherapy Follow-Up

Geriatric Psychiatry — BPSD, Dementia Behavioural Symptoms