5 Real-World Clinical Scenarios
Scenarios cover high-risk documentation domains across GP, specialist, and allied health — acute asthma, diabetes annual review, migraine with aura, knee injury, and lumbar disc herniation.
AI Medical Scribe Evaluation · 2026
Comparative assessment of MedTalk AI, Heidi, and Lyrebird across 5 clinical domains and 12 standardised evaluation criteria.
Evaluation conducted using DeepSeek AI with independent clinical safety assessment per domain. Scores reflect documentation accuracy on real-world clinical scenarios spanning GP, specialist, and allied health.
Standardised framework designed to surface clinically meaningful differences between platforms across five high-risk documentation domains.
Scenarios cover high-risk documentation domains across GP, specialist, and allied health — acute asthma, diabetes annual review, migraine with aura, knee injury, and lumbar disc herniation.
Each document scored against criteria including hallucination safety, drug name accuracy, Australian English compliance, completeness, STT error correction, safety netting, and medico-legal appropriateness.
Scoring conducted using DeepSeek AI against each criterion independently. Clinical safety assessment performed separately for each report, not derived from the aggregate score.
Each tool received a safety classification per domain: Safe for Clinical Use, Use With Caution, or Not Safe for Clinical Use — based on the presence and severity of clinically significant errors.
MedTalk AI overall score
92%
Recommended for clinical use
Heidi overall score
80%
Use with caution
Lyrebird overall score
79%
Use with caution
All three platforms scored across five clinical documentation domains. MedTalk averaged 92%, Heidi 80%, Lyrebird 79%.
| Clinical Domain | MedTalk AI | Heidi | Lyrebird | MedTalk AI advantage |
|---|---|---|---|---|
| SOAP Note — Acute Asthma Exacerbation | 94% | 79% | 82% | +12pp advantage |
| GP Consult — Type 2 Diabetes Annual Review | 91% | 81% | 78% | +10pp advantage |
| Neurology Consult — Migraine with Aura | 93% | 80% | 79% | +13pp advantage |
| Orthopaedic Consult — Medial Meniscal Tear | 89% | 77% | 80% | +9pp advantage |
| Physiotherapy Notes — Lumbar Disc Herniation | 92% | 83% | 76% | +9pp advantage |
| Overall Average | 92% | 80% | 79% | +12pp advantage |
12-criteria scoring with clinical safety assessment and tool-by-tool notes for each clinical domain.
Respiratory — Acute Asthma Exacerbation
94%
MedTalk
79%
Heidi
82%
Lyrebird
Full PEFR readings, stepwise management plan, and discharge criteria captured accurately. Minor conciseness deduction only.
Spacer device instructions and prior PEFR baseline not recorded in this evaluation. Scored under completeness criterion.
Core clinical content present. Subheadings and medication lists repeated across sections, resulting in a lower conciseness score.
| # | Criterion | MedTalk AI | Heidi | Lyrebird |
|---|---|---|---|---|
| 01 | No Hallucinations | 100% | 100% | 100% |
| 02 | Exact Drug Names | 100% | 100% | 100% |
| 03 | Generic Lowercase Names | 100% | 80% | 100% |
| 04 | Australian Spelling | 100% | 100% | 100% |
| 05 | Assessment Accuracy | 100% | 100% | 90% |
| 06 | No Extra Differentials | 100% | 100% | 100% |
| 07 | Completeness | 100% | 50% | 100% |
| 08 | STT Error Correction | 100% | 100% | 100% |
| 09 | Safety Netting | 100% | 100% | 100% |
| 10 | Conciseness | 80% | 90% | 50% |
| 11 | Format Compliance | 100% | 100% | 100% |
| 12 | Severity Appropriateness | 100% | 90% | 90% |
Endocrinology — Type 2 Diabetes Annual Review
91%
MedTalk
81%
Heidi
78%
Lyrebird
HbA1c targets, foot examination findings, and medication titration plan captured with high precision. Minor STT correction delay on insulin dose noted.
Patient self-monitoring frequency not documented in this evaluation. All other clinical content present and well-structured.
Transcription produced metformin 500 mg twice daily against a dictated dose of 1000 mg twice daily. Scored under STT error correction criterion.
| # | Criterion | MedTalk AI | Heidi | Lyrebird |
|---|---|---|---|---|
| 01 | No Hallucinations | 100% | 100% | 100% |
| 02 | Exact Drug Names | 100% | 100% | 100% |
| 03 | Generic Lowercase Names | 100% | 100% | 100% |
| 04 | Australian Spelling | 100% | 100% | 100% |
| 05 | Assessment Accuracy | 100% | 100% | 100% |
| 06 | No Extra Differentials | 100% | 100% | 100% |
| 07 | Completeness | 100% | 70% | 100% |
| 08 | STT Error Correction | 80% | 100% | 20% |
| 09 | Safety Netting | 100% | 100% | 100% |
| 10 | Conciseness | 90% | 90% | 40% |
| 11 | Format Compliance | 100% | 100% | 100% |
| 12 | Severity Appropriateness | 100% | 90% | 80% |
Migraine with Aura — Recurrent Episodes
93%
MedTalk
80%
Heidi
79%
Lyrebird
Aura characteristics, trigger identification, and prophylactic management plan documented completely. Correctly omitted unconfirmed diagnoses. Minor inference noted under criterion 6.
Neurological content accurate and well-structured. Oral contraceptive use not documented in this evaluation; scored under completeness criterion.
Possible tension-type headache included as a secondary consideration; not raised during the consultation. Scored under criterion 6. Higher verbosity across sections.
| # | Criterion | MedTalk AI | Heidi | Lyrebird |
|---|---|---|---|---|
| 01 | No Hallucinations | 100% | 100% | 100% |
| 02 | Exact Drug Names | 100% | 100% | 100% |
| 03 | Generic Lowercase Names | 100% | 100% | 100% |
| 04 | Australian Spelling | 100% | 100% | 100% |
| 05 | Assessment Accuracy | 100% | 100% | 100% |
| 06 | No Extra Differentials | 80% | 100% | 50% |
| 07 | Completeness | 100% | 70% | 100% |
| 08 | STT Error Correction | 100% | 100% | 100% |
| 09 | Safety Netting | 100% | 100% | 80% |
| 10 | Conciseness | 90% | 90% | 40% |
| 11 | Format Compliance | 100% | 100% | 100% |
| 12 | Severity Appropriateness | 100% | 80% | 90% |
Knee Pain — Medial Meniscal Tear Assessment
89%
MedTalk
77%
Heidi
80%
Lyrebird
McMurray and Thessaly test results captured correctly. Minor deduction for incomplete past surgical history documentation.
Diclofenac transcribed as diltiazem in this evaluation. Scored under exact drug names and STT error correction criteria.
Diclofenac transcribed as diltiazem in this evaluation. ACL injury included as a differential; not raised during the consultation. Scored under criteria 2, 6, and 8.
| # | Criterion | MedTalk AI | Heidi | Lyrebird |
|---|---|---|---|---|
| 01 | No Hallucinations | 100% | 100% | 100% |
| 02 | Exact Drug Names | 80% | 0% | 0% |
| 03 | Generic Lowercase Names | 100% | 100% | 100% |
| 04 | Australian Spelling | 100% | 100% | 100% |
| 05 | Assessment Accuracy | 100% | 100% | 90% |
| 06 | No Extra Differentials | 100% | 100% | 50% |
| 07 | Completeness | 80% | 80% | 80% |
| 08 | STT Error Correction | 70% | 0% | 0% |
| 09 | Safety Netting | 90% | 0% | 80% |
| 10 | Conciseness | 90% | 90% | 90% |
| 11 | Format Compliance | 100% | 100% | 100% |
| 12 | Severity Appropriateness | 80% | 80% | 80% |
Lower Back Pain — Lumbar Disc Herniation
92%
MedTalk
83%
Heidi
76%
Lyrebird
Dermatomal findings, SLR test results, and functional goals documented thoroughly. Rehabilitation progression plan clearly captured. Minor conciseness note.
Clinical content accurate. Non-standard abbreviation LDH used without expansion. Minor formatting note; no clinical content concerns.
Clinical content accurate. Section headers and treatment goal statements repeated across the note, resulting in a lower conciseness score.
| # | Criterion | MedTalk AI | Heidi | Lyrebird |
|---|---|---|---|---|
| 01 | No Hallucinations | 100% | 100% | 100% |
| 02 | Exact Drug Names | 100% | 100% | 100% |
| 03 | Generic Lowercase Names | 100% | 80% | 100% |
| 04 | Australian Spelling | 100% | 100% | 100% |
| 05 | Assessment Accuracy | 100% | 100% | 100% |
| 06 | No Extra Differentials | 100% | 100% | 100% |
| 07 | Completeness | 100% | 100% | 100% |
| 08 | STT Error Correction | 100% | 100% | 100% |
| 09 | Safety Netting | 100% | 100% | 100% |
| 10 | Conciseness | 90% | 90% | 40% |
| 11 | Format Compliance | 100% | 100% | 100% |
| 12 | Severity Appropriateness | 100% | 90% | 80% |
| Clinical Domain | MedTalk AI | Heidi | Lyrebird |
|---|---|---|---|
| SOAP Note — Respiratory — Acute Asthma Exacerbation | Safe for clinical use | Use with caution | Safe for clinical use |
| GP Consult Note — Endocrinology — Type 2 Diabetes Annual Review | Safe for clinical use | Safe for clinical use | Use with caution |
| Neurology Consult Letter — Migraine with Aura — Recurrent Episodes | Safe for clinical use | Safe for clinical use | Use with caution |
| Orthopaedic Consultation — Knee Pain — Medial Meniscal Tear Assessment | Safe for clinical use | Not safe for clinical use | Not safe for clinical use |
| Physiotherapy Notes — Lower Back Pain — Lumbar Disc Herniation | Safe for clinical use | Safe for clinical use | Safe for clinical use |
Based on aggregate performance and safety classification across all five domains.
92%
Recommended
Highest scores across all five domains (89–94%). Strong performance on drug nomenclature, safety netting, and STT error correction. Minor deductions on conciseness and one instance of minor clinical inference.
80%
Use with caution
Scores ranged 77–83% across five domains. A medication transcription error (diclofenac/diltiazem) was recorded in the orthopaedic scenario, affecting scores under criteria 2 and 8. Two completeness deductions noted across the diabetes and neurology scenarios.
79%
Use with caution
Scores ranged 76–82% across five domains. The same diclofenac/diltiazem transcription error was recorded in the orthopaedic scenario. A dose discrepancy on metformin was noted in the diabetes review. Conciseness scores were lower across all five domains.
SOAP Note — Acute Asthma Exacerbation
GP Consult — Type 2 Diabetes Annual Review
Neurology Consult — Migraine with Aura
Orthopaedic Consult — Medial Meniscal Tear
Physiotherapy Notes — Lumbar Disc Herniation