MedTalk AI Logo

AI Medical Scribe Evaluation · 2026

Clinical Documentation Accuracy Report

Comparative assessment of MedTalk AI, Heidi, and Lyrebird across 5 clinical domains and 12 standardised evaluation criteria.

Evaluation conducted using DeepSeek AI with independent clinical safety assessment per domain. Scores reflect documentation accuracy on real-world clinical scenarios spanning GP, specialist, and allied health.

How this evaluation was conducted

Standardised framework designed to surface clinically meaningful differences between platforms across five high-risk documentation domains.

5 Real-World Clinical Scenarios

Scenarios cover high-risk documentation domains across GP, specialist, and allied health — acute asthma, diabetes annual review, migraine with aura, knee injury, and lumbar disc herniation.

12 Standardised Evaluation Criteria

Each document scored against criteria including hallucination safety, drug name accuracy, Australian English compliance, completeness, STT error correction, safety netting, and medico-legal appropriateness.

DeepSeek AI Evaluation Engine

Scoring conducted using DeepSeek AI against each criterion independently. Clinical safety assessment performed separately for each report, not derived from the aggregate score.

Independent Safety Classification

Each tool received a safety classification per domain: Safe for Clinical Use, Use With Caution, or Not Safe for Clinical Use — based on the presence and severity of clinically significant errors.

MedTalk AI overall score

92%

Recommended for clinical use

Heidi overall score

80%

Use with caution

Lyrebird overall score

79%

Use with caution

Full scoring matrix

All three platforms scored across five clinical documentation domains. MedTalk averaged 92%, Heidi 80%, Lyrebird 79%.

Clinical DomainMedTalk AIHeidiLyrebirdMedTalk AI advantage
SOAP Note — Acute Asthma Exacerbation94%79%82%+12pp advantage
GP Consult — Type 2 Diabetes Annual Review91%81%78%+10pp advantage
Neurology Consult — Migraine with Aura93%80%79%+13pp advantage
Orthopaedic Consult — Medial Meniscal Tear89%77%80%+9pp advantage
Physiotherapy Notes — Lumbar Disc Herniation92%83%76%+9pp advantage
Overall Average92%80%79%+12pp advantage

Domain breakdown — bar view

SOAP Note — Acute Asthma Exacerbation

MedTalk AI94%
Heidi79%
Lyrebird82%

GP Consult — Type 2 Diabetes Annual Review

MedTalk AI91%
Heidi81%
Lyrebird78%

Neurology Consult — Migraine with Aura

MedTalk AI93%
Heidi80%
Lyrebird79%

Orthopaedic Consult — Medial Meniscal Tear

MedTalk AI89%
Heidi77%
Lyrebird80%

Physiotherapy Notes — Lumbar Disc Herniation

MedTalk AI92%
Heidi83%
Lyrebird76%

Domain-by-domain evaluation

12-criteria scoring with clinical safety assessment and tool-by-tool notes for each clinical domain.

SOAP Note

Respiratory — Acute Asthma Exacerbation

94%

MedTalk

79%

Heidi

82%

Lyrebird

MedTalk AISafe for clinical use

Full PEFR readings, stepwise management plan, and discharge criteria captured accurately. Minor conciseness deduction only.

HeidiUse with caution

Spacer device instructions and prior PEFR baseline not recorded in this evaluation. Scored under completeness criterion.

LyrebirdSafe for clinical use

Core clinical content present. Subheadings and medication lists repeated across sections, resulting in a lower conciseness score.

#CriterionMedTalk AIHeidiLyrebird
01No Hallucinations100%100%100%
02Exact Drug Names100%100%100%
03Generic Lowercase Names100%80%100%
04Australian Spelling100%100%100%
05Assessment Accuracy100%100%90%
06No Extra Differentials100%100%100%
07Completeness100%50%100%
08STT Error Correction100%100%100%
09Safety Netting100%100%100%
10Conciseness80%90%50%
11Format Compliance100%100%100%
12Severity Appropriateness100%90%90%

GP Consult Note

Endocrinology — Type 2 Diabetes Annual Review

91%

MedTalk

81%

Heidi

78%

Lyrebird

MedTalk AISafe for clinical use

HbA1c targets, foot examination findings, and medication titration plan captured with high precision. Minor STT correction delay on insulin dose noted.

HeidiSafe for clinical use

Patient self-monitoring frequency not documented in this evaluation. All other clinical content present and well-structured.

LyrebirdUse with caution

Transcription produced metformin 500 mg twice daily against a dictated dose of 1000 mg twice daily. Scored under STT error correction criterion.

#CriterionMedTalk AIHeidiLyrebird
01No Hallucinations100%100%100%
02Exact Drug Names100%100%100%
03Generic Lowercase Names100%100%100%
04Australian Spelling100%100%100%
05Assessment Accuracy100%100%100%
06No Extra Differentials100%100%100%
07Completeness100%70%100%
08STT Error Correction80%100%20%
09Safety Netting100%100%100%
10Conciseness90%90%40%
11Format Compliance100%100%100%
12Severity Appropriateness100%90%80%

Neurology Consult Letter

Migraine with Aura — Recurrent Episodes

93%

MedTalk

80%

Heidi

79%

Lyrebird

MedTalk AISafe for clinical use

Aura characteristics, trigger identification, and prophylactic management plan documented completely. Correctly omitted unconfirmed diagnoses. Minor inference noted under criterion 6.

HeidiSafe for clinical use

Neurological content accurate and well-structured. Oral contraceptive use not documented in this evaluation; scored under completeness criterion.

LyrebirdUse with caution

Possible tension-type headache included as a secondary consideration; not raised during the consultation. Scored under criterion 6. Higher verbosity across sections.

#CriterionMedTalk AIHeidiLyrebird
01No Hallucinations100%100%100%
02Exact Drug Names100%100%100%
03Generic Lowercase Names100%100%100%
04Australian Spelling100%100%100%
05Assessment Accuracy100%100%100%
06No Extra Differentials80%100%50%
07Completeness100%70%100%
08STT Error Correction100%100%100%
09Safety Netting100%100%80%
10Conciseness90%90%40%
11Format Compliance100%100%100%
12Severity Appropriateness100%80%90%

Orthopaedic Consultation

Knee Pain — Medial Meniscal Tear Assessment

89%

MedTalk

77%

Heidi

80%

Lyrebird

MedTalk AISafe for clinical use

McMurray and Thessaly test results captured correctly. Minor deduction for incomplete past surgical history documentation.

HeidiNot safe for clinical use

Diclofenac transcribed as diltiazem in this evaluation. Scored under exact drug names and STT error correction criteria.

LyrebirdNot safe for clinical use

Diclofenac transcribed as diltiazem in this evaluation. ACL injury included as a differential; not raised during the consultation. Scored under criteria 2, 6, and 8.

#CriterionMedTalk AIHeidiLyrebird
01No Hallucinations100%100%100%
02Exact Drug Names80%0%0%
03Generic Lowercase Names100%100%100%
04Australian Spelling100%100%100%
05Assessment Accuracy100%100%90%
06No Extra Differentials100%100%50%
07Completeness80%80%80%
08STT Error Correction70%0%0%
09Safety Netting90%0%80%
10Conciseness90%90%90%
11Format Compliance100%100%100%
12Severity Appropriateness80%80%80%

Physiotherapy Notes

Lower Back Pain — Lumbar Disc Herniation

92%

MedTalk

83%

Heidi

76%

Lyrebird

MedTalk AISafe for clinical use

Dermatomal findings, SLR test results, and functional goals documented thoroughly. Rehabilitation progression plan clearly captured. Minor conciseness note.

HeidiSafe for clinical use

Clinical content accurate. Non-standard abbreviation LDH used without expansion. Minor formatting note; no clinical content concerns.

LyrebirdSafe for clinical use

Clinical content accurate. Section headers and treatment goal statements repeated across the note, resulting in a lower conciseness score.

#CriterionMedTalk AIHeidiLyrebird
01No Hallucinations100%100%100%
02Exact Drug Names100%100%100%
03Generic Lowercase Names100%80%100%
04Australian Spelling100%100%100%
05Assessment Accuracy100%100%100%
06No Extra Differentials100%100%100%
07Completeness100%100%100%
08STT Error Correction100%100%100%
09Safety Netting100%100%100%
10Conciseness90%90%40%
11Format Compliance100%100%100%
12Severity Appropriateness100%90%80%

Clinical safety classification by domain

Clinical DomainMedTalk AIHeidiLyrebird
SOAP NoteRespiratory — Acute Asthma ExacerbationSafe for clinical useUse with cautionSafe for clinical use
GP Consult NoteEndocrinology — Type 2 Diabetes Annual ReviewSafe for clinical useSafe for clinical useUse with caution
Neurology Consult LetterMigraine with Aura — Recurrent EpisodesSafe for clinical useSafe for clinical useUse with caution
Orthopaedic ConsultationKnee Pain — Medial Meniscal Tear AssessmentSafe for clinical useNot safe for clinical useNot safe for clinical use
Physiotherapy NotesLower Back Pain — Lumbar Disc HerniationSafe for clinical useSafe for clinical useSafe for clinical use

Recommendations for clinical deployment

Based on aggregate performance and safety classification across all five domains.

MedTalk AI

92%

Recommended

Highest scores across all five domains (89–94%). Strong performance on drug nomenclature, safety netting, and STT error correction. Minor deductions on conciseness and one instance of minor clinical inference.

Heidi

80%

Use with caution

Scores ranged 77–83% across five domains. A medication transcription error (diclofenac/diltiazem) was recorded in the orthopaedic scenario, affecting scores under criteria 2 and 8. Two completeness deductions noted across the diabetes and neurology scenarios.

Lyrebird

79%

Use with caution

Scores ranged 76–82% across five domains. The same diclofenac/diltiazem transcription error was recorded in the orthopaedic scenario. A dose discrepancy on metformin was noted in the diabetes review. Conciseness scores were lower across all five domains.

Domain-by-domain performance

MedTalk AIHeidiLyrebird

SOAP Note — Acute Asthma Exacerbation

GP Consult — Type 2 Diabetes Annual Review

Neurology Consult — Migraine with Aura

Orthopaedic Consult — Medial Meniscal Tear

Physiotherapy Notes — Lumbar Disc Herniation