Press · Conferences & speaking
Drs. Bichey and Blythe Present Framework for Evaluating AI Tools at AAO-HNSF 2025 Annual Meeting
Private-practice otolaryngologists heard a physician-built framework for judging AI vendors at the specialty's largest national meeting.
October 12, 2025
INDIANAPOLIS — William R. Blythe, MD, FACS, and Bradford G. Bichey, MD, MPH, presented a framework for evaluating artificial intelligence products at the Otolaryngology Private Practice Section General Assembly during the AAO-HNSF 2025 Annual Meeting in Indianapolis.
The session addressed a practical question facing every practice being pitched AI: how do you tell whether a tool works? The framework the two physicians laid out drew on standard classification metrics, including F1 score and Cohen's kappa, applied against a known reference dataset rather than vendor demonstrations.
Dr. Bichey subsequently expanded the material for the Academy's private-practice community, where members asked how to run the same measurements against tools they were already trialing.
"If a vendor cannot tell you their error rate against a gold standard, they have not measured it," Dr. Bichey said.