To move forward as a field, clinicians need to design benchmarks that better align with the tasks LLMs are expected to perform upon deployment. Read the editorial by Inioluwa Deborah Raji, BAS, @roxanadaneshjou.bsky.social, and @emilyalsentzer.bsky.social: nejm.ai/3PK8v1D #MedSky #AI #MLSky