Advanced search


Search results        Search results      Copy URL to E-Mail


Evaluating the Accuracy and Educational Utility of ChatGPT in Osteopathic Manipulative Medicine (OMM) and Treatment (OMT): A Cross-Sectional Performance Study

Journal: Journal of Osteopathic Medicine Date: 2025/12, 125(12):Pages: A636–637. doi: Subito , type of study: cross sectional study

Full text    (https://www.degruyterbrill.com/document/doi/10.1515/jom-2025-2000/html)

Keywords:

AI [1922]
artifical intelligence [8]
ChatGPT [6]
cross sectional study [863]
exams [33]
OMT [3779]
osteopathic manipulative treatment [3799]
osteopathic medicine [2055]
USA [1707]

Abstract:

Concept: Osteopathic medical education includes instruction in osteopathic manipulative medicine (OMM) and osteopathic manipulative treatment (OMT), essential components of the COMLEX-USA Level 1 and Level 2 examinations. However, curricula for OMM vary widely across osteopathic medical schools, creating inconsistencies in student preparation. In response, many students turn to artificial intelligence (AI) tools like ChatGPT to supplement their learning. Although ChatGPT has shown utility in medical education more broadly, its accuracy and usefulness in OMM/OMT content remain largely unexplored. Objective: To evaluate the accuracy of ChatGPT in answering COMLEX-style OMM/OMT questions, to assess the educational utility of ChatGPT as a supplemental learning tool for OMM/OMT topics, and to examine the implications of using AI in a context where curricular standardization is lacking across DO programs. Methods: This study utilized a descriptive, cross-sectional design to evaluate the accuracy of ChatGPT in answering multiple-choice questions focused on osteopathic manipulative medicine (OMM) and treatment (OMT). A total of 50 questions were randomly selected from a larger pool within a subscription-based COMLEX-USA board preparation platform commonly used by osteopathic medical students. Questions covered a broad range of OMM/OMT domains, including viscerosomatic reflexes, Chapman’s points, counterstrain, high-velocity low-amplitude (HVLA) techniques, and osteopathic philosophy. Questions that required interpretation of figures or images were excluded. Each question was input into ChatGPT-4 using a standardized prompt: “Answer this COMLEX-style OMM question. Provide the correct answer choice and a brief explanation rooted in osteopathic principles.” Responses were recorded in a structured spreadsheet and compared against the original answer key provided by the question source. Accuracy was determined based on whether the selected answer matched the correct key. The percentage of correct and incorrect responses was calculated to quantify ChatGPT’s overall performance. Results: ChatGPT correctly answered 32 out of 50 (64%) OMM/OMT-focused multiple-choice questions. There was no consistent trend in the types of questions it answered correctly versus incorrectly. Topics varied widely across question categories (e.g., counterstrain, Chapman’s points, viscerosomatic reflexes), and no particular domain emerged as more prone to incorrect responses. When compared to aggregated student performance data available for each question, inconsistencies were observed. In some cases, ChatGPT answered a question incorrectly despite a majority of students selecting the correct answer. In other cases, ChatGPT answered correctly when the majority of students answered incorrectly. This variability suggests that ChatGPT’s accuracy does not consistently mirror common student performance patterns or reflect specific topic weaknesses. Explanations provided by ChatGPT varied in both depth and clinical relevance. While some responses demonstrated appropriate osteopathic reasoning, others not only lacked nuance but included inaccurate or misleading information regarding OMM-specific principles and techniques. Conclusion: This study demonstrates that ChatGPT correctly answered 64% of OMM/OMT-focused board-style questions, highlighting both its potential and its limitations as a supplemental learning tool in osteopathic medical education. While some explanations reflected sound osteopathic reasoning, others were vague, incomplete, or factually inaccurate. Notably, ChatGPT’s performance showed no consistent trends across question topics, and often differed from the response patterns of osteopathic medical students. In an educational landscape where OMM curricula vary widely between institutions, students are increasingly turning to AI platforms to fill learning gaps. However, this study suggests that ChatGPT, in its current form, cannot be reliably used as a standalone resource for mastering OMM concepts or preparing for COMLEX examinations. Future work should explore strategies to enhance the model’s alignment with osteopathic principles, such as tailored prompting, model fine-tuning, or the development of OMM-specific AI tools. Faculty involvement remains essential to guide students in responsibly integrating AI into their study routines, ensuring both accuracy and philosophical integrity in osteopathic training.


Search results      Copy URL to E-Mail

 
 
 






  • ImpressumLegal noticeDatenschutz


ostlib.de/data_wfyrcgpdhzuvkqmtjnxs



Supported by

OSTLIB recommends