Comparative Analysis of NCLEX-RN Questions

A Duel Between ChatGPT and Human Expertise

Cox, Rachel L. DNP, FNP-BC
Hunt, Karen L. MSN, RN, RD, CNE
Hill, Rebecca R. PhD, DNP, CNE

Rachel L. Cox, DNP, FNP-BC, is an Assistant Professor. Karen L. Hunt, MSN, RN, RD, CNE, is an Instructor. Rebecca R. Hill, PhD, DNP, CNE, is an Associate Professor. All contributors are affiliated with the MGH Institute of Health Professions School of Nursing.

Address correspondence to Rachel L. Cox, DNP, FNP-BC, MGH Institute of Health Professions School of Nursing, 36 First Avenue, Boston, MA 02129; email: [email protected].

Disclosure: The authors have disclosed no potential conflicts of interest, financial or otherwise.

Received June 05, 2023

Accepted August 14, 2023

Journal of Nursing Education 62(12):p 679-687, December 01, 2023. | DOI: 10.3928/01484834-20231006-07

Background:

Artificial intelligence (AI) has the potential to revolutionize nursing education. This study compared NCLEX-RN questions generated by AI and those created by nurse educators.

Method:

Faculty of accredited baccalaureate programs were invited to participate. Likert-scale items for grammar and clarity of the item stem and distractors were compared using Mann-Whitney U, and yes/no questions about clinical relevance and complex terminology were analyzed using chi-square. A one-sample binomial test with confidence intervals evaluated participants' question preference (AI-generated or educator-written). Qualitative responses identified themes across faculty.

Results:

Item clarity, grammar, and difficulty were similar for AI and educator-created questions. Clinical relevance and use of complex terminology was similar for all question pairs. Of the four sets with preference for one item, three were generated by AI.

Conclusion:

AI can assist faculty with item generation to prepare nursing students for the NCLEX-RN examination. Faculty expertise is necessary to refine questions written using both methods. [J Nurs Educ. 2023;62(12):679–687.]