Introducing MCQ Maker: an evidence-based genAI tool to create SBAs
MCQ Maker, now running OpenAI GPT-5, can boost your question banks while guaranteeing exam security
When we first reported that GPT-4 could replenish depleted single best answer banks without sacrificing psychometric quality1, the reception was warm and the download count encouraging. Our peer-reviewed study in BMC Medical Education showed that 152 of 220 GPT-4 generated questions survived expert review, enabling two balanced formative exams that performed on par with human-written items.
This work has underpinned MCQ Maker, our Custom GPT that turns any intended learning outcome into an SBA question:
Since the manuscript went to press the engine behind MCQ Maker has moved from GPT-4 to GPT-5, which contains a much more advanced reasoning model. Internal audits now find in excess of 80% of generated items need only minor copy edits, and latency has fallen enough to draft questions live during blueprinting meetings. In short, you get higher-quality SBAs faster.
You can coax even better output by feeding the model more than the ILOs. Colleagues report that pasting short lecture extracts, illustrative cases or guideline snippets alongside the outcomes helps the model calibrate distractor plausibility and clinical context to your cohort.
🔒 Data protection remains intact. I have opted out of data retention and training, so your prompts are processed by OpenAI solely to generate a response and are not retained by OpenAI for model training. This safeguard keeps your draft exams private while still granting full access to the GPT-5 model.
⚠️ Remember: AI drafts still require human vetting. Each item must pass the usual quality-assurance checkpoints – blueprint alignment, factual accuracy, standard-setting and copy-edit – before students see it. Treat the output as you would a junior colleague’s first attempt; impressive, but not yet final.
Happy question writing, and do share your experiences – your feedback guides the next round of improvements.
Ahmed, A., Kerr, E. & O’Malley, A. Quality assurance and validity of AI-generated single best answer questions. BMC Med Educ 25, 300 (2025). https://doi.org/10.1186/s12909-025-06881-w