Common Mistakes to Watch for with Multiple-Choice Questions

Multiple choice questions (MCQ) are the mainstay for assessing higher cognitive thinking and are commonly used in medical profession programs, certification, and board exams (NBME, 2021). Writing good multiple-choice questions (MCQ) takes time and practice. It is a knowledge base and skill all faculty should have because whether you write the question or are reviewing questions to use for an exam, it is important to be able to recognize common mistakes. The savvy exam-taking student looks for these common flaws and uses them to determine the answer not from a knowledge base but rather from their ability to spot unintended cues.  

A quick review of the structure of a single best answer MCQ:

It can be a vignette with a lead in question, an open-ended statement, or a question – referred to as the stem.

Stem as a vignette with lead in question:

A 25-year-old male presents to the ER with a stab wound to the mid-chest. 

On examination, blood pressure is 82/40, heart rate is 120, heart tones are difficult to auscultate, distended neck veins are noted, and pulsus paradoxus is present. Breath sounds are clear bilaterally. 

What is the most appropriate management of this patient? 

Stem as a question:       

In a patient presenting with jaundice, which of the following will ascertain your

diagnosis in the majority of patients? 

Stem as an open-ended statement:

The best treatment modality for a patient diagnosed with bulimia nervosa is 

Choices/Options

There are usually 4 or 5 choices following the stem. These are referred to as distractors and the correct answer. 

Assessment aims to determine whether students have learned what they needed to as defined by learning outcomes and instructional objectives. Thus, the quality of the question is essential. Here are seven common mistakes to look for and avoid. 

1. Use of absolute terms in choices (distractors)

Words like always, never, only, all, etc., are considered absolute terms. This is a quick one for students to look for and then use to eliminate those choices as viable. Rather than selecting the correct answer from a list of 4 or 5 options, they can reduce their odds of getting it wrong and increase the potential for guessing the right answer. This can affect the validity of the exam. 

2. Grammatical Flaws

Inconsistency

This commonly occurs when there is a misalignment between pleural and singular word forms and a mixing of nouns and verbs. For example, if the stem asks for a pleural answer, don’t include singular options in the choices. This is also true for nouns and verbs. Do not use verbs in the choices if the question seeks a noun answer. 

Cues

Another common grammatical mistake is to provide cues. This shows up by asking a question in the stem that doesn’t flow or make sense with the choices. This can also happen grammatically, such as having “a” or “an” at the end of an open-ended statement. Whichever one you use, be sure that all choices and the correct answer grammatically flow with the stem. 

Example:

A 47-year-old female is brought to the emergency room after being found lying unconscious on her driveway by a neighbor. After ascertaining that the airway is open, the first step in management should be intravenous administration of: 

a. MRI of the head

b. diazepam

c. examination of cerebrospinal fluid

d. glucose with vitamin B1 (thiamine)*

e. phenytoin 

3. Repeated Words

Do not repeat words in the stem with those in the correct answer. 

4. Distractor/Correct Answer Length

Aim to keep all the distractors and the correct answer close to the same length. The common mistake is that the correct answer is notably longer than the distractors.  

5. Randomizing Correct Answer Location

Two of the most common locations for the correct answer in an MCQ-type question are choice B or C. The easiest way to eliminate this dynamic is to list the distractors and correct answer in alphabetical order. 

6. Plausible Distractors

Whether you are writing or revising MCQs, plausible distractors are one of the more challenging components of a good MCQ. The reason they are called distractors is because their job is to distract. This is not with the intention of tricking students. Rather, it is meant to discern the level to which the students know the material. In the single best answer type MCQ, there should only be one correct answer given the stem. However, the other choices need to be plausible and possible, but they are not the best answer. This also means they should be homogeneous. If you are asking about diagnostic imaging tests, all choices should be diagnostic imaging tests. If you include something that is not, exam-savvy students will notice and immediately remove that distractor as a viable choice. 

Another issue that comes up with plausible distractors is the use of ‘all the above’ or ‘none of the above.’  These aren’t viable distractors and should not be used as the correct answer because if they are included, it is a cue to students that it is likely the correct answer. 

7. Complexity

A common mistake I see with written or chosen MCQs is related to the complexity of the question. When deciding on a question, in addition to ensuring it is aligned with the course instructional objectives, it is important to keep in mind where the student is relative to the curriculum. This means that the information in the question needs to be consistent with what students have learned up to that point. What happens, most commonly with vignettes, is that information that students haven’t learned yet is included. For example, giving a patient disease states that they have not yet been taught, lab results they haven’t learned yet, or medications they have not covered. Likewise, sometimes the vignettes themselves are overcomplicated. The information included should only be the information that is needed for the student to answer the question. Adding superfluous information or trying to trick the students is not the purpose of the question. 

The quality of the MCQs directly relates to exam validity and reliability. Our goal is for our assessments to effectively and accurately evaluate student knowledge based on the course instructional objectives and learning outcomes. Reviewing and watching for these common mistakes helps to avoid cuing our exam-savvy students and helps to ensure accurate assessments. 

Reference

DiSantis, D. J., Ayoob, A. R., & Williams, L. E. (2015). Prevalence of flawed multiple-choice questions in continuing medical education activities of major radiology journals. American Journal of Roentgenology: 204. 698-702. doi:10.2214/AJR.13.11963 

National Board of Medicine Examiners [NBME]. (2021). NBME Item-Writing guide: Constructing written text questions for the health sciences. https://www.nbme.org/sites/default/files/2021-02/NBME_Item%20Writing%20Guide_R_6.pdf