Assessing Facts and more
Multiple-choice (M-C) quizzes are very popular activities in online courses. They are automatically graded, alleviating some of the workload on the instructor, and feedback about performance can be instantaneous which makes students happy. Often quizzes are used to assess or guide recognition and definition of key terms and ideas. In this way they function like reading comprehension quizzes, vocabulary tests, or worksheets. In this function quizzes are part of a pattern or flow of modules that goes something like this:
- Student does reading/lecture
- Student completes mutiple-choice (M-C) quiz
- Student engages in discussion activity to construct more complex ideas through guided discourse with others
- Student does individual written assignment that is graded by instructor
- An end of unit test may include M-C quiz questions as well as essay questions
Well-crafted M-C questions can address more than facts. Below are resources that will help you write better quizzes.
Assessing more than facts
This link provides examples for constructing multiple choice items that assess comprehension, application, and analysis.
14 Rules for Writing Multiple-Choice Questions (pdf) from CTL, Brigham Young University
Pages 1 and 2 contain comparisons of multiple choice items written to assess recall versus application of knowledge. These may give you some ideas about how to transform recall items to items that require application.
Techniques for writing multiple-choice items that demand critical thinking
This Web resource provides examples of multiple choice items that assess intellectual skills, such as analogical thinking, application and evaluation.
Writing good multiple-choice exams (pdf)
This is a comprehensive resource on writing multiple choice exams. Pages 20 to 24 provide examples of multiple choice items written to measure learning outcomes of different categories of intellectual skills. On page 30 to 32, scenario-based problem-solving item sets exemplify items that test complex thinking, application and integration of knowledge.
Many suggestions for designing test questions were found. Instructors may:
- Design questions that could not be answered easily unless the individual has done the previous work in the course (Olt, 2002);
- Have students apply personal experience when answering questions (“Strategies to Minimize”, 2006);
- For courses that test using calculations, give each student the same exam with numbers changed slightly (Goldsmith, n.d.);
- Use multiple-choice questions only for ungraded assignments (Goldsmith, n.d.);
- Use “rote memory” questions for “gauging the pace of the course and identifying students who are lost” (Hollands, 2000);
- Use multiple-choice tests to emphasize important terms and concepts. Nelson said that he permits referring to the textbook for answers – “so much the better; for some, sad to say, it may be the only time they read the text” (1998, pp. 7-8 ); and
- Design open book questions so that they are more than “scavenger hunts” for correct answers (Golub, 2005).
There are also several recommendations for establishing testing procedures:
- Do not make a test available until the day you want students to begin taking it (Hollands, 2000);
- Assign a password to each exam and make it available to a student just
prior to his attempt (Olt, 2002); TCC 2007 Proceedings
- Use several short quizzes to make it difficult for students to get constant help (Olt, 2002);
- Use multiple-choice exams for practice testing only (Ritter, 2006);
- Allow multiple attempts, making testing a learning experience (Ritter, 2006);
- Set time limits (Ritter, 2006);
- Monitor beginning and submission times for students in each section
(Steuver & Harter, 2004);
- Routinely compare student answers (Rowe, 2004);
- Use multiple forms of exams, randomizing questions from a large pool;
And Randomize the answers on multiple-choice questions (Rowe, 2004).
If you have any problems or questions, please contact IT at firstname.lastname@example.org.