The Secret to Writing Effective Assessments

By Joseph Berman, Editorial Director, Science

How do you write a multiple-choice item? All you need is a question stem, one correct answer, and one or more incorrect answers (also called distractors.) But how do you write an effective item? That task is more complicated and challenging. In today’s classroom, assessments must meet a variety of requirements, including grade-appropriate readability, lack of bias, coverage of state or national standards, and targets for depth of knowledge (DOK) and Bloom’s taxonomy.

At KGL, we work with writers and editors who recognize and understand the criteria for effective assessment in language arts, math, science, and other content areas. Many of these professionals have devoted their careers to assessment, and we are delighted to work with them.

The Criteria For Effective Assessment

Everyone agrees that every assessment item needs at least one correct answer (as in multiple-choice, true/false, matching, and other formats) or a model answer (as in constructed responses or essays). This allows the student’s responses to be evaluated and graded.

After this baseline, however, the criteria for effective assessment become more subjective. I cannot cite a single, etched-in-stone set of criteria for assessment, or at least a set on which all reviewers agree. That stated, below is a list of commonly-cited criteria for multiple-choice items. All the criteria share the goal of ensuring the assessment is fair to students and useful to the instructor.

• Focus the question stem on one relevant topic.
• Avoid double-negatives (and for some reviewers, all negatives should be avoided.)
• Distractors should be non-trivial and plausible.
• Answer choices should have parallel construction.
• Answer choices should have comparable length.
• Answer choices should have a maximum length of one sentence and ~100 characters.
• Avoid “none of the above” and “all of the above”.
• For the whole assessment, the letter designations of correct answers should be random.

Just for fun, let’s try to construct an assessment item that fails as many of these criteria as possible. Here’s my nominee:

Which is a FALSE statement about life science?

1. Viruses can reproduce, even though they are not alive, are smaller than cells, and are made only of proteins and genetic material.
2. Microscopes use plastic lenses.
3. Never eat or drink during science class.
4. All of the above
5. None of the above

Wow—that’s really awful.

Asking for the false statement, and then including “None of the above” as an option, makes the item as much a logic puzzle as a science assessment. Another fault is that answer choices could be legitimately interpreted in a variety of ways, even by students who have mastered the science content. Choice A, apart from being overly long, is too vague about the topic of viral reproduction. The student may or may not infer the word “independently” as part of the answer choice. Choice B is true for some microscopes, but lenses can also be made of glass, while electron microscopes do not use optical lenses. Choice C is a rule that is widely enforced for science laboratories, but not necessarily the science class. If the day’s activity is a nature hike in hot weather, then drinking water should be encouraged.

Perhaps the overlying fault of this item is that it lacks focus, meaning that it does not assess a specific topic or skill. If the teacher wants to assess a student’s knowledge of viruses or microscopes or lab protocol, then the question stem should focus on one of these topics, not “life science” in general. Another option is to replace the item with three true-false items, one for each topic.

Here is a much-improved assessment item. It assesses one detail from the original item, a detail that provides all the focus that the item needs.

Many scientists argue that viruses are not alive. Which statement provides the strongest evidence for this argument?

1. Viruses contain protein and genetic material.
2. Viruses cause disease by infecting cells.
3. Viruses cannot reproduce independently.
4. Viruses can remain dormant inside a cell.

This item clearly assesses one well-defined topic, which is the classification of viruses. All four answer choices are complete sentences of about the same length. Only choice C provides the evidence that the question stem requests, so C is the correct answer.

The Secret to Writing Assessments

If you are tasked with producing any writing product—assessment items included—my #1 piece of advice is always the same. That advice: review your work before considering it final. I am always amazed at the new perspective I gain from a second read-through of my handiwork. And if I revise an assessment item or replace it entirely, I pass it through an additional review.

At KGL, every assessment item that we release to clients has undergone at least three rounds of review. The writer is charged with writing the item from source material. A content editor reviews the writer’s work and revises as necessary. Then a copy editor reviews the item for grammar, writing style, and other issues identified in our guidelines.

Many items receive additional review from the editorial manager, especially if issues were identified in earlier reviews.

Joseph has over 25 years of K–12 science and math product development experience. He edited high school science textbooks at Prentice Hall and served as Science Editor and later Editorial Director of Technology at Macmillan/McGraw Hill. At Scholastic, he developed math programs such as MATH 180. Joe oversees science development for KGL and can be reached at info@kwglobal.com.