This is a weblog tutorial for instructors and test writers interested in gaining a better understanding of how to use item (or test question) analysis. Item analysis provides useful information about how well test items ‘performed.’
Was a particular question as difficult as you intended it to be? Did the item do a good job of separating students who knew the content from those who did not? How effective were the item options (i.e., key and distractors)? What changes should you make before using an item in subsequent administrations of the test?
Anatomy of a Test Item
Test items can be systematically analyzed regardless of whether you use ANGEL or send bubble sheets to be scanned at Scanning Operations. Test score validity and reliability can be improved by analyzing item performance over time and making necessary adjustments.
By the end of this module, you should be able to:
- Define and explain the indices related to item analysis
- Locate each index of interest within ANGEL and Scanning Operations score reports
- Identify target values for each index, depending upon your testing intentions
- Make informed decisions about whether to retain, revise, or remove test items
Navigating the tutorial
An item analysis typically focuses on four major pieces of information: Test score reliability, item difficulty, item discrimination, and distractor information. No single piece should be examined independent of the others. In fact, understanding how to put them all together to help you make a decision about the item’s future viability is critical. Use the links at the right side of the page to guide you. Each of these pages is comprised of five sections indicated by the following questions:
What is it? Here’s where you’ll find a definition or basic explanation.
Where do I find it? Depending on the source of your item analysis information, the information you need will be located in a slightly different location. This section will help you find it.
How should I use this information? Here you’ll get a brief explanation of how to really make sense of the specific information provided. It is an elaboration on the “What is it?” section.
What should I aim for? This section provides some target values. It will help you further determine whether you’re on track with individual items or with the set of items across your exam.
How well do I understand it? Three short questions will test your understanding of the topic. You can respond to the questions, or not. Answering the questions is not required for navigating to another section of the tutorial. (Note: The section on reliability does not have quiz questions.)
About the Author