Selecting Evidence

Now that you have a set of learning objectives and a curriculum map that tells you the courses that address those objectives, you are ready to choose a method of collecting evidence that will show you whether or not your students have met your objectives. This evidence comes in 2 forms – direct and indirect.

Direct evidence shows you “directly,” through actual student performance on assignments or tests, what students have learned well and what they have not learned as well as you would like. It is the strongest kind of evidence. Examples include scores on a set of multiple choice questions or an essay, portfolios, or capstone projects.

Indirect evidence may help to support the direct evidence, but is not as strong on its own. Student surveys, or example, are indirect evidence because they rely on student perceptions. Other examples of indirect evidence include retention and graduation rates, alumni and employer satisfaction surveys, and student participation in research.

For additional examples and information, see Direct and Indirect Measures of Student Learning and the example below.

Test your ability to identify direct and indirect measures.

Questions? Contact the Schreyer Institute for Teaching Excellence at assess@psu.edu.


Assessing a psychology program – selecting evidence

If a psychology professor wants to collect direct evidence to determine the extent to which students have met the objective Students will be able to effectively evaluate a psychological question using research articles from psychological journals, she might use the following assignment, from a 400 level course called “The Psychology of Fear and Stress.”

Select a psychological question of interest to you. Search the literature and choose 2 studies that explore your research question. Compare and contrast the 2 studies and evaluate them in the context of your question.

Include the following in your paper:

Title page

  • The topic of the papers you are comparing
  • Your name
  • References for both papers in APA style

Definitions of Stress
How was stress (or whatever variable is being studied) defined? If the authors don’t tell you, look at the way it was measured. Compare the definitions.

Rationale

  • Is the rationale for the study reasonable?
  • Does the literature review provide a logical argument for the study design?

Method

  • How was stress (or the major variable) measured?
  • Why were the particular methods chosen?
  • Was there something about the research question that required one method over another (such as type of participant, method of measuring the variable, or procedure)?
  •  If there are physiological measures, which stress response system (SAM or HPAC) is reflected in those measures?

Results

  • Are the results of the 2 studies consistent with each other? If so, how?
  • Do the results of the 2 studies complement each other? If so, how?
  • Do the results of the 2 studies contradict each other? If so, how?

Conclusions

  • Combine the conclusions of each study to provide an overall conclusion about the topic of the research. What do these two studies add to the literature?