I don’t recall ever being quite so disappointed by a set of test results. I didn’t think the test was especially hard, and certainly no harder than Class Test 2. But the grades are down. The average score among those who took the test was 74% (C), down from 77% (C+) in the last class test and 79% (C+) in the first test. That is a trend so going in the wrong direction.
The specifics: A, 11; A-, 15; B+, 33; B, 41; B-, 47; C+, 38; C, 24; D, 59; Fail, 46; No shows 19. One student got 26/28 and 7 students got 100% on my ask-28-questions-grade-out-of-25 algorithm (down from 9 last time and 20 the time before). The number of A’s was down (from 15 and 42), as was the number of B+’s (down from 46 and 40). The only growth area is the C+’s (this time up from 27). So my strongly bimodal distribution of earlier tests now has less of a valley in the middle. Hardly an achievement.
So what’s going on? Possible explanations:.
- The test was impossible. But a dozen students did outstandingly well, so the test was do-able.
- There were some badly worded questions. I won’t know until I talk to the students, but looking at them, that’s not obvious to me. There were a few questions where I had slung in some class slogans (e.g. correlation does not equal causation) that were perfectly correct answers to questions I wasn’t asking. Those always seem to cause problems (ie test understanding). I think people recognize the slogan and opt for it without thinking about the question. That’s sobering. There was also a question about whether the study in the the media report was a randomized control trial (it was), but to realize that, you had to read the report and think about it. Most of the students having incorrectly decided it was an observational study then got the next two questions wrong (since RCTs but not observational studies allow you to rule out reverse causation and third variables). But other than that, I can’t see any real question-related issues.
- Nobody got the guest speakers. Five of the questions revolved around the recent presentations of our two guest speakers, Mike Mann and Doug Cavener. I reviewed both those presentations with the class following their visits. Attendance at all three of those sessions was poor (just 75% of the class present when I went back over what they’d said), and that can’t have helped. But looking at four of those five questions, the majority of the students got them right: 88%, 64%, 63% and 59% (which, if non-attendance was fatal, translates to 100%, 85%, 84% and 78%). Only 30% of the class got the fifth question right. This concerned Dean Cavener’s work to put a giraffe gene into mice. He is not doing that to test the power of new gene editing technology or to test evolution (why would he do that? — both of those have already been thoroughly demonstrated). No, he’s doing it to see if a genetic difference correlated with the height difference between giraffes and their nearest relatives is actually a cause of height difference. I really thought I went over that well in class. Clearly we need to do some work on correlation/causation, and maybe revisit gene editing and evolution, but that’s just one question — not enough to sink the class average.
- Study groups are working together and group-think is misleading them. This is a possibility. The students are supposed to do the tests alone. And they pledge to that effect. It’s an integrity violation to work with others. It can also be dangerous if the blind are leading the blind….
- Too few people have been to review sessions. That’s certainly true. Is this the problem? How to fix that? I can’t do review sessions in class. No one complained that they couldn’t make reviews last time or asked for other sessions to be put on. Maybe after these grades, some more students will make the effort to review things.
- People are really not understanding things. This is a serious possibility. The questions all variants of what I used in previous years. People are not getting all the same things wrong (exceptions above); they are getting diverse things wrong and more than in previous years. I am not sure why. This year I have been going much more slowly through material, and I thought if anything explaining things better. I wonder if my focus this year on soft skills has been taking up so too much class time. It might be better to spend more time probing the concepts from different directions (that’s what builds real understanding) rather than talking of phones, study skills and time management.
So what to do? I won’t really have a good understanding of the misunderstandings until I do the first review session and see what students are missing. Hopefully we can do that first review session tomorrow so I can get a real sense of what’s happening asap.
I want to do everything I can to help students over this bar. Except lower it.