Category Archives: Tests

Andrew Read

September 22, 2016

The plagiarism test. 14 questions. The students got to do it has many times as they wanted but they had to get 100%, and once they did, the test was dead to them.

I just noticed on the computer log: the 360 students managed to submit the test a total of 2,685 times.

The Hunger in Our Heads

Much to my delight, one of the TAs just emailed asking for a copy of Class Test 1 because Angel’s review option only shows the questions, not the media article on which half the questions are based.

This means at least one student is reviewing their test performance, immediately after the test. At least one student has ceased control of their learning!!!. Awesome.

This is the article.  I found the message a bit surprising. Intellectual work makes us hungry — and the claim is that excise reduces that brain-induced hunger. I dunno. Surely adding exercise will add hunger? Maybe the article is describing another surprising thing we did not know about ourselves — or maybe the conclusion is just plain wrong.

Class Test 1 results in

On balance, I’m pretty pleased. I made the test slightly harder than I normally do for the opener, but I also spent more time reviewing things before hand (not least, I used half of last years first class test as a Pop Quiz last week). I also deliberately covered less material in the first three weeks of semester, with a view to trying to get the basics better ground in. And it all seems to have worked! Overall, things are up on last year at this time — more A’s, a higher overall average and most importantly, fewer D’s and Fails.

For those who did the test, the average score was 79% (C+). One student got all the questions right on her first go, the first in over 1400 students to have achieved that. Two students got 26/28 questions, and seventeen more got 25/28, so a total of twenty got 100% on my ask 28-questions-grade-out-of-25 algorithm. In total, there were 42 A‘s, 32 A-, 40 B+, 40 B, 42 B-, 33 C+, 39 C, 35 D, 49 and 28 fails. We had 12 no-shows.

But that all represents a big teaching challenge: over a third of the class is on a B+ or better, but more than 1 in 5 on a D or worse. How to keep the top students stretching while lifting those in the tail of the distribution? b_veicular_geral

Of sickness and sick notes – and other absences

It’s that time of year where sports injuries and infectious diseases set in with a vengeance. Job interviews and family weddings (and troubles) also become more common. All this generates student emails offering doctors notes, excused absence paperwork etc. This is my generic response to emails about absences.

As discussed in the syllabus (p. 6, Attendance, Missed Classes and Missed Assessment), I don’t need any paperwork. The course is set up so that life events won’t cause problems for engaged students (i.e. regular attenders, those who take the tests, and frequent bloggers). Thus, I take the best two of four class tests, the best of three blog periods, and for attendance, presence at nine of 12 pop quizzes. The final exam is even live for six whole days and can be taken anywhere in the world. So bad luck can strike (indeed several times) and all is still fine. Of course, for disengaged students, it can be a train smash if things go wrong at the end. When that happens, I feel for the students and wish they had been engaged earlier.

The only paperwork I might need is for situations like chronic illness which keeps students from working for several weeks or more. Those cases get really tricky, but are fortunately super rare.

Of science — and the question

In Class Test 3, I asked Why are scientists reluctant to say they have proved something?
Q&A(a) data can fit any hypothesis
(b) correlation does not equal causation
(c) human intuition is lousy
(d) scientists make mistakes
(e) data can be consistent with more than one hypothesis
(f) it is hard to reject the null hypothesis
(g) science is anti-authoritarian
(h) all of the above.

Two thirds of the class answered ‘all of the above’. Less than 20% of the class picked the correct answer (e). This is the sort of thing that makes professors despair. I don’t think it’s a tough question. It simply can’t be ‘all of the above’ — if (a) was correct, we couldn’t do science. What I think is going on here is that students’ recognize slogans I use in class and go for them. Options b, c, d, f and g are all things we have discussed in class. They are all perfectly good answers – to different questions.

Am I really the only professor on campus expecting precise answers to precise questions? Or am I missing something?

More review questions

I learned from the mid-semester evaluation that review questions are popular, so I asked some more today. Two came from stuff I’d been over before the last class test…

…and one came from last Tuesday’s class

The first two went fine (the most popular answer was right), but the last one gave me a teachable moment about the difference between ideas/hypotheses/speculations and actual knowledge. We aren’t short of ideas, but we really have no idea why we are so bad at Monty’s problem. But bad we sure are. You should switch.

Of those three questions, only the last comes from an actual SC200 exam. But all three are analogous to questions I will ask. I always ask in different ways, in different contexts. I want to test understanding, not regurgitation or memory.

Review questions

Question-Mark3For the first time this year, I did a couple of review questions in class today to make sure stuff from last class was bedding in. When I said at the end, after questions were off the screen, that both came from last year’s tests, several people asked me to post them. So here they are.

Question 1 (Ethics of refusing experimental treatments in cancer trials)

Question 2 (Why large double-blind trials are often needed to identify dangerous medical practices).

In both cases, the majority of the class got them right. Thank goodness. There is always a slight tension when I reveal poll results – if the majority of the class have gone wrong, that would throw a spanner in the works.

Where are the men?

First review session of the year was today. I love it. Small(er) group teaching, lots and lots of interaction. You can see the whites of the eyes. You can see when they don’t get it–and best of all, you can see when the light bulbs come on. That is so gratifying.

4ib4K5xigThe major bummer: I forgot to take a photo of the group. In previous years, it’s been tragic. Today, there must have been almost 40 students. Best attendance ever. Camera Andrew? Wake up. Too busy with water, recovery from class, projection systems (that damn multitasking problem)….but I did observe the sex bias. It is always there. This time there were two blokes among the ?40. I am not sure what the class sex ratio is. But it is not 1 in 20 (maybe it is 9 in 20?). As a father of sons – and frankly, the most charming and smart young men you will ever meet – it is seriously sad that blokes don’t revise. I hate to generalize, but here goes: women work to achieve. Men assume they will.

They don’t. Somil, one of the class TAs this year, was in the top three in class last year. The next guy was in rank order #21. That’s an anecdote. but every year, I have had no options for male TAs. If I am lucky, one bloke will be up there. Most years, none are. That’s six anecdotes. Starting to sound like data.

The first results: Class Test 1

The first class test happened yesterday. After last year’s fiasco, I spent an inordinate amount of time checking the computer settings for the test, and it all went remarkably smoothly. Just a couple of people with self-inflicted computer problems, and getting those bugs out of their systems is one of the reasons I have a test early, and then only take the top two grades of the four class tests.

The actual results were disappointing in terms of student performance, but in line with the ‘wake-up-call’ results of first class tests in previous years (e.g. 2014). This is a timely reminder for me and more importantly a strong message to many of the students: SC200 not a cake-walk. We all need to focus and up our games.

The average score was 75% (C+) for those who did the test, and we had 10 no shows. Three students got 26/28 questions, and five more got 25/28, so a total of eight got 100% on my ask 28-questions-grade-out-of-25 algorithm. In total, there were 20 A‘s, 13 A-, 36 B+, 43 B, 40 B-, 54 C+, 39 C, 59 D‘s and 42 fails, a distribution which looks like this:

Class Test 1

That’s remarkably similar to the distribution this time last year. For the faint of heart (which includes me), that distribution moves substantially to the left by the time we get final grades.

The good news? There were some really great efforts (the 8 students who got 100%, and a lot of very good ones [152 with a B or better, almost half the class]). No individual questions where disasterously handled, apart from one on wormy kids and one on smoking. Both of those questions seem to me to point to students barely paying attention in class, but I guess to the students they point to me explaining things poorly – I’ll go over those again in class today. The critical assessment of the media report was pretty well handled on average, and since critical thinking is the aim of my game, that’s good news.

But obviously the bad news is the 101 students who did poorly (which I take to be a D or less); that’s almost a third of the class. For many of those students, this will be the worse grade they have ever got. Those students need to work with me now. What are they not understanding? I can help if I know what the problem is.  Otherwise, I am left guessing. So its time to implore the students to take control of their own learning. Revise the test (its there, with right/wrong, on Angel). If you don’t understand the answer, ask. Ask, ask, ask. I am going to do everything I can to get students over the bar – except lower it.

The final exam 2014: FUBAR

The-Biggest-Mistake-You’re-Making-on-TwitterWhat a cock up.

I let students have two goes at all the class tests and the final exam. I get the on-line system to tell the students how well they did after the first go, but nothing more. This means they do the test again thinking even harder. I like that: the aim of my tests is to teach (assessment is a very very secondary aim). The students like it too: they think having a second go helps them (though the reality is that their scores are as likely to go down as up). This two-shot strategy has worked really well for the 24 class tests and final exams that I have previously run. But this time….

The course management software (Angel) returns the answers as its default position. In the end-of-semester chaos, I forgot to flick the switch that stops that… And I never noticed because Angel doesn’t let you pretend to be a student doing the test ahead of time: you can only get the student view in real time. And I did not have real time.

I was at a conference in Thailand, jet lagged. I got to do the test eight hours after it went live and just a few minutes before I was due to give my keynote talk. I immediately saw the problem (oh that sinking feeling). But by then, about 15 students had done the test. Naturally, most of them had got 100% on their second go (although my favorite exception was the first student: from 46% to 93% [93%?]). I realized immediately the test was a blow out. If I switched off the answers there and then, what to do about those 15 students? I debated calling the whole exam off but decided that most of the students would have a proper go the first time, and so learn something. But as an assessment, it was clearly blown. I would have to give everyone who did the exam 100%. Which I duly did.

The fiasco generated an interesting study in humanity. Of the 181 students who did the final exam, just six emailed to point out the problem. (To put that in perspective, four other students emailed to complain about their blog grades during that same period.) One of the six covered herself in glory. She emailed me, a TA and my staff assistant to point the problem out – and then deliberately did not use the right answers before her second go. Another of the six (one of this year’s blog plagiarists) used the answers to elevate her score to 100%, and then asked for extra credit for bringing it to my attention.

Among the other 175 students, there were 15 who got all questions correct on their first and only go. It is possible that they genuinely achieved 100% through their own efforts. If so, I hope those students took great satisfaction. But on none of the previous tests did anyone get everything right, let alone on their first go. Hard to think the list of answers wasn’t floating around the class, despite the honor pledge. It’s no wonder there is so much corruption in society.baraka_monkey

Still, no getting round it: what a cock-up on my part. Apologies folks.