Category Archives: Thoughts about SC200

That's it.

xp-end-is-hereWell, time to sign off. I’m not sure what the future holds. I am on sabbatical 2017-2018, so if SC200 runs next Fall, it won’t be me running it. Whether that hiatus becomes permanent depends what’s happening when I get back from sabbatical and what the College wants to do with SC200. There are opportunities for new types of Gen Ed course now. I could do something different. I can imagine interesting things to be done now trans-domain courses are possible; I quite fancy joint teaching a course with someone in history or philosophy or literature or economics. I could also focus SC200 a bit more (perhaps entirely on to medicine and health care?) or keep the same general themes and just find different material to try to stay fresh. All things to ponder on sabbatical. I can’t help think that the themes, objectives and subtext of SC200 will become even more important in the coming months and years. Science is going to remain the best system for knowledge generation and problem solving that humanity has, but it is also a hugely civilizing process. Looking forward from this tumultuous year, it looks like we will need that side of things more and more.

But for now, a big thanks to people who made the course happen this year. Thirteen others contributed. Thanks to:

  • The class TAs, Brian, Eric and Sarah. Huge efforts, well above and beyond. The students and I really appreciate your efforts to support the students’ learning and blogging.tas-2016-cropped-pul0aw
  • The guest speakers who volunteered their time and their very different perspectives, Eberly College of Science Dean Doug Cavener, Mike Mann (Meterology) and Jason Wright (Astronomy). Thanks for making us all think, me and the students alike.
  • Monica, who kept me sane while dealing with endless emails, handouts ready in the nick of time, and pieces of paper for attendance grades and extra credit.
  • The graders, five hard working grad students who got things done under immense time pressure and, most impressively and despite the best efforts of sites@psu and Campus Press, found most of the students’ work. Thanks too for taking the time to give the students such detailed feedback.
  • Chris Stubbs, TLT tech guru who got the site up and running and then put up with my frustrations at what he no longer had control over.
2016-11-08-13-39-57

Jason Wright in class, Nov 8, election day.

And to the Class of 2016: Thanks for the challenges. Keep thinking. Have a nice life.

Self reflection

OK Andrew, what worked and what didn’t work this year?

Pluses:

  • Classroom discipline MUCH better. There was still some complaints about distracting whispering, but nothing like the last few years and none of the shameful s*** that’s happened before. None of it got me worked up, unlike previous years.  I think the difference this year was that I talked more at the start of semester about it, pitched it to the students as a deal (the deal works like this, you do x and not y and I will do a and not b). I also took a very light touch approach in class when it happened, politely asking people not to. That seemed to go better than trying to shame them into silence (and way better than getting pissed). I think walking up and down the aisles really helps too.
  • Plagiarism. Just one case this year. Broadly speaking, the countermeasures worked. Unlike the College Academic Integrity Committee, which continues to under perform in this important and delicate arena.
  • I did a lot more work with the students on soft skills. That felt good. I hope it set at least some of them up for a better College experience. No way to know of course, so like a good physician, I’ll roll with confirmation bias and assume I did good.
  • Again, no problems with the laptop ban, and progress on the phone issue.
  • Explaining to students more about why we are doing things was good.
  • Attendance algorithm worked much the same as last year. I think stick with that.

Room for improvement:

  • More active learning. Things like this (interestingly, that video is pitched as being about clickers, but in fact they are pretty irrelevant to that sort of teaching practice).
  • More debates. For example, is it worth testing homeopathic vaccines?
  • More challenge questions to stoke curiosity. Mini research projects?
  • Build a capstone class around John Oliver’s great show?
  • The students continue to find their blog grades disappointing, by and larger, but at the same time by and large don’t produce excellent work. Need to think more about why that is. Show more examples of best practice? Or am I fighting the Facebook/Twitter drivel. Perhaps students don’t read much good writing these days?
  • Talk more about cognitive failings of humans. Use the analogy with optical illusions more. Our eyes/brains can deceive us. Similarly we have cognitive defects too. Monty Hall good for this. Bring this out even more e.g. with the Linda problem. That scientists can use science to both study and (somewhat) over come these problems, even though everyone has them.
  • Talk more about note-taking earlier on. Do some examples — it worked really well this year when I reviewed (all across the blackboard) what Mike Mann and Doug Cavener had said, partly because I (hopefully) reinforced and clarified their messages and partly because the students could see what I mean about reviewing after class what had gone on.
  • The blog software. Groan.
  • Is there some way to motivate students to solve the procrastination problem? Do a class on procrastination science? That might be interesting to think about, especially since what I read does not make too much sense. It’s like a huge human blinder.

The big unknowns remain big unknowns. (1) Am I pushing the students hard enough or too hard? (2) What impact are any of my efforts having? I can imagine ways to investigate #2, though not easily. I still can’t even imagine how to investigate #1. That’s just not my problem. It’s a College universal. If I didn’t find my science so interesting, I might turn my scholarship activities to ruminating on that. I’ve made no progress on it since I first started pondering it in 2011. I haven’t seen or heard on anyone else even wondering about it, even though it goes to the heart of this huge, expensive industry we call Higher Education. Most industries reflect vigorously on what they are doing (or they are forced to by outsiders); why don’t we?

Extra credit

extra_credit_2My use of extra credit has grown over the years, despite my concerns about grade inflation. I use it to

This year I really went crazy with it, in the end offering nine different routes to extra credit. I capped it at 10% so that extra credit does not dominate the grade, but within that constraint, a student can go for it as they wish. Much to my amazement, almost no students make anything close to full use of it. It is much easier to get 10% through extra credit than it is to get an extra 10% by doing better on the tests or blog.

I offered extra credit for:-

  1. Particularly lucid, stimulating, artistic or lateral blog posts (max 5%/post). This is to encourage/reward outstanding work. I thought few students did enough to deserve this, but it’s very good to have the option to reward those who go above and beyond.
  2. Suggesting exam questions. You have to really know your stuff to write exam questions. Just 5 students offered any up, even though I’d give them 2.5% for every question I used.
  3. Finding a mistake in an class test or an exam that causes me to regrade (max 5%/mistake). A couple of students suggested mistakes, but they were typos and so did not need regrading. Nonetheless, I think this extra credit is good because it emphasizes the possibility that Professors can be wrong, it gets hypervigilance going, and students who argue with me learn, even if they are wrong (especially if they are wrong?).
  4. Partaking properly in the first blog period (1%). This is an anti-procrastination (get-off-your-butt) carrot which I was trying for the first time. It did not work at all: only about 100 students participated properly, fewer than last year when no extra credit was available.
  5. Blogging ahead of deadline (2%/deadline). This was a time-management carrot. It too did not work.
  6. Surrendering phones in class (1%/time). This did work.
  7. Writing an extra blog (2%). I asked the students to write about something they learned in class and how it had or might change their life. Just  nine students took advantage of this. Maybe that mean the course had no impact on the 310 other students. But I thought all of the nine were really interesting, especially this and this and this and this.
  8. Opt in to names in the hat (1%). A little under a third of the class did this, which says something about students, but nonetheless, I liked this solution to the problem of cold-calling students in large classrooms.
  9. A bribe to get the SRTE return rate up (1%). I wasn’t going to use this bribe this year, but with just a few days to go, only 30% of the students had given feedback through the Student Rating of Teaching Effectiveness system. Since that 30% was for sure not going to be a random sample [as is clear from what appears on Rate My Professor], I offered the 1% extra credit to everyone in the class if the class return rate got about 80%. It hit 82.5%… I’ve agonized before about this shameless bribe, but I think we have to do it if we are going to take anything meaningful from the SRTEs.

On average, the class got 4.7% extra credit. That’s pretty amazing, given that 4% would happen more or less automatically (3 x 1% for the phone-ins + 1% for the SRTE bribe). Just 13 students got the maximum extra credit and only 34 got 8% or more. I am sure I had more students just ask for more grade.

Bottom line? There is an administrative cost to all this extra credit, and I was able to keep on top of it only because I have Monica supporting the course. Without that, I am not sure I would keep anything except #1-3 and #9. But otherwise, I think worth persisting for the bullet-pointed reasons I give above. For professorial peace of mind, buffers against students complaints and begging are not to be underestimated. More positively, carrots are at least in principle a good way to nudge student behavior, even if there is not much sign they actually worked on my students. Perhaps the time management/anti-procrastination carrots need to be bigger (#4, #5). Just how much do I need to bribe students to do what’s good for them?

2016: the bottom line

I calculated the final grades almost a week ago and then let them sit on Angel until now. This is to give the students a chance to complain. That generates a bit of e-traffic but very effectively crowd sources the search for errors in my grade book. With 300+ eagle eyes on it, I am now confident there weren’t any. So the grades are officially posted today. They look like this:

grades-2016

The class average is 87.6% (B+), or 89.6% (B+) for those who passed. We started with 358 students; we ended with 317. Among the finishers, 50% got some type of A, 66% got a B+ or better and 80% got a B or better. With extra credit, 11 students got >100%. Altogether rather similar to last year.

I say it every year, so I guess I’ll say it again: what to make of this grade distribution? Is it about right or too high or too low? We had a Biology faculty meeting a while back, that I sadly missed (not often I say that), where the proportion of A’s was being discussed. In biology classes for 2013 with more than 20 students, the numbers looked like this:

% A and A- 100-200 level Bio Courses 400-level Biol Courses
Mean 24% 42%
Median 27% 36%
Range 13-39% 13-99%

Everybody except the person awarding 100% A’s thought 100% was too generous. The minutes from the meeting helpfully say: “Faculty Senate policy allows faculty to grade according to their best judgement. Although programs can provide guidelines, ultimately grades are at the discretion of the individual faculty member. Several faculty shared their experience of figuring out their grading criteria with little to no guidance. It was widely agreed that some departmental guidelines for grading would be helpful.” No such guidance has been forthcoming because I don’t think any such guidance is possible. It’s a fundamentally challenging problem. The problem is even more difficult for Gen Ed courses where there are no professional discipline-specific views on relevant standards (and how can there be?).

Is 24% about right? My grade distribution with its 50% of A’s is clearly out of line with the 100-200 level Bio courses. Does that matter? People get excessively steamed up about grade inflation, but if we worry about that from data on the proportion of A’s, it implies that the only thing that matters is relative success. And if that’s important, our job is to not what I think it is, but instead it is to identify and anoint the top x% of students.  Which is CRAZY.

Actually, thinking about this too hard might drive me crazy. Previous ruminations are here and here. I am making no mental progress on this problem at all. Worse, I don’t see anyone else even engaged with it. In the shower this morning, I had a thought: isn’t the search for an ideal grade distribution fundamentally silly? What I should care about is the impact I am making to the way students think about the world. The grades might say something about that. But probably not much. So, Andrew, think about what’s important, not what is easily measured. Ruminate on that.

Class Test Score Overall

Bummed out by the Class Test 4 scores, I decided to have a quick look at the overall score for the class tests (I take the best 2 of 4). This usually cheers me up. And indeed it did. The distribution is: A, 21; A-, 81; B+, 36; B, 68; B-, 35; C+, 24; C, 29, D, 12; Fails, 9.

So about a third of the class are on some type of A. That seems about right to me.enhanced-5890-1412873873-5

 

 

 

 

 

 

 

 

 

I wonder why that feels about right. There are no guidelines on this whatsoever. We professors set the bar as high as we want. How high to set the bar is the hardest problem in Higher Education and everyone avoids it like the plague. I suppose the reason it cheers me up to have a third of my students on an A is that no individual test turned up that many A’s. My take-the-top-two-test-grades-of-four algorithm allows improvement and fluctuating performance. So I get to challenge the students and many get well rewarded. No trade-off.

Another observation: four students got a overall class test score of 100%. None of those got 100% in all four tests. I think that is good. Even those attaining the very highest scores still have something to reach for. I feel better about that too.

Class test 4: sigh (again).

downloadThe results from the 3rd Class Test were the most disappointing I’ve ever seen.  Yesterday was Class Test 4 and the results are only a shade better. Among those who took the test, exactly the same class average as Test 3, 74% (C). But that average comes from way more A’s and A-‘s (44, up from 26 last time), fewer B’s, C’s and D’s (good) but more dreadful fails. Oddly, we also had way more no-shows (53, up from 19 last time). So the best we can say is that the decline in test performance across the semester has been arrested. The average grade among those who passed was 80% (B-).

Two students got 100% on my ask-28-questions-grade-out-of-25 algorithm, but the highest score was 25/28. The rest: A, 14; A-, 30; B+, 39; B, 33; B- 24; C+, 20; C, 16; D 43; Fail, 53.

When I look at the performance on the individual questions, there were what I would term under “performance problems” with the questions involving stuff I’d gone over in class in the most recent weeks. Perhaps that’s  a consequence of the low attendance:

  • Just 40% of students in the class thought the strongest reason to think a Zombie apocalypse couldn’t happen is that it hasn’t so far; maybe that’s because only 49% of students were in class when I made that case.
  • Just 52% of students were in class when I discussed the famous ‘experiment’ where Barry Marshall drank H. pylori laden broth, only to get sick not ulcers, thereby providing (anecdotal) evidence against his hypothesis that H pylori causes ulcers. Only 45% of the class knew that.
  • Just 60% of the class was there when I showed the video that makes clear that to a reasonable approximation, the Drake equation shows that the main determinant of whether we will detect life in the Milky Way is how long civilizations transmit detectable signals; 53% of the class had taken that on board.

Most heart breaking:

Which of the following is currently an open question in science?
(a) the nature of dark energy
(b) the safety of childhood vaccines
(c) the cause of climate change
(d) the chemical composition of celestial bodies
(e) all of the above.

A staggering 55% of the class chose (e), meaning that I failed to get across the strong scientific consensus on (b) and (c), and many were not paying attention to Jason Wright’s strong guest performance when he showed we learned how to do (d) in 1859. I weep.

Just one question seems important for me to go over in class today. That’s one which makes clear I have not made clear enough that statistical significance per se tells you little about how big an effect is. Statistical significance does not necessarily mean biological significance.

Sex bias?

This is the actual hat I use....

This is the actual hat…

According to a comment in this very interesting video about how to get large classes engaged in active learning, males are more likely to put their hand up and to be called on in class. Best practice is therefore to do random cold calls. Students hate that, and so I compromise: for extra credit, they can have their names in a hat from which I randomly select people.

The class is 44% male. 40% of those who opted to have their names in a hat were male. So no bias there. The real bias is probably in the personality types selected. That sort of bias might be much more insidious.

Culture wars

To Dr Harlene Hayne, Vice-Chancellor***, Otago University, New Zealand

Dear Harlene,

As an Otago graduate (Zoology, Class of 1984), I’ve always enjoyed your articles in our Alumni Magazine. Congratulations, btw, on five years in the job. I hope the next five are as good for you and the university as the last five.

Here at Penn State, I am a research professor most of the time. But for 15 challenging weeks a year, I teach 365 non-science majors about science. I’ve been doing it since 2010, and each year I am amazed by just how hard it is. I have a high bar (and struggle with how high set it) and I do everything I can to get the students over that bar — except lower it. I also expect (demand) that the students seize control of their own learning. But many of my students just hate it (they want A’s on a plate) and most of my colleagues don’t much care for my efforts or standards.

And so it’s a struggle. I’ve often wondered why I bother. No one would complain if I aimed low. But now, thanks to your recent article, I know where my teaching aspirations come from. You went on US tour to get feedback from the US students who do Study Abroad at Otago and, in your words, everyone

… reported that the academic standard at Otago was much higher than that of their home institution. I was constantly told that the American students – many of whom came to us from highly selective, and extremely expensive private universities – had to work twice as hard at Otago as they did at home.

They also told me that Otago required students to think for themselves and to take responsibility for their own learning; that Otago fostered a sense of independence that was initially a bit daunting to many of them.

So that’s it! My aspirations are Otago’s fault. Ironic that you, an American in a NZ university have the perspective to explain to me, a NZer in an American university, what’s going on.

Well, here’s to the ‘smart, ambitious and warm-hearted, edgy‘ Otago people who shaped me. To name just three still on your books: AlisonEwan and Alan. You’ve made me realize their reach is long and their contribution to my professional discomfort great. I am sure my own students will one day thank them. I do.

andrew

 

 

 

 

Dr Andrew Read FRS
Evan Pugh Professor of Biology and Entomology
Eberly Professor of Biotechnology
Director, Center for Infectious Disease Dynamics.

This from Otago's recruitment literature...

A major reason I had a fantastic time at Otago University (1981-1984).

***In US speak, the Vice-Chancellor is the University President.

Mid-course SELF-evaluation

I do a mid-course evaluation to find out what I could do better on the remainder of the course. For the first time, I asked the students what they could do to help themselves do better. The most common answers (in bold the number of students who said that):

Take better notes 62
Be less lazy; not procrastinate 60
Focus – pay more attention 60
Review material/class notes more after class 54
See/resort to TA’s/meet with them 53
Ask more questions 44
Turn off electronics/phone 39
Study more 38
Blog more frequently/regularly/earlier 36
Time management 21
Review pop quizzes & tests 12
Attend study sessions/review 10
Attend more office hours 9
Do more extra credit work 8
Sit up front 7

All remarkably similar to the wisdom from last year (learning, grade).

As for what I could do better to help the students learn better? Most requests actually revolved around how they could get a better grade. And the most common requests? More on what makes a good blog, more on test preparation, and could I please require less blogging and make the tests easier. I take the first two points, but I really think those things are a work in progress. As for less blogging? 65% of the students think the work load of the course is about right (which makes me think its too light). As for making the tests easier — how hard to stretch people is one of the toughest (and most ignored) questions in Higher Education. They might already be too easy: nobody thinks the course is insufficiently challenging, with 57% thinking it’s about right and 43% saying its too challenging. You gotta stretch students out of their comfort zones. With almost 60% feeling pretty comfortable, I may not have gone far enough.

Things they think I am doing well? 

Picking interesting topics that keep students engaged 73
Answers students’ questions thoroughly 24
Makes students think critically 11
Detailed explanations 10
Extra credits/many chances to improve grade/succeed 10

Most disappointing is the quarter of the class for whom the course is failing to meet their expectations or who are dissatisfied. The most disappointing thing about that is that no one has come to talk to me about anything they are dissatisfied with. I always wish we could link the feedback to the grade book. Are there engaged students who aren’t happy? That would worry me. Or are the unhappy 25% those who are failing because they haven’t done much?  And by even raising that possibility, have I become the doctor who blames his patient’s death on the severity of the disease rather than the quality of the doctoring……..???

Thanks so much to Monica for her fantastic efforts to compile all the data.

What I've done this year to try to head off plagiarism

Dear Academic Integrity Committee

After last year’s fiasco regarding plagiarism on the class blog, I thought a great deal about what to do this year. Particularly helpful were discussions with the STEM Gen Ed discussion group (their horror stories also added to my motivation). Several articles I was directed to had an impact (1, 2, 3, 4), as did the advice of the ever energetic Julia Kregenow who has actually done courses on promoting academic integrity in our student body (sad that such courses have to exist, but good that she does them and passes on accumulated wisdom). So these are the changes I instituted.

  • Whenever I was discussing plagiarism in class, I asked the students to turn off their phones. Phone-induced inattention was the only alternative explanation for one of last year’s cases (though I still maintain that case was just plain and simple dishonesty).
  • I instituted a fourteen question plagiarism test, which tested the students knowledge of what plagiarism is, what good and bad practice looks like, how the policy and penalties are implemented on SC200, and resources to turn to. I took much pleasure in using as examples of bad practice the writing you got to see last year, and more pleasure showcasing a blog post from a student who really could write complex things in her own words because she the took the time to understand what she was writing about.
  • I required students to get 100% on the plagiarism  test. They got as many goes to do it as they wanted, and the test was live on Angel for six days.
  • That test ended six days before the end of the first blog period, and I made clear in class that anyone who had realized that they might have stepped over the line could still edit out the plagiarism before the first blog period deadline.
  • I refused to grade any blog work for anyone until they achieved 100% on the plagiarism test. This meant 21 students were excluded from Blog Period 1.
  • In the syllabus, I greatly extend my discussion of academic integrity and in particular, the discussion on plagiarism. In class, I implored the students to read that discussion, although we know of course that most never read that part of syllabi. But I know you do when you are reviewing the cases I bring to you. I hope you like the wording this year. Personal highlights:
    • I tried to be as positive as I could (“My promises to you…”)
    • I tried to explain why cheating is bad, beyond the risk of getting caught.
    • I tried to make clear what is honest work, and what is cheating.
    • I made very clear my sources, and made clear that much of the wording was lightly edited from Julia’s syllabi, used with permission.
  • In the syllabus, I made explicit the penalty for plagiarism on the blog. I did this to make it clear to the students how serious this is and more importantly to limit wiggle room when we (me and you) are post-hoc trying to figure out what penalty to impose. I think it important that the penalty be very significant, totally transparent and applied equally to all offenders. The bottom line is that on SC200, plagiarists will get a maximum score for the entire course of a C+ on first offense. In practice, their score will likely be a lot lower and they may even struggle to pass.
  • Most onerously, I made it clear that if anyone wanted to use anyone else’s words in a blog post or comment, they had to email me before hand, explain why and get my express permission. This was Jackie Bortiatynski‘s suggestion. It generated a lot of email, but also made clear to me that many students really do have to be taught what is fair citation practice and what is plagiarism. If all the e-traffic heads off even one offender, that will have been more time efficient than bringing a student before you.
  • I reduced the workload in all Blog periods, but especially Blog Period 1. There is a strong indication that students are tempted to cheat if, close to a deadline, they find an overwhelming amount of work and not enough time to do it in. So I cut the number of required posts and comments from 5 and 13 to 3 and 10 for the first blog period, and made it 5 and 15 for the other two Blog Periods (from 6 and 16). I am not sure what I make of challenging students less in order to try to prevent a cheating few.
  • I discussed in class how much time blog posts might take, so students who left it close to the deadline would not be taken by surprise. I got the TAs to join this discussion, so current SC200 students could hear the experience of former SC200 students.
  • I added extra credit for students who blogged ahead of deadlines.
  • I added extra credit to encourage students to post in the first blog period so they would not be overwhelmed in later blog periods.
  • I said in class, and restated in an email to the class: If you ever feel even the slightest hint of temptation to commit plagiarism, don’t….. Instead, reach out to me. I am always available to discuss any circumstances that got you to the point of thinking about it. But not after it has happened. Once you have tried to pass someone else’s work off as your own, whatever the circumstances, I will begin academic violation procedures, as described in the syllabus.

These were on top of the things I did in previous years, namely I

  • discussed plagiarism in class, using examples from past class blogs of bad and good practice, and making clear the severity of the issue and the consequences of commiting plagiarism,
  • took class attendance on the day that discussion happened so we know who was there to listen to the class discussion, and
  • e-mailed the class discussing the seriousness of plagiarism, how to recognize it, avoid it, and reiterating the details in the syllabus.

Today is the deadline for the First Blog Period. So as the graders and the plagiarism software go to work, we will discover over the next week if all this has made any difference. We discovered three serious cases last year, two egregious beyond belief. I hope all my extra efforts this year lead to zero cases. And that if it does not, and I again find myself in front of you, that all this extra work means we can more efficiently and fairly penalize the offenders.

Yours,

Andrew