Category Archives: Thoughts about SC200

Where are the men?

First review session of the year was today. I love it. Small(er) group teaching, lots and lots of interaction. You can see the whites of the eyes. You can see when they don’t get it–and best of all, you can see when the light bulbs come on. That is so gratifying.

4ib4K5xigThe major bummer: I forgot to take a photo of the group. In previous years, it’s been tragic. Today, there must have been almost 40 students. Best attendance ever. Camera Andrew? Wake up. Too busy with water, recovery from class, projection systems (that damn multitasking problem)….but I did observe the sex bias. It is always there. This time there were two blokes among the ?40. I am not sure what the class sex ratio is. But it is not 1 in 20 (maybe it is 9 in 20?). As a father of sons – and frankly, the most charming and smart young men you will ever meet – it is seriously sad that blokes don’t revise. I hate to generalize, but here goes: women work to achieve. Men assume they will.

They don’t. Somil, one of the class TAs this year, was in the top three in class last year. The next guy was in rank order #21. That’s an anecdote. but every year, I have had no options for male TAs. If I am lucky, one bloke will be up there. Most years, none are. That’s six anecdotes. Starting to sound like data.

Fall 2015: Kick-off

It’s the end of the last Friday of the summer break. Outside my office window, the streets are clogged with parental cars disgorging the class of 2019.

At this time of year, I always ponder what I am doing well on the course, and what needs improving.  Prompted by one of my memos to self from last year, and subsequent discussions with Larkin Hood from the Schreyer Teaching Institute, I have spent some of the week thinking about the rare but important problems in past years of plagiarism, integrity failure, and whispering in class. I recently learned that there are people that research undergraduate classroom behavior, and one of them thinks (p.470) that controlling for everything else, classroom incivility (the technical term for bad student behavior) is worst in classes – get this – that have non-majors enrolled to meet graduation requirements in which the teacher is trying to infuse the class with conceptual material and attempts to teach critical thinking.

Oh my.

Conclusions to conclude

Lessons from 2014 for 2015?

  • The fundamentals are sound
  • The attendance algorithm works
  • Try to figure a way to work some class activities into classroom time. Hard to do for big classes, but evidently it can be done*.
  • Figure out how to stop the whispering in class. It has to stop for so many reasons (1, 2, 3, 4, 5…)*.
  • Obsessively check Angel test settings. Especially for the final exam.
  • Raise blog standards. I think the bar is set about right. Get more students to hit that. Point out to them that it is in fact easier than getting a good test score. I wonder if one way to do it might be to get them to do some peer grading of the first blog period. Show them more examples in class of excellent practice. Grade the first blog period harder.
  • Keep the restriction on making this a course of Freshmen. More impact. Fewer cynical students.
  • Important stress-reduction tip Andrew: do not get on an airplane during semester (1, 2).
  • A perpetual source angst for a small minority of students is the blog grading. Some students try harder in a subsequent blog period but get about the same grade. When I investigate, it is almost always fair. Get the same grader to grade the same students each time? Get the graders in the later blog periods to look at the comments and scores from the earlier periods? Make sure not generosity is going on in the grading of the first period. Try to teach students that hours/effort is not what counts. It’s well directed effort and hours.
  • Do a class project (e.g. test the cell phone hypothesis?).
  • Do a few pop quizzes early on which test run the first parts of the class tests better (general material).
  • More use of phone polls to reinforce material in class, especially early on.
  • Set up a glossary of words – something students can post to so I can see what they can’t understand.
  • Produce a final list of all the key concepts covered for review purposes. Could give this out at the beginning of the year (or put it in the syllabus). It could work as a study guide.
  • Consider pop-quiz like homework. They could hand it in next class for attendance? Could use old exams; make the answers available on Angel etc. This might work for review just before exams.
  • Throw more questions at the audience. Get the mike and show it in people’s faces? Scary stuff for the students, but if gently handled at the beginning, might work.
  • Figure some ways beyond tests and pop quizzes to do hands on learning*.
  • Do a class on Evil.

The plan is try a class of nearly 400 in 2015. This to scale the impact. But to stay sane, I’ll need to do various things.

  • Get a class email account set up SC200@psu.edu – which staff assistant Monica monitors and deals with as much as she can, re-routing as much else to the TAs as possible.
  • Get another three blog graders. Make the most experienced of the six responsible for coordination, and quality control.
  • Get an extra TA to help the students (total three).
  • Get a side-kick or two to run review sessions (one of the graders?). They review sessions are important, but I can’t do twice as many.

*Andrew: well before Fall 2015, ask the Schreyer Institute for Teaching Excellence for suggestions

Being there

Part of me thinks that students are adults and if they don’t want to come to class, that’s up to them. But another part of me thinks that students are kids who need to be helped to help themselves, and a lot say in the feedback they wished they come more…

So, each year I agonize about the best way to make them come to class. This year, I tried a new approach – I tied a whole grade to attendance and only gave them that when they hit seven pop quizzes (I take attendance only at pop quizzes). The pop quizzes come at random so they have to attend a lot of classes to hit seven. I did ten in total.

I didn’t think the system had worked, not least because of the way it looked in class.

2014-10-30 15.16.21

2014-10-30 15.16.32

I took these shots during the pop quiz on October 30. Every seat should be filled….

Turned out 96% of the students who got to the end of the course hit that attendance requirement. And now that I have got down and looked at the actual data from the last four years, this was definitely the best year. Attendance dropped to the mid-60% in the last two pop quizzes (post-Thanksgiving), but in 2013, it dropped to 50%, and in 2013, 2012 and 2011, the attendance sag happened maybe a month earlier.

So unless I can think of a better way of going, I think this algorithm is probably a keeper. I could shift the quizzes later in semester, but I need to do them early on to reinforce class material. Maybe raise the bar to 9 of 12 quizzes so I can do a few more at the end?

Of professors and physicians

doctorThere is an in-built confirmation bias in medicine. If the patient gets better, doctor assumes s/he healed the patient. If the patient dies, doctor assumes the patient was really sick and beyond help. This bias means doctors have gone on with useless or even harmful practices for years (or even centuries).

professor3Pondering the SRTE‘s, I suddenly realized I was thinking just the same way: when students said they learned things, I attributed that to my fantastic pedagogy. When students said they learned nothing — or worse, showed me that they learned stuff that was wrong — I assumed those students hadn’t come to class, or didn’t listen, or were unteachable…

Mmmm…. Well, Andrew, follow the logic you taught in class…

Ok. For sure, customer satisfaction surveys could not get medicine out if its confirmation bias. So SRTEs can not get we professors out of ours. What’s needed instead is the equivalent of the randomized control trials which frequently save medicine from itself (or more correctly, save patients from the practitioners). I need to enroll a class and then at the last minute, refuse entry to a randomly chosen half of the students so they go off and do another course. Which group would go on to best achieve the course objectives?

The endpoint of such a study is not easily defined, let alone measured. Performance on the final exam is irrelevant (and is in any case a softer-than-soft endpoint). The real test is whether a difference could be detected years later, long after graduation.

Good thing we can’t do that experiment.

What are the most important things you learned?

My other favorite answers to this question on the 2014 course evaluation questionnaire (excluding those similar to the 2013 answers):

  • I now have an understanding of what contributes to global warming
  • I learned how to think harder than I ever had to before
  • I learned to question everything
  • I learned that science does apply in real life, who knew?
  • Scientists and science can actually be interesting to learn about and discover
  • Some of the things you wouldn’t think were science actually are
  • how to connect information learned in class to everyday occurrences/news outside of class
  • I realized I do not despise science as much as I thought
  • the difference between science and faith
  • Risk evaluation. Seriously, I’ll stop fearing drowning in a pool or dying in a plane crash
  • don’t accept something just because it is ‘established’
  • our intuition is lousy
  • I learned specifics on things that are normally not given a second thought e.g. how we learned smoking is bad for you, how vaccines work etc.
  • How negative science is
  • That not all science is dull
  • Science is beyond the normal science classes most of us are used to
  • Animals can be gay and aliens most likely exist
  • The difference between x and y.
  • I learned I actually enjoy science!
  • do not believe everything you read in the media.

Those  comments, like last year’s, make me glad I teach this course rather than teaching biology to biologists, important though that is. This feels much more impactful.

By way of balance, lessons the students learned that I worry about:

  • We know nothing about the world and science is entirely theories that are either correct or wrong, but we’ll never know if it is or not because nothing can be proven correct, so it all seems kinda pointless
  • I am not a science major I will never be a science major I now never plan on being a science major
  • I didn’t learn anything

Most concerning of all was the comment: The Australian accent is charming.  If the SRTE‘s weren’t anonymous, that student would be a straight fail.

The responsibility

For the first time, this year’s class was almost entirely first semester freshmen. Much to my surprise, I felt an immense responsibility to set these students off on their Penn State careers the right way. That meant trying to impart more than ‘just’ course-related skills like critical thinking, the evaluation of evidence, distinguishing a reasoned argument from baloney, and empowering students to question what they hear from professionals, professors, peers and parents. It meant trying to get across additional things like study habits and time management, honesty and integrity, the ability to learn from failure (or low grades) and the importance of class room discipline. In email correspondence, it often also meant advocating the merits of developing a decent work ethic; or put another way, how to get an A without trying to bullshit or bully the professor.

habitsHow well did I do on any of that? I’m not sure. There were signs of serious integrity-failures. There were three plagiarism cases, despite my best efforts; I hope I handled these in a way that taught the students to never do it again but without my ruining their College careers from the get-go. I was ruthless on people who missed deadlines. I talked endlessly about the need to manage time. I am not sure if that worked. Many still left their blog posts to the last minute and it showed, just as I told them it would. I talked of the benefits of attending the voluntary review sessions. Most did not bother. I implored students to come to class and played with attendance grade algorithms to try to make that happen (I’ll evaluate whether that worked in forthcoming post). I stood up to the pleas, begging, and bullshit demands for a higher grade. I truly believe students need to earn their grades. Anything else cheats them (for the most part, life is not like that) and all those others who work honestly and hard. But I also tried to be sympathetic when students with dire personal problems reached out. It’s amazing how a little understanding from faculty, the right word at the right time, can transform a life’s direction.

The most useful question on my SRTE questionnaire is: What three things have you learned? In terms of these non-content ambitions for the course, here’s what students’ said they’d learned.

  • time management [several comments along these lines]
  • I learned how much better my learning experience is by being mindful of people around me
  • don’t sit by someone who talks
  • the impact of talking in class
  • read the questions in tests carefully
  • Andrew isn’t as scary as he seems
  • better note taking [several comments along these lines]
  • how to blog better [many comments along these lines]
  • writing takes practice [oh so true!!]
  • the importance of having an engaging instructor who was willing to help and wanted his students to succeed
  • the importance of avoiding procrastination
  • learn from mistakes
  • different study methods
  • always go to class [many comments along these lines]
  • pay attention and listen to what the Professor says
  • I learned to write better
  • that it takes work to improve your grades
  • that its ok to need help but you aren’t going to get it unless you ask (the revision sessions proved this)
  • improvement is key if you want to learn
  • how to right (sic) fantastic blogs
  • don’t just focus on the examples, but the concepts within
  • the importance of getting help when you need it instead of sitting around watching your grade plummet
  • take advantage of the TAs [I assume this is meant in a good way…]
  • the importance of weeding out people who are talking in class at the beginning of semester [said as a criticism of what I did not do well enough]
  • 1. I will not automatically succeed. 2. If I put in the work, I will succeed. 3. I don’t need 8 hours of sleep.

Most of which is gratifying. But from the responses to the question ‘What changes would improve your learning?’, it is clear I have some way to go on getting students to seize control of their own learning. That’s surely one of the most important life skills we need to get across in Higher Education. After graduation, it’s rare to have someone offering homework or review sessions or changing the algorithm to suit your time management ability. But in meaningful work and for a meaningful life, learning remains critical — indeed it might be more critical than at College. How do we better encourage students to self-teach?

The student evaluations….

ANGEL_SRTECourse and faculty evaluations come mainly in the form of Student Rating of Teaching Effectiveness. This year, my scores are up by 0.2-0.5 units (on the 7 point scale), reversing the general decline I’ve experienced over the last few years. Since I can not standardize myself (I hope I change), I bench mark the whole thing to the otherwise very dull question of the clarity of the syllabus (my syllabus is the same each year). I think of this as a measure of class orneriness. Syllabus clarity got a mean score of 6.22, up from the 5.75 of last year. So I guess the class of 2014 was 8% less ornery than last year. But since that jump was one of the biggest, it might mean that in real terms (i.e. orneriness-adjusted), the class is actually a little less satisfied than last year.

Or it might mean that I am making WAY too much of these scores, which above a certain threshold (4?), really should be taken with a pinch of salt. It’s the comments the student’s write that are valuable.

When I started the course in 2010, I wanted to hear what all the students thought, so I bribed the class with extra credit if they could get an SRTE return rate of over 85%. This resulted in my course getting one of the highest return rates on campus (c. 90%). This year I decided to do away with the bribe. That resulted in a 47% return rate. I thought the rate would be lower, especially since I only urged the students to do the SRTEs once, and then to a class room near half empty. Staring at the numbers and the comments, I don’t see any major differences from previous years. I guess the motivated and the disgruntled do the STREs whatever; I assume the missing 40% had nothing much to say. In fact, in the spirit of Christmas good cheer, I will assume they were all neutral to moderately happy. The group I’d like to have heard from were the 12 who dropped the course. They never tell you why. I hope it was not to do with me or SC200.

rate-my-professor1My own feeling is that students can’t really assess how valuable a course was, or how good their teacher, until many years later. That’s particularly so for Freshmen in their first semester. I’d love to hear from any of them as their Penn State careers progress. Better yet, after graduation. Cocktails in Zolas anyone? What was good? What could I do better?

theDryMartini02_bg

The bottom line for 2014

The class average: 88% (B+). Four of the 14 students with an A are four scored >100% through extra credit. In the absence of extra credit, the highest score would have been 98%.

Of the 198 students who started the semester, 186 made it to the end in the sense of getting a grade for the course on their transcript. Of those 186, 38% got some kind of an A, 71% got a B+ or better, and 90% of students got a B or better. Compared to last year, fewer A’s but way more A-‘s and B+’s. This despite the fact that this year I abandoned the non-merit based extra credit I uneasily dispensed in previous years.

It is difficult to calculate the impact of my final exam cock-up. It looks to me like one person passed who might not have otherwise done so, and undoubtedly some others went a grade higher than they deserved (Merry Christmas). But not many. The difference between the 100% I gave out and the 80-85% that would likely have been the class average exam score is 15%. The exam is worth 20% of the final grade – so the average contribution of my largess is just 3%. Not much in the scheme of things, and less than bribery I used in previous years to encourage students to come to class and fill out the end of course questionnaire. And like that bribery, the cock-up gave me a grade buffer. I could say to students wanting me to bump them up a grade that they were only close because of my mistake.

Of course, there is simply no way to know what to make of this grade distribution. Am I setting the bar too low, too high or just right? It is one of the mysteries of high education.