During the first day of class, Andrew told us if we did not want to think hard we should not take his class. He also said one of his goals for this semester was to make SC200 a class we would never forget. I would say Andrew has succeeded in both, making us think critically about problems and providing us with material we will remember for the rest of our college years and beyond.
Scrolling through Facebook, you tend to see studies done that are supposedly life changing. Before taking SC200 I would often skim through the article and believe the results of the study without questioning who conducted the study and how it was conducted. After taking this course, I now look at these articles on my Facebook timeline differently. Just because an article appears on Buzzfeed does not make it true. If they say you can put honey on your feet to cure cancer I should not automatically believe it. I now recognize not every study is conducted well, and even if it is, that doesn’t mean it is correct. The biggest reason how SC200 has changed my life is how I react and evaluate studies.
Say I am reading that article I mentioned earlier about honey and cancer. Now I ask myself, is it observational or is it experimental. If it is observational there are tons of confounding variables possibly lurking. I would also ask myself could the results be to reverse causality. If it is an experimental study, I worry about how the study is conducted? Could the people involved in the study be biased? For example, if they were being tested on how fast they walk, would they walk faster or slower because they know they are being tested on this? How many people are in the study? The more people, the better the study. Is it random? It ought to be. Because randomization greatly reduces the likelihood of chance, although chance is always going to be a factor.
So say the study from the article is well conducted, features tons of people, is randomized, maybe even has a placebo, and does a good job of eliminating confounding variables. Now what do we worry about? Well I mentioned chance earlier. This is another crucial takeaway from the class. Even the most well conducted experiments could be due to chance. We learned about prayer being an effective healing method earlier in the class. This study was conducted very, very well. Except its results were proved to be due to chance after many studies were conducted. This is why repeating studies over and over again are crucial. If you get a false positive or a false negative you’ll know with many studies. A meta-analysis, which compiles sometimes 1,000’s of studies together, is the ultimate way of determining if a hypothesis is correct or not. The meta-analysis was able to disprove the link between prayer and healing.
Another thing I know look at is who did the study. Before I would just assume an impartial scientists did the study. Why is this important? It’s simple. If Coke published a study saying it is good for you, they reap the benefits of this. They’d get a boost of sales because of this study, so they would do everything in their power to make sure the studies benefit them. If the results do not benefit them, they could not publish them and hide them from the public until they get a false positive to show to the public. This is the File Draw Problem. This is extremely evident in studies based on medicine. This is also a reason why I also now look at scientific journals for studies and not Facebook, because these journals are peer reviewed. By publishing a study, the scientists leave the door open for other scientists to review their studies and to see if the results are true and the experiment was conducted well.
The last this class has taught me that has changed my life has been having an open mind. We suffer from motivated reasoning, which is having preconceived notions no matter what. Whenever I read anything now, whether it be politics or science, I try my best to eliminate my confirmation bias. If I thought something was wrong, I would have done anything in my power to find a reason to reject the data, or simply forget it, no matter how convincing the data is. This course has taught me to evaluate everything I disagree with, with an open mind before forming a judgment.
SC200 has certainly changed my life for the better. As I stated, I now have knowledge of chance, the cons and props of different types of studies, false negatives/positives, meta-analysis, randomization, the File Draw Problem, peer review, motivated reasoning, and confirmation bias. Before I knew about any of this I wouldn’t know how to properly evaluate something I read, and as a result, would foolishly believe any study. By knowing about all these topics, now I can properly react and evaluate studies.
Charles — Great post; an excellent summary of much of the course. I’m glad the messages hit home. But better, I’m glad you are thinking hard about what you see on Facebook. Science in the Age of Selfies was the title of a recent PNAS paper. It argued that there was a lot of marketing going on in science, and there sure is. But the marketing of false and even dangerous stuff to the general public is going to be a bigger challenge for your generation than anything that’s come before. Remain skeptical; think critically, and we’ll all be better off.
Good luck for the future. Andrew