Like authority and bias, a source’s commitment to accuracy depends on what it is trying to accomplish. Scholarship claims to present an accurate description, derived from systematic study, of how some aspect of the world works. Standards for and expectations of accuracy are necessarily high. News also claims to present a picture of reality that is unbiased as possible. While not as strict as scholarship, editorial standards strive to maintain a high degree of ruth and objectivity. Sources that advocate for something, on the other hand, are designed to present a biased perspective. That doesn’t necessarily mean such sources are bad, but a commitment to accuracy depends on the standards of the intended audience. If they’re low, contradictory or inconvenient evidence might be ignored.
Accuracy is paramount in academic research, so let’s use scholarship as the standard for how authority is constructed. Consider the following:
Authority (For more information, click here.)
- Is the author, as an individual or corporate entity, highly regarded and widely accepted? What is his/her/its track record when it comes to providing reliable unbiased information? Is it the Bureau of Labor Statistics, a well-known government agency? Is it the Journal of Biochemistry, a topic journal in the field? Is it The New York Times, a respected newspaper? Or is it your uncle’s Facebook page?
- Does the source have the requisite knowledge and skill to discuss the topic? Expertise matters. A trained virologist arguing against the causal link between autism and vaccines deserves more trust than a random blogger.
Bias (For more information, click here.)
- Does the source advocate a point of view? Claims made by political parties, politicians/candidates, lobbying groups, corporations, salespeople, and other advocates may be true, but they can be self-serving. Polls may be intentionally rigged or data may be cherry-picked to achieve a particular result. As a result, they deserve greater scrutiny. Always compare and corroborate their claims with other sources to determine if they’re consistent with those in other information sources.
Methodology
- Does the source “show its work”? Scholars and scientists explain—in considerable detail—how they arrived at their conclusions. It’s a critical convention of scholarship. How that work is shown depends on the discipline. To describe its experimental design, scholarship in the sciences often follows the IMRAD format (i.e. Introduction, Method, Results, Analysis, and Discussion). Scholarship in the arts and humanities also “shows its work.” Problems are defined, analytical methods are described, evidence is collected, and results are presented. But the separation between sections is less distinct. (NOTE: These are ideal descriptions of how scholarship structures articles. There will always be exceptions.)
- For an experimental study, what method is used to collect the data?
- Is the sample representative? It’s impossible to study every instance of some phenomenon. Representative means that the subset used in the study represents, with a degree of accuracy, the broader population. To that end, it’s also important that the sample is random. Random means that everyone or everything in a population has an equal opportunity of being included in the study. Some news organizations and Web sites conduct “unscientific” polls with their viewers or visitors. Respondents self-select; they are not chosen randomly. These results are meaningless. Random and representative samples prevents subjects from being cherry-picked to achieve specific results. Large sample sizes are also useful, as they increase confidence and precision.
- Is there a control group? Science investigates how one variable is related to another. To establish this relationship, procedure requires a “control” to provide a valid basis for comparison. For example, in tests of new drugs one group gets a placebo, with no active ingredients, to provide a point of comparison with the group that gets the actual drug.
- What evidence is used to support claims? Is it scholarship? News? Unsubstantiated information? A more important, but more complicated, question might be – How is the evidence used? If a journal article cites a personal blog, is there reason for it? Determining if that reason is appropriate requires knowledge and experience in your discipline. As you progress in your studies, your ability to evaluate source use will increase.
- Have results been replicated in similar studies? Do other researchers agree, or do they come up with contrary findings? This requires a comprehensive review of the literature. Your ability to determine will come with knowledge and experience.
Deviations from the standard set by scholarship don’t always indicate potential inaccuracies; just as adherence to it doesn’t automatically guarantee complete accuracy. You must evaluate every message, holistically. Also, look at how it constructs authority and limits bias.
Adapted from The Battle of the Experts. (n.d.). Retrieved December 12, 2015 from Annenberg Classroom: http://www.annenbergclassroom.org/Files/Documents/LessonPlans/BattleoftheExperts/student.handout.6.testing.evidence.pdf