Argumentation Lesson – Evaluating Evidence and Online Sources (SCOM 2710)

 

Evaluating Evidence and Digital Literacy

SCOM 2710 Argumentation Lesson, Posted by Keren Wang, updated 2024

 

Overview

This week we will be focusing on understanding and evaluating evidence, a crucial aspect of constructing persuasive arguments. It explains how evidence interacts with values, and presents general tests for assessing the quality of evidence. We will also be learning how to locate and evaluate various sources of evidence, guiding you on choosing reliable information from books, periodicals, websites, and more. The chapter emphasizes the importance of digital literacy and critical evaluation of different types of sources.

Ad Fontes Media Bias Chart 6.0
Media Bias Chart published by Ad Fontes Media, 2020. Fact-checking always lags behind the emergence of new biased sources of information.

Understanding Evidence

Evidence and Values

Evidence is always interpreted through personal and cultural values. Here are some examples of how values shape our interpretation of evidence:

  • Artificial Intelligence in Employment: Evidence showing that AI can improve productivity and efficiency is interpreted by some as a positive development for economic growth, whereas others see it as a threat to jobs, fearing mass unemployment and widening economic inequality.
  • Genetic Editing (CRISPR): Evidence about the successful application of CRISPR to edit genes in humans can be seen as a revolutionary medical advancement that will eliminate hereditary diseases, or as a dangerous intervention with unknown ethical and social consequences.
  • Universal Basic Income (UBI): Evidence from trials of Universal Basic Income might show improvements in mental health and poverty reduction, which is interpreted positively by proponents as proof of the policy’s benefits. However, others might see it as fostering a culture of dependency or as economically unviable, depending on their economic values.
  • Police Surveillance Technology: Evidence supporting the use of facial recognition and other surveillance technology to improve public safety can be seen as a way to effectively reduce crime. On the other hand, it is interpreted by others as a serious threat to privacy and civil liberties, especially in communities that may be disproportionately targeted.
  • Vaccination and Public Health: Evidence showing the efficacy of mandatory vaccination for school children may be interpreted as essential for public safety by some individuals, while others may view it as intrusive government overreach or distrust the pharmaceutical industry.
Robotic Sculpture at MIT Media Lab (photo by Keren Wang 2015)
Evidence that suggests a major breakthrough in general artificial intelligence may be seen either as a major technological advancement or as ethically problematic depending on the individual’s values.

General Tests of Evidence

Herrick introduces seven general tests of evidence that can help evaluate whether evidence used in an argument is reliable, credible, and sufficient to support a conclusion. These tests provide a comprehensive approach to assessing the quality of evidence. Here’s a detailed breakdown:

  1. Accessibility: Is the Evidence Available?Evidence that is accessible and open to scrutiny is generally considered more reliable.Example: A public health official cites the number of COVID-19 cases reported by the Centers for Disease Control and Prevention (CDC). This evidence is accessible because the CDC publishes its data on a website that anyone can visit and verify.

    Counterexample: Someone claims that the government has “secret documents” showing proof of extraterrestrial contact. Since these alleged documents are not accessible for review, the claim fails the test of accessibility.

  2. Credibility: Is the Source of the Evidence Reliable?This can depend on the reputation of the author or organization providing the evidence, as well as whether the source has the appropriate credentials or expertise.Example: A research paper on the safety of vaccines authored by a team of immunologists and published in The New England Journal of Medicine is credible due to the expertise of the authors and the reputation of the journal.

    Counterexample: A claim about vaccine dangers coming from an anonymous social media post lacks credibility because the author’s qualifications are unknown, and the post does not have any verifiable authority.

  3. Internal Consistency: Does the Evidence Contradict Itself?Evidence should not contradict itself. If evidence is self-contradictory, it weakens the argument and creates doubt regarding its reliability.Example: A government report on unemployment must consistently present the same statistics throughout the report. If one section states an unemployment rate of 6% and another section states 8% without clarification, the evidence lacks internal consistency.
  4. External Consistency: Does the Evidence Contradict Other Evidence?Evidence that sharply contradicts most other reputable evidence is often seen as unreliable.Example: A study on climate change that finds rising global temperatures should align with the majority of climate research from other scientific bodies such as NASA, the IPCC, and NOAA.
  5. Recency: Is the Evidence Up to Date?Evidence that has been superseded by more recent findings may no longer be applicable.Example: Citing a 2023 meta-analysis on the effectiveness of renewable energy technologies is preferable to citing a study from 2001, as the newer study will have taken into account technological advancements.
  6. Relevance: Does the Evidence Bear on the Conclusion?Evidence that does not directly relate to the argument is not helpful.Example: If a speaker argues for increasing the minimum wage, citing research that shows increased minimum wages boost consumer spending is relevant because it directly supports the argument.
  7. Adequacy: Is the Evidence Sufficient to Support Its Claim?Adequate evidence means having enough quality evidence to convincingly support the claim being made.Example: If you are trying to prove that sugary drinks contribute to obesity, providing multiple studies from different credible sources, statistics on consumption rates, and expert testimony would collectively provide adequate evidence to support your claim.

Sources of Evidence

Herrick, Chapter 7 outlines different types of sources for evidence and their respective strengths and limitations:

  • Periodicals: These include scholarly journals, special-interest magazines, and news/commentary publications. Scholarly journals are considered the most reliable due to their rigorous editorial and peer-review process.
    • Scholarly journals are considered the gold standard due to their peer-review process, while special-interest publications and news magazines can offer accessible information but with less depth and more bias. They can be easily accessed via university libraries.
  • Books: Books can be useful sources of in-depth information, but it is important to consider the author’s credentials, publication date, and the type of publisher.
  • Documentaries: These can offer reliable insights but may be influenced by commercial interests or biases.
  • The Internet: Offers vast information, but requires critical assessment for credibility. Websites with recognizable authors and credible organizations are generally more reliable. Digital literacy has become an essential skill for identifying and evaluating online sources.

Digital Literacy

Digital Literacy refers to the ability to effectively navigate, evaluate, and utilize online information. Digital literacy is more than simply being able to use technology; it is about understanding how to critically evaluate the veracity and quality of digital content and its sources. Key aspects of digital literacy include:

  1. Critical Evaluation of Sources: Not all websites are created equal, and digital literacy involves determining whether an online source is credible, up-to-date, and relevant. It also requires recognizing the purpose of the content—whether it aims to inform, persuade, entertain, or mislead.
  2. Understanding Bias and Intent: It is important to understand the motives behind the creation of digital content. Websites often have particular political, social, or commercial agendas, and digital literacy involves identifying these biases. For example, a blog promoting dietary supplements might not be objective if it’s sponsored by a company that sells such products.
  3. Verification of Facts: Digital literacy requires cross-referencing information found online with multiple reliable sources. This helps verify facts and avoid falling for misinformation or “fake news.” For instance, a claim about a health benefit found on social media should be verified through medical publications or government health websites.
  4. Awareness of Digital Manipulation: The internet includes not only text but also images, videos, and audio clips, many of which may be digitally altered. Digital literacy involves assessing whether visual or multimedia evidence has been manipulated to present a biased narrative.
  5. Navigating Information Overload: The sheer volume of information available online can be overwhelming. Being digitally literate means knowing how to sift through large amounts of data to find high-quality, relevant information. This involves using effective search terms, recognizing authoritative domains (e.g., “.gov” or “.edu”), and understanding how search engine algorithms may prioritize certain content.
  6. Digital Security and Privacy: Digital literacy also includes understanding how to protect one’s privacy online and recognizing secure websites. For example, a digitally literate individual would know to look for “https://” at the beginning of a URL as an indicator of a secure website.

Example of Digital Literacy in Practice: Suppose you are researching the benefits of electric vehicles (EVs). A digitally literate approach would involve consulting a mix of sources, including reputable news organizations (e.g., Associated Press, Reuters), trusted independent technical professional organizations or public agencies (e.g., IEEE, European Alternative Fuels Observatory), and peer-reviewed journals (Energies, Transport Reviews, Journal of Power Sources). It would also involve recognizing potential biases—such as an oil company-funded blog questioning the sustainability of EVs.

Evaluating Websites

Evaluating the credibility of websites is a critical component of digital literacy. The internet contains valuable information but also a lot of misleading or false content. Here are key considerations for evaluating websites:

Key Considerations for Evaluating Websites

  • Language and Content Quality:
    • Credible websites typically use a moderate and professional tone. They avoid extreme or sensational language that appeals to emotions rather than presenting facts.
    • Grammatical accuracy and proper punctuation are often indicators of a professional and reliable website. Sites riddled with typos or casual language may lack reliability.
    • Fact-based Content: Reliable websites provide references, links to original studies, or citations to support their claims.
    • Example: A health website like Mayo Clinic (www.mayoclinic.org) provides detailed health information, cites medical sources, and avoids sensational claims about treatments.
  • Authority of the Site Creator:
    • Consider who created the website. Recognized authorities (e.g., universities, government institutions, established news organizations) provide credible content.
    • Look for the author’s credentials. An article on medical treatments should ideally be authored by a healthcare professional or medical researcher, with appropriate qualifications listed.
    • Example: The American Medical Association’s website (www.ama-assn.org) is a trustworthy source for medical information because it is maintained by a reputable professional organization.
  • External Consistency:
    • External consistency is about comparing the information on the site with other reliable sources. A credible website should not present claims that contradict established knowledge.
    • Cross-referencing helps determine if the information presented aligns with mainstream consensus or is a fringe theory.
    • Example: If a website claims that climate change is not occurring, a comparison with multiple authoritative scientific sources (e.g., NASA, NOAA, IPCC) may reveal that the claim lacks external consistency and therefore credibility.
  • Objectivity and Bias:
    • Recognize the potential bias or purpose of a website. Websites created to sell a product, promote a political agenda, or advocate for a specific cause may present information in a skewed manner.
    • Lobbying organizations, for example, may present one-sided information to persuade rather than to inform.
    • Example: Greenpeace’s website (www.greenpeace.org) provides valuable information on environmental issues but is also advocating for specific policy changes. It is important to note that the content is aimed at activism and may include a biased perspective.
  • Currency of Information:
    • Up-to-date content is crucial, especially for topics like technology, health, or science. Websites should indicate the date the content was published or last updated.
    • Outdated information can mislead or provide inaccurate conclusions if more recent research contradicts earlier findings.
    • Example: A website discussing COVID-19 treatments that has not been updated since 2020 may not reflect recent advancements, making it less reliable for current information.
  • Security of the Website:
    • Secure websites often indicate greater credibility. Look for “https://” in the URL as a sign of secure data handling.
    • Trustworthy websites also typically have an “About Us” page that details their mission, authors, and organization’s background.
  • Cross-Referencing Sources:
    • A good practice in evaluating websites is to cross-check information with other reputable sources. If multiple authoritative sites support the same conclusion, the information is more likely to be accurate.
    • Use fact-checking websites such as Snopes (www.snopes.com) or Media Bias / Fact Check (mediabiasfactcheck.com)  to verify claims and their sources that seem suspicious.
  • Avoiding Clickbait and Sensationalism:
    • Clickbait headlines are designed to attract attention but often lack substance or reliable evidence. Reliable websites present headlines that are informative and factual rather than exaggerated or misleading.
    • Example: Compare a “clickbait” headline like “5 Ways Coffee Will Instantly Cure All Health Problems!” with a more measured one such as “Research Shows Potential Health Benefits of Moderate Coffee Consumption.” The latter is more likely to come from a reputable source.

 

Further Reading:

  • Baly, Ramy, Giovanni Da San Martino, James Glass, and Preslav Nakov. “We can detect your bias: Predicting the political ideology of news articles.” arXiv preprint arXiv:2010.05338 (2020).Chiang, Chun-Fang, and Brian Knight. “Media bias and influence: Evidence from newspaper endorsements.” The Review of economic studies 78, no. 3 (2011): 795-820.

    Finlayson, Alan. “YouTube and political ideologies: Technology, populism and rhetorical form.” Political Studies 70, no. 1 (2022): 62-80.

    Kulshrestha, Juhi, Motahhare Eslami, Johnnatan Messias, Muhammad Bilal Zafar, Saptarshi Ghosh, Krishna P. Gummadi, and Karrie Karahalios. “Search bias quantification: investigating political bias in social media and web search.” Information Retrieval Journal 22 (2019): 188-227.

    Li, Heidi Oi-Yee, Adrian Bailey, David Huynh, and James Chan. “YouTube as a source of information on COVID-19: a pandemic of misinformation?.” BMJ global health 5, no. 5 (2020): e002604.

    McGrew, Sarah. “Learning to evaluate: An intervention in civic online reasoning.” Computers & Education 145 (2020): 103711.

    Morstatter, Fred, Liang Wu, Uraz Yavanoglu, Stephen R. Corman, and Huan Liu. “Identifying framing bias in online news.” ACM Transactions on Social Computing 1, no. 2 (2018): 1-18.

    Pangrazio, Luci, and Julian Sefton-Green. “Digital rights, digital citizenship and digital literacy: What’s the difference?.” Journal of new approaches in educational research 10, no. 1 (2021): 15-27.

    Sobbrio, Francesco. “Indirect lobbying and media bias.” Quarterly Journal of Political Science 6 (2011): 3-4.

    Tinmaz, Hasan, Yoo-Taek Lee, Mina Fanea-Ivanovici, and Hasnan Baber. “A systematic review on digital literacy.” Smart Learning Environments 9, no. 1 (2022): 21.