3. Representing and Interpreting Data

At the beginning of this unit, we noted that the same data can be represented in drastically different ways, which in turn can be used to support contrary interpretations. In his classic book How to Lie with Statistics, Huff (1954) shares numerous examples of manipulating statistical analysis to support conclusions that are different from or even opposite to what the data really says. Huff’s book (1954) also exposes a number of mathematical “tricks” that are used in distorted interpretation of data; these tricks include unrepresentative samples, selective use of mean and median as “average,” inadequate sample size, and ignoring the limitations of tests.

Although scientists and engineers often promote their research as objective, social scientists who have studied the actual process of scientific and engineering research argue that in the actual research world, pure objectivity is more of a myth than a reality (Harding, 1992). According to this argument, no researchers can be completely objective in the sense that their choices of questions, methods of inquiry, and the ways in which they report research findings all follow the “one and only correct way,” which is not influenced by any societal or individual values, concerns, and interests. Therefore, a more “objective” statement about scientific and engineering research might recognize that as human beings, we are all partial, and furthermore, epistemic and ethical values are already embedded in research activities. After all, research is a communal and institutionalized activity. To begin with, our selection of research questions is often impacted by funding opportunities, hotspots in the field, and previous research findings. Also, research design and data collection are influenced by the background and training of individual researchers, the available materials and equipment, and sometimes, by institutional factors such as the pressure to publish. In addition, a researcher’s intellectual interests and personal commitments might play a part in shaping the direction of her research. For example, researchers who are concerned about public health might choose to focus on understanding an epidemic rather than a rare genetic disease. The point is, instead of holding on to the illusion of “pure objectivity,” we can more effectively promote ethical research by recognizing and examining the various institutional, individual, and technical factors that collectively shape the generation, reporting, and use of research data.

 

3.1 Conflict of interest

Researchers often play multiple roles at the same time. These roles might include university employees, teachers, mentors, and consultants to government and industry. Each of these roles might bring with it particular obligations to different groups, and at times these obligations might compete and conflict with each other. For example, the great commercial value of biomedical research has led to numerous issues of conflict of interest. As a result, the Federation of American Societies for Experimental Biology (FASEB) organized a conference in 2005 “to allow investigators to consider and respond to serious challenges involving conflict of interest in biomedical research” (FASEB, 2006). In a report published after the conference, FASEB (2006) defines conflict of interest as “a condition in which a primary interest (institutional responsibilities for research and education) is in conflict (whether real or perceived) with a secondary interest (such as financial gain).”

A conflict of interest raises ethical questions because the interests of one or more entities involved in the situation might be unfairly compromised. For example, a university professor who also consults for a company might be required by the latter to delay the publication of her research findings, so that the company could enjoy exclusive access to the findings so as to retain a competitive advantage in the market. While delaying the publication can advance the economic interest of the professor’s industrial partner, this action might negatively impact the image of her university. Moreover, delaying the publication will  compromise the entire academic community’s interest in learning about most recent research findings.

A conflict of interest might bring about other, more severe consequences. In particularly, it is a threat to the integrity of research data and its interpretation. For example, the tobacco industry has sponsored research that attempted to confuse the public about the health risks of smoking (Bero, 2005). In a recent article, a member of the Rockefeller Family Fund openly criticized ExxonMobil for funding research that continuously challenges the existence of climate change, while the company’s own scientists and engineers had internally reported evidence for climate change and explained the contribution of fossil fuel to climate change decades ago (Kaiser and Wasserman, 2016). Besides funding research that presents findings favorable to the industry, corporations sometimes use their financial influence to suppress research findings that are detrimental to their interests. Psychiatrist David Healy personally experienced interference from the pharmaceutical industry in his academic appointment. Healy had been offered a position at University of Toronto. However, after one of his presentations implied the potential hazard of an antidepressant to patients’ mental health, the university rescinded the job offer under the pressure of an industrial sponsor, a pharmaceutical company that makes the very antidepressant questioned by Healy’s research (Healy, 2002).

Not every conflict of interest has to be eliminated. However, in almost every case, researchers who perceive a conflict of interest have an ethical duty to disclose the situation to relevant entities, such as their academic institutions and the industrial partners. The FASEB (2006) provides several suggestions for academic institutions and researchers to handle conflict of interest. For academic institutions, FASEB recommends establishing formal mechanisms to review their employees’ collaboration with the industry. For researchers, FASEB urges them to “have access to, and be involved in the analysis and/or interpretation of all data generated in the research (FASEB, 2006).” FASEB also advises researchers not to “enter into agreements with companies that prevent publication of research results (FASEB, 2006).”

 

(This case is abridged from “All in the Interpretation,” a case authored by Simil Raghavan. The full case study is published at the National Academy of Engineering Online Ethics Center. Use of this case is permitted by the National Academy of Engineering Online Ethics Center.)

 

All in the interpretation

Kate is a graduate student in Professor Brigg’s lab. She started a project examining the effects of certain video games in children during her first year of graduate school. She knows that some of the funding for her project comes from a video game manufacturer, but the money does not give the company control over how the research is conducted, and she believes she has been careful not to let the source of funds influence her project design and data collection.

Kate has collected all of the data for her project, and she has been carefully examining the trends. Looking back, she might have changed some of her data-collection methods if she could do it over again; but she knows that is the nature of research, and that lessons learned in one project generate new questions to ask in the future. She is excited to see a clear trend in her data that indicates a positive effect of educational video games, but the effect washes out after about a year or two, and she is unsure how to interpret it. She creates a rough draft of a paper that carefully outlines all of her analyses and gives it to Dr. Brigg for review. Later in his office, Dr. Brigg explains that the “Results and Conclusions” section of her paper is very weak. He says that she does not make a strong case for the importance of her research, and that the quality of the journal where her paper will be published depends largely on her ability to interpret the data. “I’m not saying to leave out data,” he says, “but the story you tell about the data is at least as, if not more, important than the data themselves.”

Kate knows that research papers are rarely air-tight. In fact, members of her lab will often spend lab meetings ripping apart a paper from another group in order to stimulate discussion about the author’s conclusions and generate ideas for future research. She feels she must choose a black or white stance in her interpretation of the effects of gaming in order to create a strong paper. She also knows that if she emphasizes the positive effects of the games, she could easily write another grant to the video game manufacturer to study the later wash-out period with a high probability of funding.

 

Questions for Case Analysis

  1. What ethical problems can you identify in this case?
  2. What role does “conflict of interest” play in this situation?