Register to comment and receive updates on new episodes!
Quick Reg!

one moment please...


Have a Question? Ask Dr. Bo! Your Question Might Be Turned Into an Episode!
Blog Home Tags:
media
science
sources
trust
Print

A Guide for Trusting Sources of Science, for the Non-Scientist

image loading... by Bo Bennett, PhD, Social Scientist, Business Consultant
posted Monday Apr 27, 2015 12:00 AM

image loading...

Bo Bennett, PhD

Social Scientist, Business Consultant

About Bo Bennett, PhD

Read all about me at http://www.bobennett.com.

It is no secret that much of the information found on the Internet today is strongly influenced by its monetary value, or to put another way, how many clicks an article can get. Unfortunately for science and humanity, accuracy, journalistic integrity, and truth are of far less monetary value than shock, emotional appeal, and fear. As a result, if scientific information does manage to make its way into the mainstream media (assuming nothing is going on with the Kardashian's that day), it is often a greatly modified version of the information, twisted by biased media sources with far greater allegiances to politics or the bottom line than science. Assuming a media source does prioritize scientific integrity, the lack of scientific education and knowledge could lead to misreporting the facts. Despite these seemingly insurmountable problems, there are ways that you can get the most accurate information possible on scientific issues, even if you don't know much about science.

For those of you who do know a bit about science and how scientific information is published, you may think that peer-reviewed journals are the definitive source of scientific information. In a way they are, but in other ways, they are not. Jeffery Beall maintains a comprehensive list of academic journals that range from questionable quality to outright pseudo-scientific. Just because something is published in a journal, does not make the information scientific. Anyone can start a journal, and reviews by peers of quacks have no scientific value. For a perfect example, check out the Journal of Creation that "strives to publish papers that promote the development of rigorously logical biblically-consistent models in various areas." Not only do they not mention the goal of being scientifically valid, but they specifically state that "the journal covers a wide spectrum of studies, not just science." Further, scientific papers can mislead non-scientists who don't have a firm grasp of the scientific process, methodologies used, and the limitations associated with the methodologies. For example, just because one study reports significant findings, it does not mean that those findings are indicative of a scientific discovery—it could be a flawed study or simply a statistical outlier (i.e., a study that shows significant results based on probability alone). For these reasons, the non-scientist is generally better off using scientifically-credible sources geared towards the general public.

While I am tempted to provide a list of sources, these would only be my opinion and certainly leave out many great sources of which I am not aware. Each specific scientific discipline has a myriad of highly credible sources, so I couldn't begin to do such a comprehensive list justice. I am better off giving you a few rules of thumb to assess the accuracy of your own sources. So here they are.

Is the source a scientist in the field he or she is providing information about? Be skeptical of generic bloggers/reporters/commentators with no academic credentials in the area of science on which they are reporting. Academics who stringently follow the scientific method certainly don't have a monopoly on scientific truth, but they are consistently more accurate and reliable than those who have no such training and rely on personal experience, anecdotes, and gut feelings.

Does the source have a good reputation within the scientific community? Google the source and see what others are saying about it in terms of scientific reliability. There will be those who object to virtually every source, but read the reasons why. Are the objectors credible? Does the source have praise from other credible sources?

Does the source have a clear political agenda? The worse place you can get your science information from is a politician, political commentator, or highly political media source. It is easy to cherry-pick the scientific literature and find data to support virtually any conclusion, and this is what media sources with a political agenda do.

Does the source cite valid sources or refer to research when making specific claims? Beware of sources that report on particular studies that fail to provide sources, or simply refer to something generic that they know their readers won't bother to investigate. Are they simply linking to another media source? This means they are not even getting their information directly from the source (the published article in the peer-reviewed journal). Another commonly deceptive strategy is to say something that sounds authoritative but does not qualify as a citation, such as "researchers from the University of Kalamazoo found that..." and not even bother linking that text to the research.

Does the information appear in any peer-reviewed, credible, journal? While the inclusion of such information in a peer-reviewed journal does not guarantee its veracity (as we have seen), the exclusion of such information is a strong indicator that the information is not scientific and highly unreliable. Visit Google Scholar and enter in some of the keywords to see if you can locate the research to which the article refers. If you can find it, then ask the following questions:

Is the source's headline consistent with the study's headline in the peer-reviewed journal? If your source's headline reads, "Researcher Proves Bigfoot is Real" and the published paper's headline reads, "Researcher finds large unknown fecal mass," then that is a good indicator of a highly unreliable source.
Is the peer-reviewed article's abstract consistent with the findings of the story? An abstract is a concise, one-paragraph summary of the article and its findings, and it is almost always freely available for all articles (without registering). If the bottom line of your source's story is that "science finally proved Bigfoot is real" and the abstract concludes that "it is suspected that the fecal mass comes from buffalo that have adapted to a new diet," then that is a good indicator of a highly unreliable source.
Due to the confirmation bias, you may be tempted not to question the conclusions of your favorite media source when the conclusions happen to conveniently align with your world view. Recognize the power of this bias and overcome it by valuing scientific integrity over consistency in your world view.

Podcast Episode: A Guide for Trusting Sources of Science, for the Non-Scientist


Private, Anonymous Comment On This Post (no login required)Your comment below will be anonymously sent to the post owner, it will not be posted, and you will not get a response. To make a public comment, post below (login required).

Send Comment sending comment...

Registered User Comments


Be Reasonable, Dammit. Buy My Book.

Reason: Book I - A Critical Thinking-, Reason-, and Science-based Approach to Issues That Matter is based on the first two years of The Dr. Bo Show, where Bo takes a critical thinking-, reason-, and science-based approach to issues that matter with the goal of educating and entertaining. Every chapter in the book explores a different aspect of reason by using a real-world issue or example.

Get the book, Reason: Book I - A Critical Thinking-, Reason-, and Science-based Approach to Issues That Matter by Bo Bennett, PhD by selecting one of the following options:


Not Much of a Reader? No Problem!

 Enroll in the Daily Doses of Reason Online CourseThis is passive course where you are sent one lesson per day by e-mail. There is no required interactivity. Each lesson averages just a few minutes.

Enroll in the Psychology of Woo Online Course. This is a crash course, designed to help you understand why your brain favors magical explanations over rational ones.

Have a podcast or know someone who does? Putting on a conference? Dr. Bennett is available for interviews and public speaking events. Contact him directly here.


About Archieboy Holdings, LLC. Privacy Policy Other Books Written by Bo Contact
 Copyright 2017, Archieboy Holdings, LLC. 

Component Viewer

A component is the HTML code for a section of a webpage that can be combined with other components to make a complete webpage. Click the component to insert the component code at the bottom of your current page, then customize it.