Register to comment and receive updates on new episodes!
Quick Reg!

one moment please...

Have a Question? Ask Dr. Bo! Your Question Might Be Turned Into an Episode!
Blog Home Tags:
climate change
scientific consensus

Methodology Over Conclusion

image loading... by Bo Bennett, PhD, Social Scientist, Business Consultant
posted Tuesday Aug 22, 2017 08:18 AM

image loading...

Bo Bennett, PhD

Social Scientist, Business Consultant

About Bo Bennett, PhD

Read all about me at

In science, methodology is the detailed process used to arrive at a scientific conclusion. As you can imagine, the more the methodology is flawed, the more likely researchers are to come to an inaccurate conclusion. Virtually every published scientific paper has a detailed methodology section for three main reasons: 1) so others could assess how reliable the conclusion is based on the methodology 2) so others could assess that the methodology is actually measuring what it is supposed to be measuring (validity) and 3) so other researchers can duplicate the study to check for consistency in the results. We can think of our everyday reasoning process like methodology. If we have a flawed or inconsistent reasoning process, we are likely to come to flawed and inconsistent conclusions. But if we know how we arrived at a conclusion, we can much more easily spot the flaws in our reasoning, and make corrections.

Morality from the Bible

If you ask the majority of Americans where we get our morality, they will say from "God" or the "Bible." If you believe that God is a perfectly good being, then this sounds great. The problem is, it is a horribly flawed methodology because it is ambiguous, unreliable, and invalid.

Ambiguity. When people say they get their morality from "God," some believe that a "moral code" is "written on our hearts," but most will say that the Bible contains moral commands, which it does. The problem is, the moral commands range from good moral advice (e.g., love your neighbor) to horrific moral advice (e.g., stone gays to death). So how does one know which moral advice in the Bible to follow? Just New Testament commands? Well, no, since the 10 Commandments in the Old Testament are commonly referenced as God's moral commands and the New Testament certainly has its share of terrible moral advice. To make the methodology unambiguous, we would need clear "rules" that we can apply to distinguish the good moral advice from the bad. Absent of this clear methodology, we run into people using "God" and "The Bible" to justify horrendous actions and behaviors, which leads us to the next two problems.

Unreliability. A reliable methodology is one that produces the same results time after time when the same methodology is applied. If both christian #1 and christian #2 say that "The Bible" is their moral foundation, yet christian #1 is a member of the KKK and christian #2 is a vocal opponent of the moral views of the KKK, then we are dealing with a problem of reliability. Again, the result of an ambiguous methodology.

Invalidity. A valid methodology is one that does what it is supposed to do. In our KKK example, members of this organization believe that they have the moral high ground. In fact, virtually every religious person who makes a moral argument believe that they are following the moral commands of their god. This invalidity problem is not necessarily a problem with the Bible; but a problem with the ambiguous methodology one uses to derive their morality from the Bible.

Science, Reason, and Conspiracies

How do you determine what scientific facts and theories you accept, and which you reject? If you are like most people, you do a lot more rationalization than methodology. Meaning, you start with your conclusion the rationalize or justify the conclusion by cherry-picking data that supports your conclusion. This is problematic for the same reasons: ambiguity, unreliability, and invalidity.

Here are five controversial scientific consensus statements with overwhelming support:

  • Recommended vaccines are very safe and serious adverse events resulting from them are quite rare
  • The universe is roughly 13.7 billion years old
  • Evolution is the reason for biological diversity
  • Observations throughout the world make it clear that climate change is occurring, and rigorous scientific research demonstrates that the greenhouse gases emitted by human activities are the primary driver
  • There is no credible evidence that genetically engineered crops are less safe than foods from non-genetically engineered crops

Now if you generally disagree with most or all of these, you mine as well stop here and refer to my other writings on trusting scientists and trusting scientific consensus. If you agree with all of these, keep reading, because almost certainly there is some area of science where your personal views depart from the scientific consensus. For the rest of you who claim to trust the scientific method and rely on overwhelming scientific consensus, but disagree with one of the previous statements, read on.

So what is your reasoning process (methodology) for deciding which scientific ideas with an overwhelming consensus to accept and which to reject? Perhaps you are expert in a scientific area where you have information that other experts don't have. Perhaps, you have a powerful political or religious bias that is getting in the way. There might be other reasons, but you won't know until you understand your own methodology by making sure it is unambiguous, then checking for reliability and validity.

Let's use the example of the human contribution to climate change. You generally accept overwhelming scientific consensuses, but not in this case. Here, you just don't buy it. Perhaps you think the scientists are all on the take and will fabricate facts to get funding. Perhaps you think scientists don't know what they are talking about when it comes to human involvement. Essentially, you believe that there is a worldwide conspiracy to suppress the truth that human activities have very little or nothing to do with climate change. If you were to document your methodology for coming to this conclusion, it might look something like this:

Generally accept overwhelming scientific consensus unless there is a political bias involved

We need to stop right there since virtually every scientific conclusion has some political ramifications. Take the age of universe and evolution. Teaching creationism/intelligent design is a highly political issue. If this is your criteria for rejecting an overwhelming scientific consensus you have a few problems. First, it is ambiguous. What exactly is "political bias" and how is it measured? Without knowing this, we can't apply it fairly to other issues. Second, why are you assuming political bias invalidates the conclusion? Bias can cause people to come to the wrong conclusions, or it can just cause them to more passionately support the right conclusion. Third, why are you assuming political bias has only a one-sided effect on the data? In the example of climate change, although the majority of climatologists and academic institutions are left-leaning, those organizations that oppose climate science are strongly politically biased or fueled by self-interest (e.g., Big Oil).

Perhaps your reasoning process goes something like this:

Generally accept overwhelming scientific consensus unless there is a good chance that the researchers' conclusions will be affected by self-interest, such as obtaining grants or status within the scientific community

This applies to all politicized issues. To be consistent, you need to reject all scientific consensus' where this applies, which is all on the above list. Of course, this would be highly problematic for many reasons already addressed in my previous writings, but the point here is consistency with your reasoning process.

I can go on with dozens of flawed methodologies, but only one makes sense. Accept overwhelming scientific consensus on any issue*; even when the conclusion goes against your political or religious beliefs. If you inject conspiratorial thinking (e.g., the scientists are all lying to keep their jobs), then you need to justify why this same conspiracy doesn't apply elsewhere.

Reasoning is about the process, not the conclusion, meaning that sometimes you might be wrong but for the right reasons. I strongly believe that this far better than being right for the wrong reasons (a flawed reasoning process) because, in the long run, a consistent reasoning process is far more likely to bring you to the correct conclusions, even when those conclusions seem "counterintuitive" or conflict with your political or religious ideologies.

* This is a general rule and like most general rules, there are exceptions. It is possible that new data has been discovered and the scientific community has not had time to assess the data and revise their conclusions, so the consensus is outdated. There are undoubtedly other scenarios, but these are extremely rare and should never be used as a reason to reject the scientific consensus based on the probability alone.

Podcast Episode: Methodology Over Conclusion

Private, Anonymous Comment On This Post (no login required)Your comment below will be anonymously sent to the post owner, it will not be posted, and you will not get a response. To make a public comment, post below (login required).

Send Comment sending comment...

Registered User Comments

Be Reasonable, Dammit. Buy My Book.

Reason: Book I - A Critical Thinking-, Reason-, and Science-based Approach to Issues That Matter is based on the first two years of The Dr. Bo Show, where Bo takes a critical thinking-, reason-, and science-based approach to issues that matter with the goal of educating and entertaining. Every chapter in the book explores a different aspect of reason by using a real-world issue or example.

Get the book, Reason: Book I - A Critical Thinking-, Reason-, and Science-based Approach to Issues That Matter by Bo Bennett, PhD by selecting one of the following options:

Not Much of a Reader? No Problem!

 Enroll in the Daily Doses of Reason Online CourseThis is passive course where you are sent one lesson per day by e-mail. There is no required interactivity. Each lesson averages just a few minutes.

Enroll in the Psychology of Woo Online Course. This is a crash course, designed to help you understand why your brain favors magical explanations over rational ones.

Have a podcast or know someone who does? Putting on a conference? Dr. Bennett is available for interviews and public speaking events. Contact him directly here.

About Archieboy Holdings, LLC. Privacy Policy Other Books Written by Bo Contact
 Website Software Copyright 2019, Archieboy Holdings, LLC. 

Component Viewer

A component is the HTML code for a section of a webpage that can be combined with other components to make a complete webpage. Click the component to insert the component code at the bottom of your current page, then customize it.