Discussion about requiring references for all claims

Question

Our current policy on requiring references can be found at the following link:

FAQ - Must all claims be referenced?

This question contains the discussion that lead to the adoption of the current policy.

Answer 1

I would personally not upvote answers that make significant claims that are completely uncited. I might make an exception for claims that I know to be true, but then I would probably add the reference in a link or edit it into the question itself.

I think we should try to uphold higher standards for referencing our claims, but the way to enforce it is upvoting and downvoting. That is the canonical way for all StackExchange sites to evaluate the quality of the answers, and we should make good use of it.

Answer 2

The Wikipedia community relies on the No Original Research core policy to combat unsubstantiated claims from finding their way into articles:

Nutshell http://upload.wikimedia.org/wikipedia/commons/c/ce/Nutshell.png This page in a nutshell: Wikipedia does not publish original thought: all material in Wikipedia must be attributable to a reliable, published source. Articles may not contain any new analysis or synthesis of published material that serves to advance a position not clearly advanced by the sources.

The very purpose of this site as defined by the FAQ contains nearly exactly the same idea (my emphasis):

Skeptics - Stack Exchange is for skeptics, rationalists, free thinkers, or anyone who questions pseudoscience. Skeptics is about applying skepticism — it's for researching the evidence behind the claims you hear or read.

The voting system of Stack Exchange is largely meant to relax any need for specific policy regarding what constitutes a valid answer (and by and large it accomplishes this) — however, due to the nature of Skeptics, the community (and perhaps the FAQ) ought to promote the idea of no original research to encourage healthy voting.

Bad:

• "...since I have personally experienced it numerous times"
• "Anecdotally, I've seen it..."
• "I heard that it was also caused by ..."
• "I'm not sure if this idea comes from some research"

Good:

• "There is a fairly comprehensive article on the topic at..."
• Even better, multiple sources
• No citations, but a thoughtful experiment

Answer 3

I think we should strongly encourage users to cite all significant claims they make in their answers. But I don't think we need citations for basic science, say around high school level. Citations are essential for claims that aren't obvious to most people, but I think we have enough people with basic science knowledge that would poke holes into answers that get those wrong.

There are some types of questions that we can safely answer without needing references, mostly claims that blatantly violate some laws of nature or known scientific facts. If I debunk a claim about a perpetuum mobile, I might link the laws of thermodynamics from Wikipedia, but that would only be a convienence for the readers, not a citation as such. If it is possible to debunk a claim using only logic and basic scientific facts, that's perferctly acceptable to me.

Answer 4

In my opinion, it should be mandatory to source answers on Skeptics.SE at all times, either by an hyperlink or by citing the offline resource you used. Answers that do not meet this standard should be downvoted.

The principal reason for doing this is that there is no such thing as "an expert on applying skepticism."

While all of us might be skeptics, the knowledge we share come from another field of knowledge. Answering a question on homeopathy requires knowledge of the clinical test that have been made. An answer on climate change requires knowledge of climatology. If sources are not given, most users won't be able to peer review. We'll just have to take it at face value.

Ironically, that would be very credulous of us.

Answer 5

I say no, because not all claims have been evaluated with any rigor, and unless we're going to stop and evaluate every claim with due scientific rigor, we're going to have to content ourselves with reasoned, rational analysis. We're not here to establish truth or falsehood, we're here to point out flaws, problems, and concerns. Lot of questions have been raised that have obvious answers, and that's not rubbing me the right way.

As far as I'm concerned, we're failing if we don't question everything, at least a little. Applied Skepticism demands no less. I think cogent, logical objections are a valuable when there is a lack of concrete empirical evidence.

Answer 6

A lot of claims comes without any source, and can be plausibly denied just with rational thinking and common knowledge. Often the claims are so broad and unspecific, that you hardly can prove them wrong.

If we establish a mandatory for references, is wikipedia allowed and random blogs, newspapers? Or just scientific material?

If you have scientific material, which can be cited over the net, that's fine, but how long will the link be intact? What, if the material isn't written in English?

Answer 7

I'm not smart enough to think of a mechanism of how the process could work and therefore the process doesn't work isn't a logical argument.

That line of reasoning lead doctors ignore that washing their hands before surgery is a good idea at a time when they didn't know about bacteria.

Medicine progressed a lot when it switched from being based on rational justifications to being based on empirical evidence.

Logic is the process of coming to new conclusions based on already accepted claims. Depending on the context different claims need a different level of proof. A logical answer often begs the question and don't address the actual issue.

If you simply build on claims that are accepted by all sides of the argument than you can use a bit of logic.

Answer 8

Logic is never enough.

1. Humans are already too fallible to consistently, accurately judge what is "pure logic".
2. ... especially when it's their own logic.
3. ... especially when it's a topic they are passionate about.

Entire lives of brilliant men have been spent in the pursuit of defining pure logic.

The problem with logic is it's all too easy for fallacies to be introduced. Take a recent example on the WTC7 collapse on 9/11 (from memory, numbers are probably off, the answer was deleted):

1. Steel melts at 2500 degrees
2. Building fires are usually around 2000 degrees
3. If the steel didn't melt than the steel melting could not have caused the collapse

#3 sounds like pretty pure reason, right? So where is the fallacy? Well, I could probably Google #1. But what about #2? I haven't ever measured the temperature of a building fire. Who has? Have they measured enough to know that average and the standard deviation? What about environmental factors? What about the primary fuel for the fire? etc. etc. In the end, is it enough to conclude, categorically, that the temperature of the fire inside WTC7 was no more than 2000 degrees? No sources were provided, and even if they were, they would need to reach that conclusion.

I agree with Fabian that there is a certain level of "obviousness" that we can assume of our readers. But, this must be a very basic level, and we must never excuse the need for a reference based on "logic" as it is too easily twisted and abused. After all, we are skeptics, aren't we?

Answer 9

I think that most answers will need to be referenced. I have only posted one answer so far that was not referenced. This answer pointed out flaws in other people's logic and pointed out the difficulties with figuring out how difficult the situation is. Ideally I would have referenced this, but these aren't easy ideas to find references for. My argument was too long to post as a comment and I didn't want to allow incorrect ideas to stand unchallenged and so I posted an unreferenced answer.

I think the most important test is whether the answer helps bring us towards the solution - or really just wastes our time.

Answer 10

While I understand the need for sourcing the non-obvious claims, to what extent this should go? Do facts which are result of reasoning require sourcing as well? What if the reasonaning/proof is a bit complicated, therefore many people are unable to follow it? What about "common knowledge"?

Take, for example Placebo answer. There are many "non-obvious" things mentioned in the answer, but I think most of them can be considered "common knowledge" and therefore I would not say they require sourcing. Off course, providing some links to a clinical studies dedicated to a placebo would make the answer even better, but I do not think it is bad as it is.

---

After a few months spent here I think I have to revisit my stance, and I now agree even "common knowledge" requires sourcing, as not requiring it does more harm.

Note:

1. sourcing obvious claims is quite easy (I think we agree even a plain Wikipedia link is enough, and it is not necessary to link to a double blind peer review study on facts like apples fall down)
2. not requiring sourcing them leaves the gray area, which is abused by answers like those discussed in How to handle answers based on «logic»?

Answer 11

It isn't logic alone, but also known, undisputed, or so far undisputed facts (how painting a wall works). If somebody claims, corn circles were build by magnetic power, I don't need study, and wouldn't know where to find it, that wheat isn't sensitive to magnetic power.

Answer 12

I'm frustrated because my first answer got downvoted for what I think is not a good reason.
Sometimes people make claims which you can't debunk with "reputable references", simply because scientists don't go about publishing in peer-reviewed journals papers saying "popular belief X ain't true". Sometimes because they have never even heard of popular belief X, but most of the time because popular belief X is kwatsch, and they don't get research grants for publishing on that.
So what are "reputable references" here?