How do we deal with doubt about an original source for a [quote] question?

Question

In many cases, quote questions can be answered by finding sources or direct evidence for the quote in question, and that ends the matter.

But some of the time, it all comes down to one source claiming that it was said... which could be seen as the necessary evidence, but it could also be called the original source of the claim, and be subject to the exact same question of "is this claim credible?"

As a result, it would generally be preferable to have multiple corroborating sources, or more direct evidence, in order to be able to definitively say that the claim is true.

What should be done when such corroborating evidence is not available? If a single journalist claims that person X said Y, and all other instances of the claim flow from that one, do we accept the journalist's claim?

Where there is no other reason to doubt it, I can't see a problem with accepting the claim. But what happens when there are plausible arguments for why the claim may be false on the basis of that source?

Usually, these arguments are much less solid than the typical standard by which answers are judged. But at the same time, it would be poor form, as skeptics, to accept the claim on a single original assertion.

This dilemma is exacerbated by the fact that much of the "reasons for doubt" would likely fall under "original research", at least to some degree or from a certain perspective.

One could argue that we would need a reputable source discussing these reasons for doubt. But if it were, say, a scientific paper that wasn't peer-reviewed, it wouldn't be unusual to identify reasons to doubt it as an answer where definitive confirmation or rejection isn't possible.

How should we handle this?

Answer 1

It is a simple fact of life that we cannot know everything. What was said or done behind closed doors often cannot be resolved with reproducible double-blind tests. Questions about quotes are frequently not able to be verified from multiple sources.

Generally, answers should provide more rigorous evidence for or against a claim than the original claimant.

For a quote question, the evidence hierarchy from most trusted to least trusted is probably roughly this:

• An audio recording with a reasonable provenance.
• Multiple witnesses.
• Direct confirmation from the person alleged to have made the quote.
• A professional journalist/historian reporting the quote.
• Multiple pieces of evidence that the quote is in keeping with the person's espoused views. Example
• A random (perhaps biased) person reporting the quote.
• Speculation or conjecture about what a person might have said. [This would generally be below our minimum standards for evidence.]

(This is for illustration purposes only - I don't want to have to spend much time defending it in detail.)

Whatever the evidence chosen, we should ensure that answers do not overstate the confidence that can be drawn from the quality of evidence provided. For these types of questions, we will never be able to definitively answer them.

Answer 2

This reminds me of Do the Amish have a lower rate of autism? Also, there was a meta discussion for it: Can we fix the Amish/Vaccination/Autism question? As I mentioned in my answer to the question, everything I found pointed at Dan Olmsted as being the source of the claims. In this case I wasn't able to find anything that directly disproved the claims, but I was able to find quite a few sources that show Olmsted's claims lack credibility. So I'm of the opinion that there are definitely times when it's appropriate to question the validity of the original source.

I think that if you see a source and something looks odd about it, mention it in a comment. If there are enough things that are odd, I think what you did for Did Einstein comment on feeling the presence of Jesus while reading the Gospels? was appropriate. From there, we might have to leave it to the community to decide if those are legitimate discrepancies, or if you are the only one who feels they are odd.

The community should also be trying to resolve or strengthen the discrepancies. For example, @CarstenS commented that "bon mot" entered the German language in the 18th century. Carsten should have, however, included a source for that claim. The Google books N-gram viewer does show bonmot was used in German in the early 1900s, but hopefully someone would be able to find a better source than that. Given a sufficiently good source, it would be appropriate to remove that paragraph from the answer.

This also might to be a good time to use a Community Wiki answer. People can add discrepancies as they find them, and remove them as they find sources that resolve the discrepancies, or strengthen them by adding sources that confirm the discrepancies.

The goal would be to provide enough sources that the answer either shrinks until it's worth deleting it, or gets strengthened enough that the consensus is that the original source isn't credible.

I'm not sure how well that would work in practice, though. Going back to the example, "bonmot" showing up in German might be sufficient evidence for one person, but another person might not feel like it's enough. That could lead to a rollback war for that question.