16

Sometimes claims are made that are unfalsifiable. Evidence could conceivably produced to show the claim is true, but no meaningful test could be produced to show the claim is false.

A silly example might be "Are there any white crows?" could easily be proven True if someone found a white crow, but is practically impossible to prove False.

A more real example is Has there been an Encounter of the Third Kind?. It is feasible to prove, with extraordinary evidence, that extra-terrestrials have been in contact with humans, but it is impossible to prove it to be False, even though that is the typical skeptical position.

Another example is about whether governments have broken encryption algorithms. Note: This question does not rely on the supernatural in any way.

A lot of unsupported conspiracy theories thrive on this unfalsifiability.

What should we do with such questions?

  • Should they be closed?
  • Should they be left open until evidence is found (i.e. presumably forever)?
  • Is there an acceptable unreferenced answer that can be provided in these cases?
  • Should they be downvoted into oblivion?
  • 2
    Ilya: I think this is an important question, and it seems to have come up before, but I can't find a broad policy to cover it. I hope you don't mind, but I pan to broaden your question so we can use the answers more widely. – Oddthinking Apr 6 '13 at 7:32
  • @Oddthinking The general SE policy of making questions fit the Q&A format seems to cover it. Questions that cannot be answered should be closed as "unconstructive". – Django Reinhardt Apr 13 '13 at 23:26
  • 4
    @Django: It's more subtle than "cannot be answered". It is "cannot be answered No, can be answered Yes", which is especially galling if the skeptical position is "No". It smells and tastes like a question, so the general SE policy doesn't help distinguish it. – Oddthinking Apr 13 '13 at 23:41
  • @Oddthinking I'm not sure I follow. It's impossible to prove that there are no white crows, therefore it is an unconstructive question, but rewording the question to "Is there any evidence that white crows exist?" allows the question to be asked and answered. The default stance of a skeptic is, as you quite rightly say, "no". The burden of proof always should be on those who claim phenomena, not the other way around. – Django Reinhardt Apr 13 '13 at 23:55
  • This feels like the counterpart to Pseudo Answers Are The Enemy (meta.skeptics.stackexchange.com/questions/383/…). Surely questions that cannot be conclusively answered are "Pseudo Questions"? :) – Django Reinhardt Apr 14 '13 at 0:00
  • 3
    @Django: The distinction is that it WOULD be possible to conclusively answer it, if there IS a white crow? ("An albino crow was discovered by field biologist Dr Smith in 1987, and here is a picture.") So it isn't obviously unanswerable, until you've tried to research it. You are absolutely right about the burden of proof... in the real-world. Skeptics.SE's format inverts the burden from the claimant to the answerer, which is why we get ourselves into knots like this. – Oddthinking Apr 14 '13 at 0:22
  • 1
    @Oddthinking Sorry! Perhaps you could give an example because I still don't follow you :-/ If someone is making a claim like "White crows exist", there should be a REASON for that claim. As per the FAQ, public claims or common beliefs are what should be investigated here, not random thoughts. This site is for investigating evidence for those claims, not doing original research into whimsy. So if someone wants to randomly ask if white crows exist, why isn't that considered a pseudo question? – Django Reinhardt Apr 15 '13 at 12:50
  • @DjangoReinhardt, If a not so reliable news source, like the DPRK news agency claiming that white crows were spotted. – SIMEL Apr 15 '13 at 13:14
  • @IlyaMelamed That would be an example of a public claim. – Django Reinhardt Apr 15 '13 at 13:23
  • 1
    @Django: I feel we are talking at cross-purposes, but I am not sure where. I chose 'white crows exist' as a non-supernatural claim, that I thought everyone would agree was false, but would agree white crows could conceivably exist, just as black swans surprised ornithologists when Western Australia was settled by Europeans. Presumably to be a valid question, there would have to be a public claim, including a newspaper article or Facebook meme, but I am not sure why that is relevant to the discussion. – Oddthinking Apr 15 '13 at 13:36
  • @Oddthinking Yes, perhaps we're talking cross one another here, but to answer your question, by making the "white crows" related to a public statement, it is no longer "unfalsifiable". By forcing questions to focus on the claims made, rather than what the claims are claiming (if you follow me), this allows such questions to exist on the site, without falling into the trap of unfalsifiability. – Django Reinhardt Apr 15 '13 at 15:37
  • @Oddthinking So for example, if the question regarding white crows comes from (for the sake of argument) the DPRK news agency, the question might be: "What evidence is there to support DPRK's claims about the existence of white crows?" – Django Reinhardt Apr 15 '13 at 15:44
  • @Beofett All those are falsifiable questions, aren't they? Which is to say, they can all be answered with evidence. As such, I don't understand how they factor into this? – Django Reinhardt Apr 15 '13 at 21:10
  • @DjangoReinhardt The first link isn't the one I had in mind when I started writing the comment... I must have grabbed the wrong link. I agree the second one is a bad example; I had a line of reasoning in mind that, in hindsight, seems not very useful. Hazards of multi-tasking while in a hurry :) The third link I listed, though, Is there any evidence Jesus existed? does seem unfalsifiable, and doesn't meet your criteria. Yet it has 130+ upvotes. – Beofett Apr 16 '13 at 12:02
  • @Beofett Ah, thanks for clarifying! I still don't see how the Jesus question is unfalsifiable, though. It can be answered using evidence based means. – Django Reinhardt Apr 16 '13 at 12:27
10

Honestly, I think that the accepted answer to "Are yowies real?" likely handles this in the best possible way:

Probably not, but also likely unfalsifiable.

Granted the answer would likely change a bit depending on the question, for example, in the case of the PGP encryption the best answer would be something along the lines of:

There are no known techniques other than a brute force attack; however, if there were they might not be disclosed to the general public.

Truth be told, there are a lot of questions out there - especially if you study history - where the only real accepted answer is you can speculate and provide evidence to back up your conclusions but we may never have a definitive answer or the definitive answer might be years away. I'd say as long as people occasionally check the accepted answers to make sure they are still valid we should be fine.

  • 4
    Agreed, since this site isn't a scientific journal, I don't see the problem with allowing unfalsifiable questions here. Even if the answer is "It's impossible to prove that white crows do not exist, but it's very unlikely because of the following reasons..." that still provides useful information and the next time someone asks, they can be redirected to that question to show why such a thing is very unlikely. (or ideally, they've already found that someone already asked their question and are satisfied with the answer) – Johnny Apr 18 '13 at 19:40
9

Here is a proposal. It is based on the example question "Are there any white crows?"

First, I explain the current system, then I propose a new form of acceptable answer.

Acceptable Answers:

Currently, the following are acceptable, in order of preference:

  1. Yes, there are white crows. [Ref with examples]

    Assuming it is true, this is the best answer because, hey, we all love being surprised by the world.

  2. No, there are no white crows. This was proven by a careful examination of every single crow that is alive. [Ref detailing method of global experiment]

    This answer is unlikely because of the infeasibility, but if it were true, it would be acceptable.

  3. Experts have looked for evidence of white crows. They found none. If white crows existed, we would have expected the various bird-watching expeditions to have seen one by now. Absense of evidence where evidence has been predicted is evidence of absense. We should provisionally accept white crows don't exist until evidence turns up to the contrary. [Ref to literature review describing the copious expeditions and the lack of evidence, or to expert declaring that they have been unable to find any literature.]

    This answer is less satisfying because (a) it isn't a 100% definitive answer and (b) it relies on expert opinion. Nonetheless, we have a history of accepting such answers, as they are the best we can expect in the real world. Citing an expert shows it isn't just the answerer's inept Google-searching that failed to find evidence.

  4. Leaving the question unanswered.

    We have questions that I expect will never be answered; in theory they are waiting for science to catch up, but science is unlikely to ever achieve that goal.

Unacceptable Answers:

Currently, the following are unacceptable.

  • There are no white crows. No-one's ever found one. Trust me.

    This has no references, and cannot be verified. It might be the posted by someone incompetent, a troll, a shill for Big Crow, etc.

  • According to my Bayesian model of bird movements and twitcher behaviour, the probability of there being an unseen white crow is 1 in 10,000,000.

    Original research isn't trusted. Mathematical models aren't trusted without a lot of support for the assumptions, premises and the appropriateness of the calculations performed.

New Proposal

None of these answers match the majority view amongst the skeptics. The skeptical position is to provisionally accept that there are no white crows, but to be open to having our mind changed if sufficiently robust evidence was produced.

We should be able to say that. Leaving an obviously false claim unanswered is indistinguishable from us having no position at all, and the question simply being ignored.

I propose that we allow a placeholder answer, copied from a community-approved template.

It would briefly explain (with a FAQ meta-question link for a fuller explanation) that we don't have evidence to prove a claim is false, but the onus is on the claimant to provide empirical evidence that that the claim is true. Until then, we provisionally accept it as false.

The answer should be Community Wiki; no-one deserves real rep for cutting-and-pasting a boiler-plate. Nonetheless, it is likely to be voted up, as people agree that that is their position.

If a counter-answer appears, with evidence, the templated answer is wrong and should be flagged and deleted.

This isn't a great form of answer. I would prefer any of the acceptable answers, above. However, the answer serves to

  • explain to the OP and other interested visitors how skepticism works and what the onus is on the believers,
  • takes the question off the unanswered list, so the questions that are addressable with research get more attention,
  • removes the pressure to close/delete questions that are unfalsifiable, but are still of interest to the Internet.
  • 4
    There are issues with the new proposal. 1. When do we decide a question is unanswerable? Who does it? 2. How do we choose between yes there are and no there aren't without facts to guide us? 3. What if there are 3 answers, all unref, (yes, no, maybe)? 4. what purpose does it serve anyways? Explain how skepticism works - not really on topic for us. Question off the list, bad thing, question is not really answered. Giving a poor answer is not the way to deal with an intersting question (e.g. 10k views on opinion only answer is not good presentation for us). – Sklivvz Apr 15 '13 at 8:05
  • 1
    I think a better solution would be to discourage unfalsifiable questions/encouraging them to be re-worded into falsifiable ones. – Django Reinhardt Apr 16 '13 at 14:13
  • Further to this: "Are there any white crows?" is probably an invalid question. Is it a real, meaningful claim? It is my impression that it is simply a "Russell's teapot" question. Compare with the better claim "Are there any white crows in the London zoo?". Clearly this is answerable. The former could (and should) be closed as too broad. – Sklivvz Jan 14 '15 at 11:10
  • @Sklivvz, I disagree that such questions should be deleted... I think that leaving them open, and presenting an Answer reflecting the actual stance by Skeptics contributors and moderators, and allowing voting to do the talking, is far better than making all this invisible through deletion. – elliot svensson Oct 29 '18 at 14:43
7

We do not need to reinvent the wheel here. All that is not falsifiable, is not scientific by definition.

Falsifiability or refutability is the trait of a statement, hypothesis, or theory whereby it can be shown false by way of some conceivable observation practically possible to achieve. In this sense, falsify is synonymous with nullify, meaning not "to commit fraud" but "show to be false".

By the problem of induction, no number of confirming observations can verify a universal generalization, such as All swans are white, yet it is logically possible to falsify it, as by observing a black swan. Thus, the term falsifiability is sometimes synonym to testability. Some statements, such as It will be raining here in one million years, are falsifiable in principle, but not in practice.

The concern with falsifiability gained its great attention by way of philosopher of science Karl Popper's scientific epistemology falsificationism. In this, Popper stresses the problem of demarcation—sorting the scientific from the unscientific—and lays the demarcation criterion falsifibility, such that the unfalsifiable are unscientific, and the practice of declaring an unfalsifiable theory proved true by scientific method is pseudoscience.

http://en.wikipedia.org/wiki/Falsifiability

This doesn't mean "wrong" or "right". It could be either — the problem is that the truth cannot be found though scientific and fact based means, and hence it is not on topic on this site, however interesting or relevant it may seem.

  • Exactly! This is what I've been trying to say. – Django Reinhardt Apr 16 '13 at 12:25
  • @Beofett The first of those questions is too vague to be answered as it is literally written. (Without complete access to top secret files, how could anyone know for sure what the US government does or does not know?) But people have ignored that and tried to answer the spirit of the questions (i.e. "How safe is PGP?"). – Django Reinhardt Apr 16 '13 at 19:11
  • @Beofett The second is perfectly falsifiable, although there's some leeway with the word "credible" (at least the OP tried to define exactly what he meant), as it asks specifically for REPORTS. (The over generalized version of that question would be: "Are UFOs real?".) – Django Reinhardt Apr 16 '13 at 19:14
  • @Sklivvz Actually, they kinda did: pgp.net/pgpnet/pgp-faq/pgp-faq-security-questions.html Edit: Hmm. Well, kind of. – Django Reinhardt Apr 16 '13 at 19:26
  • This is a better link, an essay from the creator himself: philzimmermann.com/EN/essays/index.html – Django Reinhardt Apr 16 '13 at 19:36
  • @DjangoReinhardt That's, I think, the distinction I am having a problem with. You seem to be presupposing that any evidence that might exist is simply unobtainable, and therefore the question is bad because we can't get access to the evidence (similar to the arguments that it isn't valid to ask if CNN is paid to run stories because if it were, CNN probably would hide it). This is substantially different than the idea that a claim is over-generalized or too vague to be answered. I think the difference between "only allows speculation" and "key evidence may be unobtainable, but present" is big. – Beofett Apr 16 '13 at 19:42
  • @Beofett There's no presupposition. Without looking at every single crow in the world, it's impossible to answer negatively with evidence. Hence, unfalsifiable. – Django Reinhardt Apr 16 '13 at 20:00
  • @DjangoReinhardt I'm referring to your comment that the question on whether PGP has been broken by the government cannot be answered because "Without complete access to top secret files, how could anyone know for sure what the US government does or does not know?". – Beofett Apr 16 '13 at 20:05
  • @Beofett Okay: There's no presupposition. Without searching through the government's classified files, it's impossible to answer negatively with evidence. Hence, unfalsifiable. – Django Reinhardt Apr 16 '13 at 20:20
  • 1
    @DjangoReinhardt The presupposition is that such evidence could only exist in classified files, and that there has never been a public acknowledgement or other source of evidence. – Beofett Apr 17 '13 at 12:08
  • 1
    In fact, assuming the government has cracked it, and assuming it would be able to keep it secret, and assuming no one else has cracked it, then it would be impossible to prove - but such is life, you never know if you can get all the evidence you need! The point is that there is no fundamental infeasibility. For example, an NSA agent reading this would still be able to know with evidence the answer to this question even if the three assumptions are correct. – Sklivvz Apr 17 '13 at 12:55
  • I'm sorry, I just assumed that we all know that PGP has not been officially cracked by anyone -- and if it had been cracked by any government, they would keep it a secret for for obvious reasons (and like they've done in the past). Therefore unfalsifiable. (Although to be honest, I guess the question would be better suited to security.stackexchange.com any way.) – Django Reinhardt Apr 17 '13 at 14:24
  • Yeah, that's the presupposition I was talking about (that we all know all the information available already, prior to any attempt to answer the question). Whether a claim is unfalsifiable or not should be self-evident from the phrasing of the claim, and not dependent on assumptions about the availability of data. – Beofett Apr 17 '13 at 15:21
  • @Beofett it's a thin line, what you don't want is questions that need the whole dataset to be answered. In this sense, the PGP question would be unanswerable if it asked about "broken by a non-disclosed government or secret organization", because then one would need to know that all attempts at decryption failed, whereas knowing whether a singular organisation succeeded is actually something which is well known in the right circles - thus answering the question is possible. – Sklivvz Apr 17 '13 at 15:30
  • 1
    How can non-scientific claims not be on-topic for this site when the first sentence of the FAQ says "Skeptics - Stack Exchange is for challenging unreferenced notable claims, pseudoscience and biased results.". A pseudoscience claim by definition is not "scientific" and would be off-topic if only scientific claims were acceptable. – Johnny Apr 18 '13 at 21:06
1

I feel that those question should be allowed as well. The only real answer to them is "No proof is found of that". It's not perfect, but till it's proven correct, it's considered false.

The only other possible answer I can think of is a speculating/using logic. In lack of better options I think that should be fine.

1

I think the questions themselves should be worded in such a way they can be conclusively answered. This is standard across all StackExchange sites: Questions should fit the Q&A format.

So for example, instead of asking:

  • "What proof is there that ghosts aren't real?" or even "Is it true that ghosts exist?"

The question should be:

  • "Is there any evidence that ghosts exist?"

The burden of proof should always be on evidence, not on the lack of evidence.

This is similar to not asking overly broad or unanswerable questions on other SE sites (e.g. "What's a good book to learn to program PHP from?" or "What's the best Bruce Willis film?").

So for Have extra-terrestrials ever been in contact with humans?, the question it points to is a great example of asking the same question while making it answerable: Have there been any credible reports of unidentified flying objects?

Likewise, https://skeptics.stackexchange.com/questions/4163/has-pgp-been-broken-by-government-intelligence-agencies would be better asked as, "Is there any evidence that PGP encryption has ever been broken?" (Indeed, that's actually what the accepted answer answers.)

In regards to the "Do white crows exist?" question, I don't think questions like this should be entertained here unless there is a public claim or other disputable evidence for the existence of white crows. So the question should be: "Is there any evidence to back up the claim that white crows exist?" or "Is the evidence for the existence of white crows refutable?".

I would propose that questions framed in such a way that they cannot be answered with evidence are "Pseudo Questions" :)

  • 1
    FWIW: I regularly edit questions that say "Is there any evidence for [proposition]?" to "[proposition]?" on the grounds that all questions here implicitly have that expectation (and that simpler titles are intrinsically better for skimming.) i.e. The question title "Do white crows exist?" automatically implies the real question "Is there evidence for or against the proposition that white crows exist?" – Oddthinking Jan 8 '15 at 22:50
  • 2
    @Oddthinking I suppose I see your point about question titles (might be annoying to see 100s of questions beginning with "Is there any evidence of...?"), but the questions THEMSELVES should be answerable. – Django Reinhardt Jan 12 '15 at 16:48

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .