For a question in the form of:

Is XYZ effective in achieving goal ABC?

Is it valid to post an answer in the form of:


This authoritative source lists N mechanisms (M1,... MN) for achieving goal ABC.

Since XYZ does not employ one of those N mechanisms, XYZ is not effective

The specific properties of such an answer that I'd like to highlight are that

  1. The answer does NOT contain any link to research explicitly testing the efficacy of XYZ

  2. The answer nowhere sites that the list of N Mechanisms is 100% exhaustive, basically relying on the logic of "If another mechanism M100 was effective, that authoritative source would have listed it. Absence from a listing is therefore enough to assume the list is exhaustive".

  3. The cited authoritative source does not contain any proof that their list is 100% exhaustive (or even a claim that it is in the first place).

  4. The authoritative source does NOT discuss XYZ specifically as not being effective.

  • Note: the title is a bit imprecise, feel free to re-word to better match the body.
    – user5341
    Sep 3 '14 at 16:20
  • You can improve your question and making it much more answerable by not painting the situation in black and white. "What kind of conclusion can be supported by this level of evidence?" is much better than "Is it valid?"
    – Sklivvz
    Sep 3 '14 at 17:32
  • Also, this probably has an answer here: meta.skeptics.stackexchange.com/a/2925/96
    – Sklivvz
    Sep 3 '14 at 17:53
  • 1
    @Sklivvz - I'm unsure if that falls under "Negative searches" or not... I think this is a separate very specific case (which I saw several times on S.SE - while your answer was of course the impetus for me posting this question, it was not the only case I saw)
    – user5341
    Sep 3 '14 at 22:27
  • @Sklivvz - my exact problem is when the conclusion presented is a stark yet unproven "No" (as opposed to a significantly less problematic "unknown, but there are reasons to think the answer isn't a yes"). However, you're welcome to try to rephrase if you think you can keep the intent of my question intact.
    – user5341
    Sep 3 '14 at 22:28
  • You are asking for an absolute answer to a relative problem. If someone asked whether there's a teapot in orbit between here and Mars, I could show there's no teapot in the list of asteroids and exclude it -- you would not take it as warranted, but there's such thing as a null hypothesis and the concept of having proportionate evidence to the claim.
    – Sklivvz
    Sep 3 '14 at 22:31
  • @Sklivvz - actually, given that the list of asterouds (AFAIK) doesn't include objects the size of the teapot, that answer would be useless and NOT good evidence. However, the bigger problem with your example is that the example has nothing to do with the type of question I am proposing, since the teapot theory isn't practically falsifyable (given limitations of current astronomical technology) whereas efficacy of a drug or a remedy is clearly and easily falsifyable, especially in an easy-to-test case like bug repellants.
    – user5341
    Sep 3 '14 at 22:39

For any question on the site, the evidence provided needs to be solid, and the conclusion drawn proportionate.

Lacking direct testing, it's sometimes impossible to build a direct case. However it's possible to create a weaker circumstantial case with an indirect approach. The important thing is matching the conclusion with the quality of evidence.

As a general rule, voting on this site is the mechanism by which we judge the quality of the evidence and the conclusions drawn from it.

I do not think we should disqualify answers based on a subjective qualitative judgement. It depends on the case.


I believe answers that claim efficacy knowledge* must 100% be based on efficacy testing (if they aren't, they must either be edited to remove the certainty, OR deleted as lacking sufficient evidence to support the claim).

In the absence of efficacy testing of the subject of the question, the answer may not claim anything definitive, and must clearly indicate the lack of knowledge based on explicit testing.

Less categorical conclusions may be supportable by evidence other than testing, but must be fully transparent on why the place their degree of certainty on their evidence.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .