Status of a research finding with no first-principles justification

Question

As this Stackexchange site insists on objective evidence primarily from peer reviewed and published sources and the participants appear to be well versed in researching the same, I believe that a question pertaining to the specifics of the mechanics of the same would be in place here.

If there is a case where a certain phenomenon bears itself out in a well-run study, with robust methodology and yields irrefutable results, but is unjustifiable based on current beliefs, is it accepted by the scientific community? Or, in other words, if a result can not be explained away using first principles based on the current knowledge, would the results still be acceptable and candidates for publication?

As a specific example, consider this question:Has most peer-reviewed research on homoeopathy given positive results?. I can see that the methodology and biases have not been accounted for and the members of skeptics.se have been able to poke holes in the study sited. However, hypothetically, if the study had succeeded in proving positive results of homoeopathy, given the absurdity of the "water memory" premise and "dilutions/succession", would the same hold any currency in scientific literature?

Answer 1

Your own example answers your question.

It explains that there have been trials where homeopathy was statistically significantly better than placebo. If homeopathy didn't work, you would expect that to happen for one study in twenty at the p=0.05 level. (Even higher if you considered publication bias and experimenter bias.) However, when meta-studies are used to evaluate the evidence, the statistics are clear: there is no significant effect.

Stepping away from homeopathy, we can see other areas where results contradictory to accepted theories have been published:

• We've recently seen articles showing neutrinos apparently travelling faster than the speed of light. That was published, despite the absurdity of the claim given modern physics. (It didn't take that long for some plausible sources of flaws to be found, as the authors themselves had anticipated.)

We can point to a number of areas where the conjectures or experimental results were originally considered absurd, but publications were made so that eventually the evidence required the existing theories to be discarded, and new ones accepted:

• Wegener and du Toit's questions that lead to the theory of Plate Tectonics (Ref)
• Lenard's observations of the Photoelectric Effect

Finally, we can point to areas where today's theories are being re-evaluated in the light of new evidence that has been published:

• Accelerating expansion of the Universe has suggested there must be some sort of Dark Energy, although the physics is still unclear.

In conclusion: experimental data is publishable, even if it doesn't fit the current theories. (Arguably, especially if it doesn't fit.) After it is published, it will undergo further scrutiny (including, if possible, reproduction) as will the theories it apparently disproves, until, slowly, a new consensus can grow.

Answer 2

There is a statement often used by Bayesian statisticians associated with loss functions: "Extraordinary claims require extraordinary evidence" and that applies here too.

If something inherently dubious can be and has been frequently repeated by those doubtful of the claim then it inherently becomes more credible. One example is the question Does hot water freeze faster than cold water? now often called the Mpemba effect. But something which is basically consistent with other understandings needs less evidence to be accepted.

Single studies are less convincing, both when supporting a claim (they often overstate how substantial an effect is) and when failing to support it (often because not enough evidence has been collected to distinguish an effect).