6

If I think a single research is meticulous enough to clarify a subject, is it right to only cite that one article?

The one I'm interested in references previous work, so there would be different sources, but it's still one single starting point.

13

Here's how personally I see it:

We have a range of sources of information, in which we put a differing amount of trust. Here is a rough ordering. (I am sure it could be debated and enhanced, but it is just for illustration.)

  • Systematic reviews
  • Multiple meta-analyses
  • Meta-analysis
  • Accepted Text-books
  • Several peer-reviewed, properly-blinded experiments with randomised controls
  • A single such experiment
  • Quick experiments
  • Statements by experts
  • Case studies
  • Random web-sites
  • Personal Anecdotes
  • Speculations
  • Facebook memes

We have minimum quality standards here that forbid evidence that is personal anecdotes or below. We look at anything below peer-reviewed experiments with some leeriness.

The job of the answer is to address the claims of the source in the question, and address them with evidence that is higher in trust.

Ideally, that would be from the top of the list, but often that's not available, and we have to resort to lower quality sources.

I would offer this question as an example: Do McDonald's burgers have an unnaturally long lifespan? It is a question unlikely to be examined by an expensive lab. Nor is the answer (which is predictable by food scientists) likely to earn a place in a peer-reviewed journal.

Questions about history and attribution of authorship also tend to require answers from sources lower on the totem pole.

However, we should be offering better quality evidence than the original source.

I recall a question (I can't find it now - I think it was about fluoridation.) where I answered citing a single experiment. The OP pointed out the existence of studies with opposite conclusions - demonstrating the dangers of relying on a single study! I had to fundamentally rewrite my answer after doing more research - I found meta-analyses that included the cited studies, and made a much stronger case for the original conclusion.


So, to answer the question:

  • Answers citing a single paper should not be rejected out of hand nor banned.
  • Answers that cite multiple papers, responses to papers, reproductions of experiments and meta-analyses of multiple papers should be trusted more, and voted higher.

Edit: Here is someone else's attempt at the same idea of an evidence hierarchy:

enter image description here

  • Maybe you could add something about the "here's some new evidence" case? – Sklivvz Oct 11 '13 at 21:56
  • @Sklivvz: Could you please elaborate? – Oddthinking Oct 11 '13 at 22:22
  • So assume there's already an answer and, later on, someone wants to add significant new evidence. They shouldn't be required to add all the old evidence plus the new. – Sklivvz Oct 12 '13 at 0:16
  • @Sklivvz True, they can add to an existing answer. But, if they want to make a stand-alone answer that is better than the existing answers, to should repeat any of the old evidence that isn't subsumed or superceded by the new evidence. – user5582 Oct 12 '13 at 4:34
  • @Oddthinking Where would you slot exploratory experiments that are just searching for promising new directions... often lacking good controls and heavily suffering from publication bias? They're an important part of science to help focus limited resources, but I'd put them somewhere close to "quick experiments". – user5582 Oct 12 '13 at 4:48
  • @articuno, it's not always feasible or correct to modify an existing answer by someone else, do example when doing so would change the meaning of it. – Sklivvz Oct 12 '13 at 8:14
  • @Oddthinking This is a great list, and I think we should keep improving it. I have a suggestion to differentiate between systematic reviews and meta-analyses (ncbi.nlm.nih.gov/pmc/articles/PMC3024725), placing systematic reviews as more reliable meta-analyses. – user5582 Oct 25 '13 at 9:03
  • @Articuno: I am not convinced I have been using those terms consistently. Thank you, I will try to use them right from now on. – Oddthinking Oct 25 '13 at 9:07
  • @Oddthinking I'm just learning about this distinction myself. cochrane-net.org/openlearning/html/mod3.htm – user5582 Oct 25 '13 at 9:27
  • I'm not sure quick experiments should rank higher than statements by recognized experts. Also, there are different types of experts out there and the word of a nobel laureate is going to carry a different weight than someone else. – rjzii Dec 4 '13 at 18:41
  • @Oddthinking I found myself reading this post today after having been pointed to it from another meta question. The second bullet point of your answer raised another meta question I just posted based on something I recently read: How to deal with the decline effect?. I posted a new Q because I felt like it deserved to be its own Q (and not hijack these comments), but I would be interested in your thoughts on it, in light of your answer here. – tniles Jun 1 '16 at 15:55
6

Yes. Sometimes that’s the best there is.

Other times, on the other hand, a single source is trumped by multiple, aggregated results. Use judgement.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .