6

If I think a single research is meticulous enough to clarify a subject, is it right to only cite that one article?

The one I'm interested in references previous work, so there would be different sources, but it's still one single starting point.

2 Answers 2

15

Here's how personally I see it:

We have a range of sources of information, in which we put a differing amount of trust. Here is a rough ordering. (I am sure it could be debated and enhanced, but it is just for illustration.)

  • Systematic reviews
  • Multiple meta-analyses
  • Meta-analysis
  • Accepted Text-books
  • Several peer-reviewed, properly-blinded experiments with randomised controls
  • A single such experiment
  • Quick experiments
  • Statements by experts
  • Case studies
  • Random web-sites
  • Personal Anecdotes
  • Speculations
  • Facebook memes

We have minimum quality standards here that forbid evidence that is personal anecdotes or below. We look at anything below peer-reviewed experiments with some leeriness.

The job of the answer is to address the claims of the source in the question, and address them with evidence that is higher in trust.

Ideally, that would be from the top of the list, but often that's not available, and we have to resort to lower quality sources.

I would offer this question as an example: Do McDonald's burgers have an unnaturally long lifespan? It is a question unlikely to be examined by an expensive lab. Nor is the answer (which is predictable by food scientists) likely to earn a place in a peer-reviewed journal.

Questions about history and attribution of authorship also tend to require answers from sources lower on the totem pole.

However, we should be offering better quality evidence than the original source.

I recall a question (I can't find it now - I think it was about fluoridation.) where I answered citing a single experiment. The OP pointed out the existence of studies with opposite conclusions - demonstrating the dangers of relying on a single study! I had to fundamentally rewrite my answer after doing more research - I found meta-analyses that included the cited studies, and made a much stronger case for the original conclusion.


So, to answer the question:

  • Answers citing a single paper should not be rejected out of hand nor banned.
  • Answers that cite multiple papers, responses to papers, reproductions of experiments and meta-analyses of multiple papers should be trusted more, and voted higher.

Here is someone else's attempt at the same idea of an evidence hierarchy:

Pyramid with horizontal segments, labeled from top to bottom: Systematic Reviews and Meta-analyses, Randomized Controlled Double Blind Studies, Cohort Studies, Case Control Studies, Case Series, Case Reports, Expert Opinion, Personal Opinion

11
  • Maybe you could add something about the "here's some new evidence" case?
    – Sklivvz
    Oct 11, 2013 at 21:56
  • @Sklivvz: Could you please elaborate?
    – Oddthinking Mod
    Oct 11, 2013 at 22:22
  • So assume there's already an answer and, later on, someone wants to add significant new evidence. They shouldn't be required to add all the old evidence plus the new.
    – Sklivvz
    Oct 12, 2013 at 0:16
  • @Sklivvz True, they can add to an existing answer. But, if they want to make a stand-alone answer that is better than the existing answers, to should repeat any of the old evidence that isn't subsumed or superceded by the new evidence.
    – user5582
    Oct 12, 2013 at 4:34
  • @Oddthinking Where would you slot exploratory experiments that are just searching for promising new directions... often lacking good controls and heavily suffering from publication bias? They're an important part of science to help focus limited resources, but I'd put them somewhere close to "quick experiments".
    – user5582
    Oct 12, 2013 at 4:48
  • @articuno, it's not always feasible or correct to modify an existing answer by someone else, do example when doing so would change the meaning of it.
    – Sklivvz
    Oct 12, 2013 at 8:14
  • @Oddthinking This is a great list, and I think we should keep improving it. I have a suggestion to differentiate between systematic reviews and meta-analyses (ncbi.nlm.nih.gov/pmc/articles/PMC3024725), placing systematic reviews as more reliable meta-analyses.
    – user5582
    Oct 25, 2013 at 9:03
  • @Articuno: I am not convinced I have been using those terms consistently. Thank you, I will try to use them right from now on.
    – Oddthinking Mod
    Oct 25, 2013 at 9:07
  • @Oddthinking I'm just learning about this distinction myself. cochrane-net.org/openlearning/html/mod3.htm
    – user5582
    Oct 25, 2013 at 9:27
  • I'm not sure quick experiments should rank higher than statements by recognized experts. Also, there are different types of experts out there and the word of a nobel laureate is going to carry a different weight than someone else.
    – rjzii
    Dec 4, 2013 at 18:41
  • @Oddthinking I found myself reading this post today after having been pointed to it from another meta question. The second bullet point of your answer raised another meta question I just posted based on something I recently read: How to deal with the decline effect?. I posted a new Q because I felt like it deserved to be its own Q (and not hijack these comments), but I would be interested in your thoughts on it, in light of your answer here.
    – tniles
    Jun 1, 2016 at 15:55
6

Yes. Sometimes that’s the best there is.

Other times, on the other hand, a single source is trumped by multiple, aggregated results. Use judgement.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .