5

I'm seeing an increase in questions that cite as their source of claim an article that would itself count as a reputable source for this site. It's from a reputable source, it's peer reviewed, or it cites plenty of other sources. More usually many of the above.

The question then takes a claim from the source and says "Is this true?".

The trouble with this is that 1) it's very unlikely that any contributors here are going to be a better position to judge the truth of the claim than the original source 2) since the original source is reputable and referenced we could theoretically say "Yes it is" simply by citing the original claim 3) Usually the only reason to doubt the claim is that the question personally doesn't believe it, or simply doesn't like the conclusion.

Can we have a close reason or something else to deal with question like this?

  • 3
    Some examples would be helpful here. – Andrew Grimm May 6 at 21:48
  • How is "not notable" not an appropriate close reason? If skepticism is not justified then it's not a notable claim. – fredsbend May 7 at 2:51
  • Someone may take a claim from a peer-reviewed paper and the appropriate response is "see the follow up paper, that discusses the claim". Not all people are aware of the follow up paper (and it may even be paywalled for some people), that may verify or refute the claim, so the question/answer is useful. – allo May 8 at 10:43
  • 3
    There are too many contradictory results in the peer-reviewed literature for this to work unless the site is happy to have lots of contradictory answers. While not everyone can read the original literature, enough users can to give a summary of the science not just the original paper. What we should discourage is answers based on single papers in areas where the literature has plenty of alternative views. – matt_black May 8 at 18:54
  • Oh, and "very unlikely that any contributors here are going to be a better position to judge the truth of the claim" is wrong as, if users can read the original claim, then they can also read other papers from the same journal that disagree with it. As can other users. Ignoring the diversity of scientific opinions/results in answers is a disservice to any reader who cares about truth rather than 'any reference that agrees with my opinion' which is all single citation answers often provide. – matt_black May 8 at 18:58
11

I'm pretty sure that these questions are on-topic and have been for most of Skeptics' existence. This was discussed back in 2011 and the highest voted answer by Mad Scientist says "I don't think we should shy away from critically examining scientific papers".

The questions should be reworded as needed to focus on the claim instead of the article so that answers can address the real problem (whether something is true or not) instead of just learning if the article's results are useless or not. Also, saying "debunk this" makes it sound like you've already chosen a side.

I have my own reasoning for why these questions should be allowed:

  • Many of these claims can be found repeated elsewhere. It's basically a waste of time to shut them down since there's a chance someone will ask the same thing later on.
  • Not everyone can read scientific articles. Even if you understand the words, you might not know the difference between blind and unblind experiments; statistical significance and chance; or something peer reviewed and something predatory.
    • On a similar note, some articles are paywalled and nobody wants to pay.
  • Then there's the fact that so many results cannot be reproduced. Science isn't about doing something once and for all. A good answer will address the article that's being asked about and all the other good research out there on the subject.
    • I learned about just how bad the problem of reproducibility is from an episode of Adam Ruins Everything. The sources for that episode are here if you want to check them out.

See also How much can we trust peer reviewed scientific literature?

1

Just to add to what Laurel said, the reason why meta-analyses, reviews and so forth exist is that a single study might not be so conclusive. (The better meta-analyses nowadays also estimate publication bias etc.)

Furthermore, in areas like epidemiology (and not only) there are a lot of observational studies. It can be tricky to infer causation from these and studies will sometimes disagree in their conclusions, e.g. by including different confounders.

Never mind allegiance bias etc.

1

These questions should not be allowed here. Take for example, this question. This basically says, "Is this study sound?" The current answer then just says, here are some ways to evaluate a study's soundness without actually analyzing the study. So that answer could essentially be added to any question on the site that refers to a study's soundness. Questions that all get the same answer are not useful.

Worse, it creates a situation where we might have cross-question debates. Question one asks about a popular source claim. It gets answered with a reference to one or more studies. Another question now asks if that study is sound. Someone mentions a counter study, which then gets its own question...

For questions like "Is this study sound?" we should generally favor people asking on a more specific stack like Physics.SE, Economics.SE, Chemistry.SE, Politics.SE, CrossValidated.SE, etc. where domain-specific experts are more common. And we might direct them to a FAQ where soundness is discussed in general, preferably in the form of a claim if it is on the main site. Or it may be a Meta FAQ if we can't find a notable claim that more directly justifies that type of answer.

The main point of this site is to say, "Yes, that claim reasonably represents what this study said." Or "No, that claim misrepresents what the study said." Or "You can look at that both ways, because..." We should not try to be a singular arbiter of all truth. We create footnotes, not do peer review.

  • The question asks: Is X true?. Wouldn't it make sense to close all questions that ask Is this study about X valid? as a duplicate of that question? Also, if you don't like the answer, shouldn't you downvote it instead of closing the question? – Laurel Jun 2 at 20:56
  • I am less disturbed by the answer and more disturbed by the fact that the question lends itself to that type of answer as responsive to the question asked. – Brythan Jun 2 at 21:35

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .