4

This question questions a published research paper: Does "Unskilled and Unaware of it" prove that incompetents have a higher self-appraisal than competents?

As does this one: Do the recent CLOUD results have significant implications for global warming?

And this one: Is the US Newborn Mortality Rate higher than 40 Countries?

And this one: Is this new study on cell phone / cancer connection legitimate?

Two are criticized (and presumably close voted) on daring to question "peer reviewed and published" and two are not.

Are we just using "peer reviewed and published" as a reason to shut down contentious political topics? If that is the case, would it not be better just to eschew current political hot-topics as a site policy?

1
  • I'm trying to remember the name of the English guitarist who in relating what he learned in the US said "Americans want to have fun, while the English want to be right." IMHO it seems wanting to "be right" can take the form of being excessively critical of people with off-mainstream (therefore less- or not-published) views. I'm thinking of what Elizabeth Kenny went through with her Polio therapy, or Ignaz Semmelweis, with antiseptic procedures. I've experienced it re. performance tuning. Anything can be wrong, no matter how well accepted.
    – Mike Dunlavey
    Sep 4, 2011 at 17:12

2 Answers 2

Reset to default
7

I would consider any question about the presentation of a scientific paper in the media to be on-topic. Examining if the conclusions of the paper are presented accurately in the media is perfectly fine.

When you are asking about a specific paper that has not been mentioned in the media at all, then we're certainly crossing into the research-level area and we should probably close the question.

I don't think we should shy away from critically examining scientific papers, even though it is problematic with our huge scope and the lack of experts in all fields. If we restrict it to papers mentioned in the media, the vast majority of what is left is going to be about common skeptics topics like evolution, alternative medicine and climate change. There are often skeptical bloggers and others writing about the details of those widely publicized papers, so there should often be enough material for us to work with, even if we don't have an expert in that field on the site. But we should be able to attract experts for the most popular field over time on the site, which should greatly improve our answers to such questions.

The CLOUD question is about a widely publicized paper and I'm seeing it more as the question if the media representation is accurate and also if there is any contradicting evidence in the peer-reviewed literature. I would regard this as on-topic.

The newborn mortality rate question is about a statistic that is very often used as evidence in comparisons of the US healthcare systems with other countries. I also think this on is on-topic, though I think it would be a better question if it asked whether this statistic means that the US healthcare system is worse than those of the other countries. I suspect that this claim is the motivation behind the question, but it is really a related, but different question.

1
  • Thank you. I think this clarifies the situation well.
    – Russell Steen
    Sep 5, 2011 at 15:18
4

I think this is a case of "What is there to be skeptical of here?"

Simply put, if you wonder whether the study is sound, read it. Unlike unbacked claims, a peer-reviewed study one is backed with the methodology and the data used to draw the conclusion. You have everything at your disposal, just read and decide for yourself.

If you don't understand what is being said, that I can understand but it isn't skepticism at this point; it's a need for vulgarization, no? I don't think that fits, or should fit, in our scope.

Now, if it's an article reporting a peer-review study's finding, it's a different story. Journalists have, let's say, a pretty bad record of accurately reporting science. I think such questions are acceptable, if only because poor scientific journalism must not go unquestioned.

4
  • 1
    This doesn't make sense in the context of my question. My question is why we are treating some peer reviewed studies differently than others. All four questions question a peer reviewed study. Why are two bad, and two not? Should all four be bad? If so then we are we only dinging the political ones?
    – Russell Steen
    Sep 3, 2011 at 0:39
  • To be clear, what I'm trying to get at is: What is the policy? Is the policy that any question which directly challenges a peer reviewed study is bad? If that's the case, that's cool. If it's not the case, I'd like to know what the policy is.
    – Russell Steen
    Sep 3, 2011 at 0:46
  • It has to be worth asking how to deal with the (ridiculously common) situation where there are contradictory peer reviewed studies. Epidemiology is riddled with contradictory claims about small effects and it is often the existence on just one of these that drives media frenzies. Skepticism can't stop when we find a published study with a result we like.
    – matt_black
    Oct 10, 2011 at 23:51
  • 1
    @matt_black: I think such contradictions are out of our scope. I think makes more sense to leave this to the field's SE site. With that said, anyone who uses a study to make a point falls under our scope; we focus on claims, not studies. An intellectually honest person, faced with an unclear portrait of reality, should be agnostic about the nature of reality. If the person is not being intellectually honest by overstating the rational degree of certitute, it falls under this site mission to point it out.
    – Borror0
    Oct 11, 2011 at 2:14

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .