6

I fail to understand how other people know what sites are reliable or not and how much.

Making a sort of example, let's think in percentages: how do I know if a site is 50% reliable and another one is 60%? When you answer questions you already have in mind some sources that might fit or you just start by googling and from there you research? If the latter, how do you know what sites are reliable among those you find?

7

There's no such thing as a 100% reliable source - Skeptics.SE cannot hope to only present correct answers, but we can aim at minimum level of accuracy which is much, much higher than the typical sources on the web and newspapers.

So, the following guidelines are merely heuristics - every single one is fallible.

  • Look for sources that explain how they know what they say is right - either by referencing other reliable sources, or by showing how they gathered the empirical evidence for themselves.

  • Look for sources that are peer-reviewed - that other experts have carefully examined and ensured that the results are valid.

  • For journals, look for a high impact factor (or other measure of them having a reputation that the editors will find worth defending).

  • For experiments, make sure they are using a control. Make sure it is blinded (the person doing the measurements shouldn't know if they are measuring a control or not.) If the experiment is on humans, it should be double-blinded (the patient shouldn't know if they are receiving a treatment or a placebo). Make sure the choice of treatment/control is randomised.

  • If the experiment isn't all of the above (randomised, controlled, double-blinded), don't let people tell you there is causality where there is only correlation.

  • For experiments, make sure the sample size is large enough to cover the variability. (Generally, check the statistical methods used are appropriate, and whether any confounding variables have been controlled for.)

  • For experiments, look for ones that have been reproduced by others. (Tough one!) Even better, look for meta-studies, where independent reviewers have scrutinised the quality of a number of reproductions, and accumulated the results to produce a more statistically powerful result.

  • Look for people who are working/writing about their area of training/expertise. (Hint: When talking about science, most celebrities, journalists and Skeptics.SE contributors aren't.)

  • Generally, look for sources with a history of reliability. To help people out, we maintain a list of resources that we have found to be reliable in the past. Again, that doesn't mean that they will always be reliable, but they make a good starting point.

When you answer questions you already have in mind some sources that might fit

Yes. For example, if it is about health, I go straight to the Cochrane Collaboration (see the resource list, because if there is an entry there, the answer is practically written already.

If it is another subject I have passable knowledge about, I might go to Google Scholar, to find papers.

If it is a subject I know nothing about, I might visit Wikipedia - I avoid using it as a reliable source, but it surprisingly frequently points directly to the papers that answer a question. (Note: I commonly use Wikipedia to explain unusual but uncontroversial terms in an answer - where reliability isn't an issue.)

  • Nice answer! Thank you, these are nice rules of thumb. :) – Alenanno Feb 5 '12 at 14:13
  • Note: Sometimes the sources on wikipedia pages are terrible. Please be careful and use some healthy skepticism. – Sklivvz Feb 5 '12 at 14:16
  • 2
    Impact factor is not a good measure, it's very dependent on the specialization of the journal. There are some crappy peer-reviewed journals out there, but you can only identify those by the stuff they actually published, impact factor is not a great help there. – Mad Scientist Feb 5 '12 at 14:24
  • @Sklivvz: Agreed. Wikipedia is useful as a starting point, highlighting the some sources you probably want to look at, but, yes, think about what you are reading. – Oddthinking Feb 5 '12 at 14:32
  • @Fabian, yes. I was trying to make a point explaining why I trust an article in Nature over an article in Journal of Complementary Medicine and Methods - Nature has much more to lose by publishing poor science. (Think of the damage to the Lancet brand that the Wakefield debacle caused.) Can you think of a better way to tell how reputable a journal is? – Oddthinking Feb 5 '12 at 14:43
  • 4
    The curious thing is that journals like Nature or Science are probably less reliable than the more specialized, reputable journals. The big journals place a lot of value on the relevance of the article, they usually get the more sensational topics which are more likely to be wrong than the "standard" science articles about incremental improvements to our understanding. There is no easy way of determining how good a journal is in a specific field, but recognizing the very bad ones is possible, mostly by looking what other bad articles they have published. – Mad Scientist Feb 5 '12 at 14:52

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .