5

I'm (probably pointlessly) trying to persuade an antivaxxer that vaccines do not cause autism. They pointed to this link that references "OVER 50 PEER REVIEWED SCIENTIFIC Studies from the US National Library of Medicine National Institutes of Health". One big red flag is that the studies were not published by the US National Library of Medicine National Institutes of Health; the website is simply pointing to an index of a whole bunch of studies from various different journals. I started looking at some of the actual journals that published them and did notice some oddities (The "North American journal of medical sciences" with India as its country of publication, high submission fees that could indicate a predatory journal that will publish anything).

What's the best way to determine if a journal is reputable and likely to have high-quality content? Also, what's the best way to determine the opposite - that it is likely that anything could be published in a journal, making the content in the journal no more reliable than if it had been published on someone's personal blog?

1

Expertise

Academic journals are not intended for public digestion. The best ways of evaluating the quality of a journal require specific subject-matter knowledge and experience.

For example, you should be able to tell if a journal in your subject-area is reputable because reputable authors publish there, because the journal is familiar to you through your research, and because the content of the journal is of acceptable quality for your field.

If you aren't an academic, consider asking an academic librarian. They are subject-matter experts on research materials. They would likely be able to provide some excellent guidance.

Evaluating Testimony

Relying on the expertise of a subject-matter expert doesn't mean just believing everything they tell you. There are several things you can do to provide some assurance that what they are telling you is fundamentally correct:

  • Important factual statements can be confirmed. Don't bother checking unimportant details, but focus on things important to understanding the subject.
  • Ask probing questions to satisfy yourself that the expert is providing useful and correct information.
  • Ask multiple experts, perhaps with different viewpoints, to see if they agree. Disagreement doesn't mean unreliability, but it may highlight important differences.
  • Have multiple evaluators confirm the results of the inquiry. In an personal conversation, it would be helpful to have multiple listeners in the interview with the expert. This reduces the risk that the listener distorts what they are saying. With written correspondence ask someone else what their conclusion is.
  • So to determine if a journal is reputable we must trust some knowledge gatekeeper? Why are so many journals published publicly if they are not intended for public digestion? – daniel Jul 8 '18 at 13:18
  • You don't have to trust anyone, you can go gain that knowledge and experience yourself. Assuming you aren't willing to invest that time, yes you will have to rely on someone else who has made that investment. As far as publishing, that's how they are made available to other experts. Not every publication is intended for a general audience. – indigochild Jul 8 '18 at 17:58
  • @daniel After giving it some thought, I added a new section addressing how to evaluate testimonial evidence. You may have to rely on a subject-matter expert, but that isn't the same as trusting them. – indigochild Jul 10 '18 at 16:46
  • 1
    The edit makes it better, before for me it read too much like "the bible, I mean science is written in Greek so we can translate it for the peasants, and choose which bits they hear" – daniel Jul 10 '18 at 20:39
-1

I don't think there is one, academic journals are now used in post truth politics.

I don't think a consensus of journals has much weight either, it quickly turns into a statistic about a group of peoples opinions, and as the saying goes. The scientific method doesn't require a consensus, and scientific breakthroughs are often about showing that the current scientific understanding is incorrect.

You might decide what a good journal is off what organizations are associated with it, if the papers are government or privately funded, until it publishes something you disagree with and then you ditch it saying that its been politically subverted.

Also the quality of a study or paper might not have anything to do with where it is published.

But its not all bad, some amazing and useful technology is being developed, and we still have cartoons!

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .