1

This article claims that Google does not filter out articles from journals poor reputations.

Because predatory publishers perform a fake or non-existent peer review, they have polluted the global scientific record with pseudo-science, a record that Google Scholar dutifully and perhaps blindly includes in its central index. Most predatory journals are included in Google Scholar. The database does not sufficiently screen for quality, in my opinion.

Google Scholar works well for known-item searches, for example, when you quickly need to locate a known article or a paper by a known author.

Is this true?

migrated from skeptics.stackexchange.com Nov 15 '14 at 15:38

This question came from our site for scientific skepticism.

  • is asking for resource to identify better solutions to the [enter problem here] a skeptical question? The provided answer, so far, just gives a indication and an opinion. – woliveirajr Nov 14 '14 at 16:02
  • If that is a problem, I can delete the followup question. – pooter03 Nov 14 '14 at 16:05
  • 1
    Are there junk articles and junk journals in Google Scholar? Sure, that's easy to prove. Is it "filled" with junk? Depends. What does that mean? – Oddthinking Nov 14 '14 at 16:21
  • also as I point out, not all papers in predatory journals are junk. This would be a better question for AcademiaSE, perhaps it could be migrated (as it is a topic worth discussing somewhere if not here)? – Dikran Marsupial Nov 14 '14 at 16:25
  • I think this question actually fits on Meta Stack Exchange, because it is relevant for finding reputable sources. – gerrit Nov 14 '14 at 19:19
  • Google Scholar is not restricted to journals: this link shows something on my website which has been cited in journals but never appeared in one. As ever, documents linked from other high ranked pages are more likely to appear near the top of a Google search, but if your search is too precise there may only be one example – Henry Nov 14 '14 at 19:47
  • @gerrit: Do you mean Meta.Skeptics? I'm inclined to migrate too. – Oddthinking Nov 15 '14 at 2:39
  • No, but it can leave out journals that are known for promoting junk science like homeopathy or vaccines causing autism or any other topic that has already been thoroughly debunked. – pooter03 Nov 15 '14 at 13:16
  • @Oddthinking Oops, yes, I mean Meta.Skeptics. I think the question is poorly phrased but could be reformed to fit on Meta.Skeptics. – gerrit Nov 15 '14 at 14:10
  • Google scholar does filter for quality (sort of), note that the papers at the top of the list tend to have a large number of citations (which is an indication of quality assigned by the research community). – Dikran Marsupial Nov 15 '14 at 14:55
8

At the bottom of the http://scholar.google.com home page is an "About" link. Within those pages are Google Scholar's criteria for inclusion:

Content Guidelines

Google Scholar includes scholarly articles from a wide variety of sources in all fields of research, all languages, all countries, and over all time periods. Chances are that your collection of research papers will be a welcome addition to the index. To be considered for inclusion, the content of your website needs to meet the two basic criteria.

  1. Scholarly articles

The content hosted on your website must consist primarily of scholarly articles - journal papers, conference papers, technical reports, or their drafts, dissertations, pre-prints, post-prints, or abstracts. Content such as news or magazine articles, book reviews, and editorials is not appropriate for Google Scholar. Documents larger than 5MB, such as books and long dissertations, should be uploaded to Google Book Search; Google Scholar automatically includes scholarly works from Google Book Search.

  1. Showing abstracts

Users click through to your website to read your articles. To be included, your website must make either the full text of the articles or their complete author-written abstracts freely available and easy to see when users click on your URLs in Google search results. Your website must not require users (or search robots) to sign in, install special software, accept disclaimers, dismiss popup or interstitial advertisements, click on links or buttons, or scroll down the page before they can read the entire abstract of the paper. Sites that show login pages, error pages, or bare bibliographic data without abstracts will not be considered for inclusion and may be removed from Google Scholar.

There is no mention of any requirement that the site contain peer-reviewed articles, nor any suggestion that it will be screened for the quality of the articles hosted on the site.

  • Does Google Scholar screen for quality? No, nobody seems to be claiming that they do.

  • Should Google Scholar screen for quality? This is a matter of opinion which is beyond the scope of this site.

  • Is Google Scholar filled with junk science? This is vague (what percentage would qualify as "filled"?) and subjective (how do you determine whether or not an article is "junk science"?), so not a question that this site can answer.

7

For a resource that helps identify predatory journals, try Beall's list (ironically the source of the article ;o).

As an academic, I don't think Google Scholar should filter out papers from predatory journals because in science the source of an argument is irrelevant, what matters is the correctness of the argument and the support it receives from the data. Peer review is only the first step in acceptance from the research community, not the last. The number of citations a paper has is a good indication of its acceptance by the research community and Google Scholar already sorts papers by citations. So google does filter by a measure of quality, just not in the way that Beall wants it to.

Just because a paper is published in a predatory open access journal does not mean it is a bad paper (although it is suggestive), so omitting papers from predatory journals will stop people from finding some pieces of good work that have been published there. Filtering by citations will hide most of the junk, but without hiding the good papers that have been published in not so good journals.

There is also the problem of the definition of "junk science" to consider. Some might say that acupuncture, or homeopathy, or the theory that the sun is made of iron, or climate change skepticism, or autism/vaccination links are "junk science". Whether this is true depends on the nature of the argument and the reasoning put forward, and you can't judge by keyword, you do have to look at the paper to find out. Good science quite often turns out to be wrong, so you can't judge what is junk science by whether it is right or wrong. What differentiates between good science and junk science has more to do with whether it follows scientific method (e.g. Popper's criterion of falsifiability).

If someone conducts a double blind trial to determine whether homeopathy actually works, then that is good science. If someone writes a paper on the mechanism underpinning homeopathy that is non-falsifiable, then that might arguably be junk science. By that definition, Google Scholar has almost no junk science, although it has plenty of bad science.

There is a good reason why it has plenty of bad science and scientific papers that are wrong, which is that science is performed by scientists, who are only human and hence fallible, and working at the very edges of what we know, where errors are much easier to make than on more mundane questions.

I don't find the article particularly persuasive. Google scholar actually does quite a good job of exposing bad science by demonstrating how little it is generally cited.

  • Rather than blindly listing all articles, or filtering our those from known predatory journals, how about something in between? For example, marking or labeling articles that come from journals that are known to use fake peer-reviews, for example. Some sort of caveat emptor for journals that have poor reputation? – pooter03 Nov 14 '14 at 14:57
  • Hardly any journals are known to use fake reviews, so that would be ineffective. However this is missing the point, which is that in science (at least) the source is irrelevant and you can't dismiss the argument simply because it was published in a dodgy journal. In science you need to argue the case for and against. I suspect it would also be self defeating as it would be regarded as censorship by those with non-mainstream views on e.g. climate or autism-vaccine links and only serve to make rational discussion of those topics more difficult than it already is. – Dikran Marsupial Nov 14 '14 at 15:11
  • I don't disagree with you about the scientific process. Shoddy articles in dodgy journals shouldn't influence scientific consensus on that topic. However, Google Scholar opens up journal articles to everyone, the majority of which are not trained in how to identify properly written and vetted articles from those that are frauds. This Nature article (which also interviews Beall), talks about this issue more indepth. – pooter03 Nov 14 '14 at 15:35
  • Granted, an article posted in a predatory journal with no and/or false peer-review doesn't necessarily mean the facts are wrong, but I do think that it can't hurt to let us know when the source is dodgy. – pooter03 Nov 14 '14 at 15:36
  • Google scholar is a tool for scholars (as the name suggests), it shouldn't compromise on fulfilling that role just because it is open to everybody. Besides, in my experience the general public doesn't use Google Scholar to look up dodgy argument about climate or autism-vaccine links etc, they look at blogs. Changing Google scholar would have little effect on the promulgation of the sort of issues Beal raises in my opinion. – Dikran Marsupial Nov 14 '14 at 15:55
  • 1
    No problem. I don't have a strong opinion on this matter and was playing devil's advocate. Thanks for your thoughts! – pooter03 Nov 14 '14 at 16:04
  • This post does give a reference to Beals' list, which answers the question as originally posed, but which has since been edited. As the question is likely to be migrated there doesn't seem much point changing it until the question has stabilised. – Dikran Marsupial Nov 15 '14 at 11:29
  • 1
    @DikranMarsupial: Note it has now been migrated. That doesn't guarantee it has stabilised, but I hope it helps. – Oddthinking Nov 15 '14 at 15:40
  • 1
    cheers @oddthinking, it stabilises it enough as at least it means that the discussion of the opinionated part is now more on-topic. – Dikran Marsupial Nov 15 '14 at 16:14

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .