I use Media Bias Fact Check very often to check articles and their sources when I find them on social media and generally when I'm reading news or doing "whatever" research. I find it to be reliable. However, I hear from people, including those on the main Skeptics Stack, that it's biased, unscientific, or various forms "unreliable". To be clear, several of the people who disagree with the site simply do so because it disagrees with their (demonstrably biased) opinion, but some people do have their reasons.
I'd like this to be a canonical source for anyone that tries to argue for or against using Media Bias Fact Check to show how reliable sites are when answering or making comments on the non-meta Skeptics site.
For:
Doing my own researching on this, I see numerous fact checking sites use or suggest Media Bias Fact Check for people to do their own fact checking. These seem to all be libraries, but at various universities, so I'd have to assume that they did their own research on the sites to validate the information on their site, even if they are using the same software to build the site. And this is just a sample, not an exhaustive list of sites that recommend Media Bias Fact Check.
- https://libguides.volstate.edu/evaluate/news
- https://guides.csbsju.edu/c.php?g=621995&p=4332693
- https://libguides.nsu.edu/c.php?g=647162&p=4540895
- https://csus.libguides.com/c.php?g=824928&p=5924793
- https://libguides.ucmerced.edu/news
I also see it used by MIT, University of Michigan, and others to not only train AI fact checking bots, but also for other uses.
http://en.wikipedia.org/wiki/Media_Bias/Fact_Check
From my use, I see that it generally matches up pretty well with the Ad Fontes Media Media Bias chart.
Against:
The people that say it's unreliable almost invariably use the same 2 articles as their source. Both sites are listed in various places as reliable. Also, both articles are listed on the the Wiki page above.
"Amateur attempts at such tools already exist, and have found plenty of fans. Google “media bias,” and you’ll find Media Bias/Fact Check, run by armchair media analyst Dave Van Zandt. The site’s methodology is simple: Van Zandt and his team rate each outlet from 0 to 10 on the categories of biased wording and headlines, factuality and sourcing, story choices (“does the source report news from both sides”), and political affiliation.
https://www.cjr.org/innovations/measure-media-bias-partisan.php
Media Bias/Fact Check is a widely cited source for news stories and even studies about misinformation, despite the fact that its method is in no way scientific.
https://www.poynter.org/fact-checking/2018/heres-what-to-expect-from-fact-checking-in-2019/
To put a little perspective on this, though, Tamar Wilner is the author of the article on the CJR site and also writes for Poynter about media bias. It could be that her article in January 2018 or a potential personal connection with the Daniel Funke and Alexios Mantzarlis influenced the Poynter article in December of 2018.
There's also Quora, which has varying views on the site, and references the two articles above.
https://www.quora.com/How-trustworthy-is-the-site-called-Media-Bias-Fact-Check
And Just Facts Daily disagrees with their rating on Media Bias Fact Check, even after their rating is reviewed higher.
https://www.justfactsdaily.com/media-bias-fact-check-incompetent-or-dishonest/
More research
As far as being able to determine media bias scientifically, there's been little success in that, so far as I've been able to research. Most of it says that bias is a perception and uses a wide range of words or phrases to create the bias, so saying that not being able to scientifically demonstrate a site as biased denounces a media bias site seems less than accurate.
- https://www.oxfordhandbooks.com/view/10.1093/oxfordhb/9780199793471.001.0001/oxfordhb-9780199793471-e-44/version/0
- https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0193765
- https://www.sunypress.edu/pdf/63392.pdf
As an aside, AllSides, Snopes, and others have been subject to similar arguments against them, as seen in this article about Media Bias Fact Check.
https://www.insidesources.com/one-website-sets-classify-news-expose-fake-news/
I also got into length discussions about this topic on another Question here on Skeptics.
So, the question is still: Is Media Bias Fact Check a reliable site?
Edit:
I guess I need to define a few things, since people evidently don't understand what I'm trying to ask here. Unfortunately, this might mean I have a few things wrong about this site, since I thought Skeptics was about getting facts, rather than just opinions.
Media Bias
- How critical is a source or an author to the political Left or Right.
- How the wording of an article or source (in general) supports the political Left or Right.
- Doesn't include which stories the source puts on the "front cover".
Media bias is the bias or perceived bias of journalists and news producers within the mass media in the selection of many events and stories that are reported and how they are covered. The term "media bias" implies a pervasive or widespread bias contravening[*] the standards of journalism, rather than the perspective of an individual journalist or article.
https://en.wikipedia.org/wiki/Media_bias
Contravene: Violate the prohibition or order of (a law, treaty, or code of conduct)
Reliable
Consistently good in quality or performance; able to be trusted.
https://www.lexico.com/en/definition/reliable
1: suitable or fit to be relied on : DEPENDABLE
2: giving the same result on successive trials
https://www.merriam-webster.com/dictionary/reliable
Do I really have to continue on with dictionary definitions of words, or do I need to define what "is" is, too?
What I want
I'm not interested in debating people's personal opinion, I want facts compared to facts. As in "does the Media Bias Fact Check site represent the facts as accurately as possible". I'm not interested in confirmation bias, false dilemmas, casual fallacies, strawman arguments, or other fallacies. I will, however, appeal to an authority if that authority is a real authority on the subject, which means it's not a logical fallacy at that point.
I'm looking for facts, not truths, since "the truth" can be changed by a person's perspective/bias.
Through this, I want to find out if Media Bias Fact Check is an authoritative source themselves. I'm tired of debating with people who don't actually know anything about a subject and want to call them in their BS. If this is the wrong site for this information, I guess that's a major failing of a site that calls itself "skeptics". Sub-edit: The fact that this question has been closed seems to speak for itself.
Edit 2:
This question is about the Media Bias Fact Check site, not the Ad Fontes Media Bias chart. They are two different things on two different sites, run by two different people. Nat's comment and Answer has nothing to do with the Media Bias Fact Check site I'm actually asking about.