7

I had an interesting discussion with the poster of a popular answer that has been in the Hot Network Questions for a couple of days. I think the discussion is important, so I'm copying it here to Meta as a means to get the views of others.

I'm not saying the answer is wrong (or correct.) I'm talking about requirements for sources.

I'm wondering what the community thinks.

The reason I'm including my comments here is because they summarize perfectly the views I wanted to express in this post.


Fiksdal: IMO this answer could be improved by including the actual numbers and hard statistics these statements are based on. Or at least some notable examples of numbers.

SalvadorDali: In your answer you told Numerous studies but provided nothing to back it up. All you have is some NY times article, and report that tells that innumerable studies have confirmed.

Fiksdal: The question is about statistics, this answer cites no statistics.

Fiksdal: @SalvadorDali Exactly. Can you imagine if someone posted an answer here, but the sources of such loose statements were Breitbart or Fox News? They would have (quite rightly) been downvoted and asked to provide hard numbers and statistics. Why should we hold the WSJ to a different standard? We know FN and BB are ideologically biased and dishonest, but are we just going to give the WSJ carte blanche? This is inappropriate for a skeptics site. This answer desperately needs hard statistics, and in the absence of that, it's very low quality as per the help centre, IMO.

rougon (OP): @Fiksdal, you are right, the WSJ journal is definitely biased--it's a conservative paper. If you look at the sources, they discuss the difficulty in looking at national crime statistics. Or how about: there are 0 legitimate places that provide statistics that undocumented immigrants commit more crime.

Fiksdal: @rougon I don't care what political affiliation the WSJ purports to have. I don't care who says what. This is Skeptics and for a question about statistics we need answers that cite hard statistics. I don't know why this is upvoted so highly. You're saying that there are no credible statistics on this. (I think I agree, BTW.) If that's the case, then how exactly did the WSJ come to their conclusions? Do you know? Or did you just take the WSJ at their word? Because that would be the opposite of skepticism, and completely contrary to the help centre. CC: @SalvadorDali

rougon: @Fiksdal Your comment seems to charge the WSJ with bias. If you reread the question, it only mentions statistics at the end. Also, if you look at the studies mentioned, stats are discussed, albeit more in passing than you would like. They may not have smoking gun stats, but the various studies and reports seem to show that there is a clear trend. Now, you are more a math person than me, but it still looks like pretty solid evidence, certainly enough to make it appropriate for the site.

Fiksdal: @rougon It's human nature to be biased. I'm charging all of us with bias. I know nearly nothing about the WSJ. And I haven't read the whole article you link to either. If the article cites hard statistics, then you need to include those in the answer itself. Yes, the question asks for statistics in the title itself by the word "likely". Probability is only assessed by hard numbers. When I Googled this, I found a Fox News article saying the opposite. But I didn't find the statistics convincing enough to be included in an answer. And you don't even mention a single number in your answer.

rougon: @Fiksdal Fair enough. I think we have different expectations about sources, evidence, and what the answer calls for.

Fiksdal: @rougon Clearly we do, if taking someone's word for how likely something is is your idea of skepticism.

rougon: @fiksdal I don't think I'd characterize my answer as blind trust in someone's word, but I respect and appreciate your commitment to quality statistics.


Does this answer have solid sources in accordance with the requirements put forward in the help centre?

The answer is highly upvoted, but so is the other answer saying "we don't know". Also, it seems likely that the Hot Network Questions has something to do with the scores here.

The OP has claimed that the question doesn't ask for statistics. The question asks for how "likely" something is. This is a scientific site, and what can "likely" mean, other than "statistically likely"? Do loose statements from a number of "credible" or authoritative people suffice?

  • I am not quite sure about why you are showing the comment thread. We certainly do not want to move the back-and-forth discussion here (there's Skeptics Chat for that). If you have a specific opinion on this, do use an answer and so we can vote on it :-) – Sklivvz Oct 24 '16 at 16:47
  • 1
    @Sklivvz I'm showing the comment thread as a summary of my views on the subject. Comments are often deleted, so I figured meta would be a good place to post it. The reason I'm posting my comments here is because it's exactly what I wanted to say in this meta post. – Revetahw says Reinstate Monica Oct 24 '16 at 16:53
  • @Sklivvz Answered. – Revetahw says Reinstate Monica Oct 24 '16 at 16:59
  • 4
    Totally agree with you. I like to read skeptics because interesting and hard questions are answered with rigor and details, without PC and other bullshit. But this answer was different. I was surprised to see that it got so many upvotes answering completely different question and backing it up with dubious articles – Salvador Dali Oct 24 '16 at 19:17
  • 2
    I really think someone ought to ask the question about why so many notable sources claim a huge body of knowledge on this topic, but we can't find it. Does it not exist, can we not find it, is it paywalled, private research, etc. I don't want it to be me, since I already am distracted from my work far too much, and attaching my name to anything remotely political before US national elections are decided and all parties have achieved sufficient vote or conceded, will certainly net me more messages than I can handle. And even more than I care to read. – CWilson Oct 25 '16 at 13:04
  • @CWilson Agreed, that it s a very interesting question indeed. – Revetahw says Reinstate Monica Oct 25 '16 at 13:17
9

I think this is an example of a very bad answer. In my opinion, this questions is important and hard, so it requires hard data to back up the answer. I have seen no such data.

This is why I criticize this answer:

  • First of all it answers completely different question. OP asked Are illegal immigrants more likely to commit crimes. The first citation of the answer addresses immigrants (not illegal), the same is with second, third and forth. Only in the last article (only link is included), the author claims that the word illegal exists somewhere in the end of the article. BTW, as far as I know, at least 20% of US-based employees in Microsoft, Google, Facebook and many other big IT corporations are immigrants and legal immigrants are really different from illegal ones
  • secondly, the question does not have a credible support. I am really new to this site, but as far as I understood Wall street journal, Washington post and other newspapers are mostly viewed as opinions (if they are not backed up by real data). American immigration council claims that innumerable studies have confirmed but I have not seen any support for this (if you have many studies, it would not be hard to include at least one or two)
  • To be fair, "illegal immigrants" are still "immigrants", so unless illegal immigrants are explicitly excluded from the sources, it may well include both. – Alexander O'Mara Oct 24 '16 at 19:39
  • 1
    @AlexanderO'Mara you are totally right, the problem is that they skew the statistics. Let's assume that native's crime stat-rate is 0.05. If you have 100 legal immigrants and one of them commits a crime, your statistics now becomes 0.01 (which rejects the claim), if you have 2 illegal and one commits a crime, your stats is 0.5 (supports it). But when you aggregates them, you got 0.019 which brings you back in comfortable rejects claim state. And most probably legal immigrants are more common and commit less crime. – Salvador Dali Oct 24 '16 at 19:45
  • Agreed, I'm mainly referring to the sentence "The first citation [...] addresses immigrants (not illegal)" which perhaps could perhaps read "immigrants (not necessarily illegal)" since the distinction is not being made, we also can't assume the opposite. – Alexander O'Mara Oct 24 '16 at 19:49
5

I'm going to answer a slightly different question. Not "Is this acceptable?" but "Why haven't I taken any moderator action to delete it?"

The general policy on references is that they are required, they should be relevant.

However, there is no policy on moderator action whether the references are "good" or "reliable". That is appropriate, because the mods are not authorities on every topic who can rule on this.

We expect the answer to use sources that are more reputable than the sources in the question, but that doesn't mean taking mod action. There is no requirement for statistics, per se, but empirical evidence is something we look for. (This becomes difficult with some of the quotes and history questions, which causes some controversy.)

(Actually that question is practically a duplicate: Does SE.Skeptics specifically require "scientific" references?)


Stack Exchange has, as a bedrock, a philosophy that popular votes (+ accepted answers) are a reliable method of determining the best answer. Even though we know full well that that is an argumentum ad populum fallacy, and that science is not resolved with a vote, we go along with this here. We have amassed a community of critical thinkers that assess the evidence provided, and I am generally pleased with the voting.

However, when you take a political issue, and share it on the Hot Questions Network, we sometimes get an influx of less critically minded people, and the system falters. Go and vote up this proposal to help resolve this issue.

  • Thank you for a thoughtful answer and for bringing up the HNQ. There's a very useful thread about that on the main Meta, you might have already read it. Also, I very much support moderators taking a careful approach. Judging the validity of sources can be a slippery slope, so I totally understand this restraint. – Revetahw says Reinstate Monica Oct 25 '16 at 9:41
  • Sorry to dredge up an old issue, but the post was recently bumped. I see you and sklivvz have removed a heavily subjective and polemic section of the answer, and I appreciate that moderation. Here, you offer an explanation for non deletion, but I don't see it explicitly. Are you saying merely the references provided are relevant? Then are you saying that the those references are not bad, therefore they pass the bar? Is that the bar? References must be at least not garbage? – fredsbend Nov 1 '18 at 18:46
  • For me, it's a valid complaint that these are simple news sources that don't support their own claims. Deletion is probably heavy handed in this instance, but it should at least be addressed with a mod notice banner. – fredsbend Nov 1 '18 at 18:46
4

I do not have an opinion on the question that triggered this meta question. Or, I do not wish to express an opinion on that here. I really do not have an opinion on the verity of the answer that triggered this question. I do have a slight opinion on the quality of the answer to that specific question, and that is being discussed well in some other answers here. But, I think that this meta question is bringing up an underlying assumption important enough to require an answer of its own.

Does every skeptics answer always require statistics? The OP is working with the assumption that the answer is yes, but I suggest that this position is not unanimous, nor is it well documented. On the contrary, the sources quoted by the OP in the answer seem to suggest the opposite. I have not (yet) found sources that explicitly support that position.

Therefore, I am skeptical that it can, currently, be well asserted, with sources, that every skeptics answer requires statistics. And, absent statistics to the contrary, I chose not to support that position.

Now, going into the assertions in a slightly more detailed way: The question did use the word "likely", and did state "I'll accept statistics..." Obviously the questioner did want well sourced statistics. Me too. That does not mean I will get it. My understanding of this Skeptics.SE is that, while questioners can want what they want, the best answers are not necessarily what the questioner wants, but the best information/scholarship on the topic. Perhaps the specific answer in question is not 'the best' because it does not contain accessible statistics that are well sourced, but it appears that one claim of the answerer is that such statistics do not exist. If that were the case, then it seems an answer in that vein does meet the quality standards as currently laid out in the Help and FAQ.

Unless the sources (FAQ and Help) are inaccurate, and need updating (possible), it would appear that the best course of action, for OP and those with similar inclinations on the specific question/answer pair, would be downvote. Not claiming that the answer is inappropriate for the site.

One likely counter to that argument, that is a bit off topic here but still relevant in this specific case, is that, with so many upvotes (almost certainly from 'drive by' HNQ visitors), a few downvotes will not have the desired impact. I say that is a different problem, with a different solution altogether, that is being discussed elsewhere.

Hope my cold style does not lead anyone to think I am cold toward any single person or group of people. This is fun for me, and I don't know you people.

  • You raise good points. To clarify, I don't think anybody here thinks all questions have to cite statistics. I certainly don't. It's just that this particular question is by its very nature a statistical question, and it would have been even if it used different wording. So it's that exact type of question that I think requires statistics, not all. Regarding the help center issues and the rest, I largely agree with you. – Revetahw says Reinstate Monica Oct 25 '16 at 13:13
  • I don't think a lack of stats is really the problem, but a result of the problem. The issue with that answer is that it cites simple news sources that don't cite their own claims in turn. These sources claim there's a large body of evidence, then don't cite it. That's a major issue in my mind. – fredsbend Nov 1 '18 at 18:51
2

As requested by Sklivvz, I'm posting an answer so that people can vote.

I believe this question requires hard statistics. It's a question about probability, so it needs actual numbers. I've already outlined my reasons for this for this in the question itself.

Since the answer features zero statistics or numbers, I don't believe it satisfies the criteria outlined in the help centre. It's a question that basically demands hard statistics.

If the article linked to in the answer does feature such numbers and they are edited into the answer, I will upvote and approve of the answer.


I've been asked to clarify why I believe this is not in accordance with the help centre. From the "How to answer" section:

What, specifically, is the question asking for? Make sure your answer provides that.

This question asks for likeliness or probability of something. Not quoting actual numbers at all fails to adress the question about likeliness or probability. Indeed, "more likely to" means "statistically more likely to". What else would it possibly mean, on a scientific site?

Links to external resources are encouraged, but please add context around the link so your fellow users will have some idea what it is and why it’s there.

OP says the reference does adress statistics, but these numbers are not included in the actual answer. These numbers are absolutely vital and need to be included in the answer. Readers should not have to go the referenced article to find these statistics.

Moreover, the FAQ dictates that all answers cite sources. This answer does. But not a statistical one. There is an attempt to answer a statistical question without referring to actual numbers. That is not a proper source, IMO. If the source itself does refer to statistics, they have not been included in the answer. Either way, there's a problem.

Again, if convincing statistics are added, I will upvote the answer and commend it.

If credible, relevant statistics don't' exist then the only scientific answer is "we don't know at present."

  • Perhaps you could clarify exactly which criteria from the help center you say are not satisfied? I'm not finding anything about a requirement for statistics or numbers. – Alexander O'Mara Oct 24 '16 at 20:19
  • @AlexanderO'Mara You're right. Done. – Revetahw says Reinstate Monica Oct 24 '16 at 20:41
0

I believe this answer is a very good summary of the problems with the answer that prompted this discussion.

I agree it is a bad answer, that does not meet the quality standards of this community.

I also agree with this answer explaining why moderator intervention has not occurred for the post, and why it should not.

What is, in my opinion, missing from both of these answers is a suggestion for a course of action.

This particular problem happens to be fairly widespread among a variety of SE communities thanks to the HNQ list, albeit with different nuances specific to each community. Unfortunately, I have yet to see a really successful solution.

The best course of action remains to downvote an answer if you feel it doesn't meet the community standards, and, if possible, provide a better answer that does. This rarely fixes the problem of popular-but-poor-quality answers that have been artificially boosted by being on the HNQ list, but it can mitigate it.

-2

If this is inappropriate for an answer, I will be happy to delete it, but I wanted to chance to defend whether or not my answer is inappropriate for the site.

I do not think that the original question demanded statistics, nor do I think they are necessary or, in this case, particularly available. When I began looking into the topic, I found one article from a generally-conservative paper that contradicted the line or argument that Trump offered, so I found that notable. I found another report by a non-profit that agreed. Then I found a news article that cited 4-5 studies that agreed. Any investigation I made returned the same answer: undocumented immigrants were less likely to commit crime. The uniformity of the trend seemed to outweigh the need for precise statistics.

In addition, several of the pieces mentioned how difficult it was to collect hard data on the subject. I appreciate wanting numbers, etc., but they seemed hard to pinpoint with any accuracy and (perhaps) unnecessary in light of the general findings.

If enough people would like, I would be more than happy to edit the answer to mention any of this, but I find the charge that it is unfit for the site to be a bit reactionary.

For the people who think my sources are dubious or untrustworthy, I would really like to see you produce a good source that contradicts my findings.

  • 1
    "More likely to" means "statistically more likely to". What else would it possibly mean, on a scientific site? – Revetahw says Reinstate Monica Oct 24 '16 at 20:48
  • 2
    It would appear that this is a notable claim, made by notable and generally trustworthy sources, and many such sources, but not backed up by data. Sounds like a perfect question for this site. I wonder how similar such a question would be to the question being discussed in this meta post? – CWilson Oct 24 '16 at 20:51
  • @CWilson Great point. The quotes from that answer belong in the question part of Skeptics, and actual statistics belong in the answer part. – Revetahw says Reinstate Monica Oct 24 '16 at 21:14
  • 1
    You're talking about finding multiple studies and multiple sources, etc. Well, weren't any of them based on actual statistics? If they are, than it should be possible to find them, and include them in the answer. If they were not, then they aren't good sources. Regarding "I would really like to see you produce a good source that contradicts my findings.": Nobody (AFAIK) are claiming your answer is wrong. The burden of proof is not on others to show that your answer is wrong. (I'm not even claiming it is, I don't know.) The burden of proof is on you to show that your answer is correct. – Revetahw says Reinstate Monica Oct 24 '16 at 23:29
  • @Fiksdal The American Immigration Council have a number of stats and charts. They may not make a specific distinction between legal and illegal, but the Washington Post piece suggests the results are so weighted it almost certainly would not make a difference. – Alexander O'Mara Oct 24 '16 at 23:39
  • @AlexanderO'Mara Are you saying these stats and charts should be included in the answer? – Revetahw says Reinstate Monica Oct 24 '16 at 23:43
  • @Fiksdal Not necessarily, there's a lot of them, and I think the summary/interpretation is sufficient and the source very credible. – Alexander O'Mara Oct 24 '16 at 23:45
  • @AlexanderO'Mara If you're summarizing statistics, then the summary should include numbers, as in averages, etc. Also, this site is about facts rather than interpretations. – Revetahw says Reinstate Monica Oct 24 '16 at 23:48
  • @Fiksdal An interpretation of stats can be factual... – Alexander O'Mara Oct 24 '16 at 23:51
  • @AlexanderO'Mara Yes, and it can also not be. That's why it is not suitable for the answers section on Skeptics SE. – Revetahw says Reinstate Monica Oct 24 '16 at 23:52
  • @Fiksdal Why not? These interpretations are sourced by credible sources, and the stats are available. – Alexander O'Mara Oct 24 '16 at 23:54
  • @AlexanderO'Mara What constitutes a credible source for these interpretations? Some people think Breitbart and Infowars are credible, others think WSJ, others think NY Times, etc. Who gets to dictate what is a credible source? If these "interpretations" are actually based on statistics, include the actual statistics, and let people judge for themselves. Relying on some journalist for the interpretation seems like quite a slippery slope for a site about scientific skepticism. – Revetahw says Reinstate Monica Oct 24 '16 at 23:58
  • @Fiksdal I think you have a false faith in statistics as some kind of magic bullet. If you compare a source like Breitbart and the NYT, there is very clear difference in references to facts and details of evidence for claims. Take, for example, the WP article that I quoted: it referenced 4-5 studies and provided links to those studies, as well as evaluations of aspects of those studies. That is quite a difference than most Breitbart pieces I've seen! It seems like you think stats are objective and neutral. – rougon Oct 25 '16 at 0:15
  • @rougon Yes, that means the WP article is better, but not simply because it's the WP! It's because it cites those 4-5 studies. Giving certain newspapers extra weight is a well known logical fallacy known as argument from authority. The argument that a certain newspaper or journalist said something should be used with extreme caution, especially when dealing with statistical matters. I also view statistics with great skepticism, but they are tons better than the summary or interpretation of some journalist. – Revetahw says Reinstate Monica Oct 25 '16 at 0:20
  • Stats are objective and neutral (unless someone has tampered with or forged them) until people start misrepresenting them. In this case, I don't even know if they've been misrepresented or not, because the answer doesn't even show them. It cuts straight to the interpretations of people I know nothing about. – Revetahw says Reinstate Monica Oct 25 '16 at 0:25
-3

This looks like a big non-issue to me.

What does the question ask?

Are illegal immigrants more prone to committing crimes then the average US citizen?

It doesn't even ask for statistics directly, and "yes", "no", and "unknown" all seem likely perfectly acceptable answers to this question. The question does ask a question with an answer presumably based in statistics, so what is wrong with an answer that answers this question, with sources based on statistics? Anyhow, more on this below.

If we take a look at the full section from the How do I write a good answer?:

Answer the question

What, specifically, is the question asking for? Make sure your answer provides that – or a viable alternative. The answer can be “don’t do that”, but it should also include “try this instead”. Any answer that gets the asker going in the right direction is helpful, but do try to mention any limitations, assumptions or simplifications in your answer. Brevity is acceptable, but fuller explanations are better.

After reading some of the research, this looks like a "viable alternative". It does mention the "limitations" and "simplifications" of the sources (so I'm really not sure why this keeps being harped-on). One might argue that a fuller explanation is better, and "fuller" might include statistics, but there is a lot of said stats, and they do seem to require some interpretation to connect them to this specific question (remember, they are an alternative to direct stats, which are apparently unavailable).

"But I want statistics!"

Well, you're in luck! At least some of these sources have the statistics they used available, and because references are required, you can read them!

"Why not just let the statistics speak for themselves?"

I was under the impression specifically answering the question and showing why the sources back that answer was encouraged here. Anything less would seem like a dumbed-down version of the sources, which have both. Also, refer to earlier section about how these are alternative stats.

"But I want an answer that isn't binary, and shows the full spectrum!"

Well, I have more good news! This answer gives you some resources to make that answer! If you want this answer, what are you waiting for? Go ahead and post it!

"But I still don't think this answer is useful!"

Well, lucky for you, Stack Exchange thought to provide a system for this feedback! It's called voting, and it's very important. I'm not seeing anything here that makes this case special enough to require moderator intervention. There's also a bit of irony in expecting it to be remove for lack of statistics, when statistically the voting shows many people have found it useful, even before it was on the Hot on Network list.

Alternately if you are not looking for moderator intervention, I'm confused why commenting and voting were not sufficient to voice your concerns.

  • I want to clarify that I have never called for this answer to be deleted by moderators. That sounds quite drastic, indeed. (I probably wouldn't oppose it either, though, but I haven't called for it.) My motive for starting this discussion was merely because I thought it was important and will help keep the site policy on sources sharp, and maybe even get OP to improve the answer by adding statistics. I didn't envision having the answer deleted by moderators. Regarding votes, Oddthinking had some great points about that in his answer. – Revetahw says Reinstate Monica Oct 25 '16 at 10:03
  • I went to bed without hitting post, and then saw your answer after I posted this morning. Probably not much difference between our reasonings, I hope you don't take offense at my late entry. – CWilson Oct 25 '16 at 12:54
  • @CWIlson No worries. – Alexander O'Mara Oct 25 '16 at 13:34
  • @Fiksdal Fair enough I guess. I guess I'm just confused why this issue deserved more than some passing voting and commenting then. – Alexander O'Mara Oct 25 '16 at 15:51
  • @AlexanderO'Mara That's what meta is for. Discussion on meta of site policy and such things are great and a sign of a healthy community. Meta is a much better forum for discussion than comments on the answer. Meta brings a lot more eyeballs to the subject as well. – Revetahw says Reinstate Monica Oct 25 '16 at 15:56

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .