16

We keep on getting asked questions on ISIS. I have no problem with the subject, but most of these questions are about current news, have little-to-no evidence and are all around receiving weak or downright poor answers. This in turn generates nonconstructive discussions and ill will.

Also, they seem to be focused on the shock value of the claims in order to attract upvotes.

We clearly need to address this in some way, because all the discussions are not good for the community.

What should we do to maintain quality on the site?


Explaining the problem better:

  • Our current policy is not to answer unless some positive evidence is found. In other words, proof that the claim is either wrong or right. These questions are unanswerable by our current standards, until the events are not current and historical evidence accumulates.

  • Our current policy is to only allow better sources to confirm or dispute a claim. You can't dispute a news item with another news item. You need a better source, like a primary source.

  • We do not allow questions based on original research. The reason that we are experts in evaluating the quality of studies and of reference, not at actually producing studies or investigative journalism.

  • We only want conclusive answers, not answers which are predictably going to be obsolete in weeks if not days.

  • Voting is clearly not punishing these bad quality answers. The reason for this is "drive by" upvotes coming through the hot questions list.

The problem is that these questions get continuously answered with posts that are in direct contradiction of one or more of these criteria. I can see only three alternatives:

  1. We don't change anything regarding policies on questions or answers. This means deleting all these poor answers even if highly upvoted.

  2. We don't allow answers until the dust is settled, thus discouraging "let's debate the news" answers and voting via putting the question "on hold".

  3. We change the site rules so these answers are acceptable. I strongly oppose this option.

  • 3
    Perhaps combine the questions into a single "Is there reliable evidence for atrocities committed by ISIS?"? – Dikran Marsupial Aug 13 '14 at 13:00
  • @DikranMarsupial Can you make this an answer so it can be voted upon? – Sklivvz Aug 13 '14 at 13:09
  • I have argued in the past for a "not answerable until more journalism is done" close reason, but the problem is that it makes a presumption about a lack of evidence. I cant think of a standard we could use that doesnt make some presumption like that. We could make a one-time decision that applies only to ISIS though. – user5582 Aug 13 '14 at 13:09
  • I worry about content based decisions rather than standards based decisions. That is how ive always defended criticisms of censorship here. – user5582 Aug 13 '14 at 13:11
  • 1
    I don't think that a content-based decision is applicable. We should be able to determine what is actually wrong and make a general rule out of it. – Sklivvz Aug 13 '14 at 13:12
  • @sklivvz okay, im fully on board with that. – user5582 Aug 13 '14 at 13:14
  • 1
    Related question: meta.skeptics.stackexchange.com/questions/2711/… – Oddthinking Aug 13 '14 at 13:43
  • 4
    Yes, enough already! I get it, those ISIS guys are nasty human beings... – Larian LeQuella Aug 14 '14 at 2:02
  • 1
    @LarianLeQuella, it's worse than that, after a while it almost seems like we're becoming an apologetics extension for ISIS: "Oh no, they didn't do that, or that". – Benjol Aug 14 '14 at 7:39
  • 5
    @Benjol distasteful as it is, I'm in favour of debunking false claims about ISIS. – Andrew Grimm Aug 14 '14 at 8:06
  • 2
    Yes, the problem is that we can't go beyond a generic "don't trust all you read in the news, folks" which is an extremely poor performance given the number of questions. – Sklivvz Aug 14 '14 at 8:24
  • 1
    YES YES YES YES YES! +100. – George Chalhoub Aug 14 '14 at 10:15
  • "Our current policy is to only allow better sources to confirm or dispute a claim" -- Is that a fact, that that's our current policy? I agree that mere reprinting/quotes of the same article as referenced in the OP add nothing, but what about other articles in other papers, which include/add more detail that the referenced article? Which identify the source of the claim/evidence, unlike the article referenced in the question? Which are independent confirmation of the quoted claim? Or which are made by a higher-reputation newspaper? – ChrisW Aug 18 '14 at 12:23
  • "We only want conclusive answers" Really? We prefer conclusive answers, that's true, but the bottom line is that we only want answers which reference evidence. If there's evidence for and against the claim, then both sets of evidence are on topic, and the answer would be that more research would be needed to make a definition conclusion. – ChrisW Aug 18 '14 at 12:27
  • 1
3

I think this is a special case of what was once called "Too localized." That is to say, the evidence available in determining the answer changes rapidly. What appears to be the correct answer today may be proven wrong by information discovered tomorrow.

Would it therefore be appropriate to put a ban on current events questions? Of course, that would require a meaningful definition of "current events" and I would propose something along the lines of:

Questions about unresolved current events and issues currently under investigation by a court of law, government, or other similar investigative body are off-topic.

  • I like this in theory, but we would need to be careful in applying it. A historical question that relates to a current event (like the "stolen land" Israel/Gaza question) would be on topic, even if questions about the event aren't. This close reason would also address questions like this one which are basically "Does the legal complaint say what the media says it does?" – Bobson Aug 13 '14 at 18:37
  • What if we changed "relating to" to "about"? Would that address my issues while still preserving the intent? – Bobson Aug 13 '14 at 19:03
  • @Bobson: Done. Of course, those phrases are usually synonymous, but they do carry slightly different connotations here... so I agree with the change. – Flimzy Aug 13 '14 at 19:05
  • I like this: what about that we put current event news with no answers on hold and reopen them as appropriate? This would prevent answers but not voting or editing, and the community would have all the "undo" power needed to make sure moderators don't go rampant. It would be up to the community to flag or vote to reopen questions which have become answerable. – Sklivvz Aug 14 '14 at 8:28
  • That seems reasonable to me, @Sklivvz. – Flimzy Aug 14 '14 at 9:20
  • Now, this is an interesting use of closing... Sort of the reverse of what Mi Yodeya does for Purim Torah. You could even put together a SEDE query that would turn up questions likely to need revisiting. – Shog9 Aug 14 '14 at 13:31
  • 3
    Bonus points for naming the close reason "INSUFFICIENT DATA FOR MEANINGFUL ANSWER" – Shog9 Aug 14 '14 at 13:32
  • 3
    Ok, I've prepared the off-topic reason and bounced it off to the other mods for review. – Sklivvz Aug 14 '14 at 13:51
  • 1
    @Shog9 google.com/search?q=INSUFFICIENT+DATA+FOR+MEANINGFUL+ANSWER suggests these questions can reopened, in about ten trillion years from now. – ChrisW Aug 14 '14 at 14:35
  • 1
    This is what I've taken to calling an "allergic" reaction - where we ban something harmless or useful (like all current events) because they are similar or correlated with an actual threat (questions about which there is no verifiable truth. Some of the most interesting questions are current, and information is most useful at the time misinformation is most visible. What we want to address is questions that appear to lack verifiable facts. But that's hard, because we often don't know what's not out there till we ask. TL;DR -Banning all current events would do more harm than good, IMO. – Jaydles Aug 14 '14 at 14:48
  • @Jaydles - Can you link an example or two of good current event questions? – Bobson Aug 14 '14 at 15:00
  • @Bobson, here are two I found quickly that would be very likely to get caught up in that kind of rule, since they are clearly about current events, even if they're not what it was aiming at: skeptics.stackexchange.com/questions/22345/…, skeptics.stackexchange.com/questions/10507/… (I know the intent is much narrower, but don't forget that "too localized" quickly became a ban on almost any question specific to a place. – Jaydles Aug 14 '14 at 15:05
  • @Jaydles - Both good examples. That's exactly the kind of question I was attempting to preserve by suggesting changing the original "related to" to "about", although in this case the "unresolved" part might be more applicable. I think there's a very fine line to walk there. I think this close reason is the right solution, but we should be conservative about applying it. – Bobson Aug 14 '14 at 15:09
  • 1
    @Jaydles I am fine with both cases being closed for a week or two in order to give a more balanced answer. Remember that we are not banning news -- we are banning news when they are current developments. – Sklivvz Aug 14 '14 at 15:11
  • 2
    @Sklivvz The ISIS questions (beheading, burial, drinking blood, women as slaves) were also all allegedly concluded/factual/past events. So it's not about whether the events are in current development, it's about whether there has been or will be further/sufficient reporting and investigation of the evidence? – ChrisW Aug 14 '14 at 15:47
6

I think most of the ISIS questions are getting good-quality answers.

Did Snowden reveal that ISIS is a USA-Israel tactic? has received three answers. One with 20 upvotes and +75 for bounty (unawarded?), one with 5 upvotes and an acception, and one with 3 upvotes from a user with a history of so-so answers

Is ISIS beheading children in Mosul, Iraq? has one answer with 15 upvotes and an accept, which seems to accept the need for skepticism.

There is reason to be skeptical, as the recent news about the call for female genital mutilation by ISIS and some other grand claims were wrong too.

The other answer, though only having a net score of +1, seems to have done some research work.

To use more Wikipedia policy, Skeptics.SE is making the internet not suck.

  • The beheading question has terrible answers, which are very localized and non-specific at the same time. One resorts to linking a google search as reference and the other to other question to "prove" that we need to be skeptical about news... and then proceeds to cite more news. Upvoted != good quality – Sklivvz Aug 18 '14 at 8:51
  • The Snowden question has four answers only one of which is based on good evidence (yours). The two most upvoted ones say "I could not find evidence..." and the last one, with negative score, cites a blog post as source. Your answer is clearly the only one up to our standards, but it's not being voted accordingly. – Sklivvz Aug 18 '14 at 8:53
2

In this comment, Jaydels wrote,

This is what I've taken to calling an "allergic" reaction - where we ban something harmless or useful (like all current events) because they are similar or correlated with an actual threat (questions about which there is no verifiable truth. Some of the most interesting questions are current, and information is most useful at the time misinformation is most visible. What we want to address is questions that appear to lack verifiable facts. But that's hard, because we often don't know what's not out there till we ask. TL;DR -Banning all current events would do more harm than good, IMO.

IMO the real motives are revealed in the OP:

  • We keep on getting asked questions on (the same topic)
  • They seem to be focused on the shock value of the claims
  • All the discussions are not good for the community

The fact that the claims are based on unverifiable evidence is another factor but perhaps not the main or only factor. You could ignore (not try to make a point of) the "it's hard to find other evidence for this" aspect of the problem, and instead concentrate on three bullet-points above, for example:

  • This situation happens rarely (don't need a new, general rule for it)
  • When it happens it can be dealt with by moderators using their existing discretion / freedom and power of execution action / benevolent-dictatorial powers / intra-moderator consensus, e.g. as outlined in this answer.

IOW a moderator could close it with a comment like, "We've already had enough questions like this; more like this one wouldn't be good for the community", perhaps adding some words like "community values".

1

I don't believe the questions should be grouped into a bigger question + answer set just on the basis of them being similar. As Sterno put it on the gaming stackexchange site:

We are not a wiki. We should not go out of our way to act like one.

I think the problem with these claims is that they come from current news events. Several will be cleared up (or forgotten about) within a week or two. I am not in favor of banning current-event questions, rather I would suggest a compulsory wait-time before they show up. Tag + shelve, and open back up in a pre-determined time. A similar thing happened to one of my questions, and I see perfect sense in it.

  • 1
    How do you propose we do that? – Sklivvz Aug 13 '14 at 16:57
  • @Sklivvz By closing the question if the topic is too recent, and allowing it to be reopened later if anyone remembers or asks about it after the compulsory wait-time has passed? – ChrisW Aug 14 '14 at 2:08
1

+1 to Spork's answer for the idea of letting the question chill for a while. In my experience some questions such as this one needed at least a week, for the story to be investigated and the evidence to be published.

  • The value of this site is that it publishes evidence, maybe 'sober second thought' about a subject.
  • Conversely perhaps it's not the purpose of this site to express an opinion on subjects for which there is (as yet) insufficient evidence, i.e. on "breaking news" stories.

Flimzy's answer said,

Of course, that would require a meaningful definition of "current events"

One way to do that would be to say that a (notable) claim must be at least one or two weeks old, before it's allowed as the topic of a question.

Alternatively, say that new/recent claims are allowed, however the subject/topic of the claim must be an alleged fact that is at least a week or two old.


Enforcing this rule could also be seen/excused as an extension of wanting the OP to do some minimal research of their own before posting a question: IMO some of the "ISIS" questions in particular (and "current affairs" / news questions in general) suffer from, "This question does not show any research effort; it is unclear or not useful".


Enforcing this rule also fits with the site's "no original research" policy: i.e. we're not allowed to research topics, therefore other people must have had time to research and publish evidence on topics.


There's a real-world value to allowing 'fastest gun in the west' answer on StackOverflow (i.e. it's helping people to fix their programming problems in real time).

It's much less clear that it's worthwhile to encourage real-time answers on this site.

  • The trouble with a "two weeks" limit is that wouldn't address issues like, say "Did OJ Simpson murder Anna Nicole Smith?" That trial lasted over a year... And there are also other cases where a claim is fully addressable immediately. (Not that any of this means my suggested verbiage is anywhere close to on-target!!) – Flimzy Aug 13 '14 at 18:41
  • OJ Simpson If the question were posted while the trial was happening, the answer could point to evidence presented in the trial (and elsewhere). I couldn't necessarily/accurately predict the future (how the trial will end), but there's plenty of evidence to present. – ChrisW Aug 14 '14 at 0:06
  • @Flimzy Charles Shaw wine That's example of a recent claim about an old fact, the old fact being the previous years' vintages, IOW answerable per the "say that new/recent claims are allowed, however the subject/topic of the claim must be an alleged fact that is at least a week or two old" suggestion. – ChrisW Aug 14 '14 at 0:08
1

I think the biggest risk with 'hot off the press' topics is of a rapid loss of interest as the 'next big thing' comes along: leaving in its wake a load of half-answered (and half-arsed) answers, which may never get cleaned up.

And even if they do get cleaned up, it's broken windows in the intervening period.

And cleaning up is a heck of a lot of work. How many hours have we collectively put into trying to salvage this question and its answers? Was that time well spent? (OK, that wasn't a fast-moving current event, but you get my point: leaving 'naff' question because they 'might get better later' is NOT productive).

0

The upvoted and accepted answer to What is the standard of proof here? says,

Different disciplines will have different standards. An historical "proof" cannot be as convincing as a chemistry "proof". Hard science experiments must be reproducible, other sciences can only examine existing evidence. Some statistical sciences can only examine correlations. The standards of proof differ from field to field, and from case to case.

I think that the important bit is to disclose the level of confidence we can attribute to a claim.

We are not interested in passing judgement, but only to present and summarize the available evidence.

In the OP you write,

most of these questions are about current news, have little-to-no evidence and are all around receiving weak or downright poor answers.

Are the answers poor? If an answer provides some evidence but the evidence is insufficient for proof, is that a poor answer or is it just poor evidence?

IMO:

  • If an answer provides any evidence (even circumstantial evidence, insufficient for 'proof', or a media report) then the answer is on-topic
  • Users can upvote or downvote an answer based on the quality of the evidence
  • Users can use comments (or answers of their own) to criticise the quality of evidence presented
  • An answer with poor evidence isn't necessarily a poor answer: it may be the best answer possible given the poverty of the evidence available

Better evidence can always be presented later, if better evidence becomes available/known. Until then:

  • Even poor evidence might be better than no evidence
  • It might be worth knowing (worth being able to say in an answer) that the evidence is poor
  • 1
    Yes, they are poor. They present irrelevant evidence or non-evidence ("I can't find anything"). – Sklivvz Aug 13 '14 at 23:39
  • IMO you and I seem to disagree about the purpose or standards of this site: your opinion is that poor evidence shouldn't be presented at all (i.e. that it's better to have no answer), whereas my opinion is that although good evidence is better than poor evidence (if good evidence exists), sometimes only poor evidence is available and that's better than no evidence at all. Your words "irrelevent evidence or non-evidence" imply that's totally worthless: which is IMO incompatible with saying "We are not interested in passing judgement, but only to present and summarize the available evidence." – ChrisW Aug 14 '14 at 0:01
  • No, it's not like that at all. I am stating that just adding random evidence does not make it weak evidence. Making a bad argument does not equate to better than nothing. You are conflating weak/strong evidence (which is indeed an opinion, and is judged by the community with votes) with valid/invalid/no evidence (which could result in deletion). If an answer judges a claim based on the reliability of where it's published, it's not providing weak evidence, it's providing invalid evidence. If an answer is based on not finding evidence, it's providing no evidence. This is not an interpretation. – Sklivvz Aug 14 '14 at 0:24
0

Also, they seem to be focused on the shock value of the claims in order to attract upvotes.

Most of the questions are about claims which do have an inherent shock value , the shock value makes the claim popular and notable. Unfortunately such claims are also basis for foreign policy of the western world including united states hence the importance of answering such questions early on with the available evidence is far important and useful then letting the claims float with available evidences that contradict it.

Rather than closing the questions altoghether which may even go into oblivion as people would later forget about it, we should counter the individual answers for lack of evidence and remove them on sight.

  • I agree that that motive is kind of what Skeptics can be for. But, the users of the Skeptics site cannot do that unless/until reliable evidence is created and publicly available for reference. I suppose the current close reason is intended to prevent (poor-quality) answers from being posted before any (high-quality) evidence is created and publicly available. – ChrisW Aug 18 '14 at 8:19
-1

Perhaps combine the questions into a single "Is there reliable evidence for atrocities committed by ISIS?"? Most of the questions seem to be trying to tease out the same basic issue, but with different examples, and I suspect the answers will be much the same for all of the examples (more journalism required before the question is really answerable - as Articuno suggests).

I didn't originally post this as an answer as I don't know if it is possible, or would cause the moderators more trouble than it is worth with the ensuing complaints! I definitely agree though that there is diminishing value in having so many closely related questions.

  • I like this approach, except for the specific wording. "Is there reliable evidence for" is implicit in every question. – user5582 Aug 13 '14 at 13:17
  • 1
    "Have atrocities been committed by ISIS?"? – Dikran Marsupial Aug 13 '14 at 13:20
  • Although... This could conflict with our advice to ask about specific to examples rather than the broad category when lack of evidence or practical falsifiability is an issue. Here, we'd be doing the opposite. – user5582 Aug 13 '14 at 13:20
  • Related: meta.skeptics.stackexchange.com/q/2748/5582 – user5582 Aug 13 '14 at 13:23
  • I think it would still be O.K. as the combined question would include the specific examples mentioned in the original questions, so answers could provide evidence on one or more of the specific examples. – Dikran Marsupial Aug 13 '14 at 13:26
  • Yes, that would distinguish this case. Looks good! – user5582 Aug 13 '14 at 13:41
  • I would personally like the combination of the questions and answers, as I think answering each disproportionate claim with 'probably not' leads to a skewed vision on the subject matter as a whole. Instead of being merely skeptical, people start to assume that ISIS never does anything wrong. – Spork Aug 13 '14 at 15:42
  • But there are examples on other stackexchange sites too e.g.: meta.gaming.stackexchange.com/questions/7901/… – Spork Aug 13 '14 at 15:42

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .