# Should questions asking to verify a disproportionate statistic be allowed without meeting certain criteria?

My question is motivated by questions such as:

To me (and this is where I'm looking for correction and/or clarification) it seems as if this would be impossible to confirm or debunk without defining either

1. An acceptable range that is defined as being "in proportion" or within normal limits.

For example, a bag of skittles is expected to contain an average of 56 pieces of candy(1), and close to 20% of each of the five colors. A variation is expected because the bags are machine filled based on the weight of each color(2), which can vary due to dyes and other factors.

Based on this data it is possible to define a proportionate range of color distribution, such that 50% of a single color would qualify as disproportionate.

2. An explicit definition of what is considered disproportionate. Drawing from the same example above, let's take two hypothetical questions:

• "Is it true that bags of Skittles contain a disproportionate amount of red skittles?"
• "Is it true that bags of Skittles contain a disproportionate amount (>50% by count) of blue skittles?"

The first is subjective because the phrase "disproportionate amount" is open to interpretation.

The motivating questions listed above contain neither 1 or 2, and as a result for at least one answer it is implied that the norm is supposed to be correlate with the sub-group's overall percentage of the population, but that is speculation.

I'm not looking to go on a revision spree on skeptics, but I do think this discussion would produce better answers on the site overall, and possibly serve as a precedent moving forward.

## References for Skittles lovers

• – Fizz Jan 27 at 3:41

If there is a notable claim that bags of skittles (generally) contain >50% blue skittles by count, then that should be the claim in the question.

If there is a notable claim that bags of skittles (generally) contain "a disproportionate" number of blue skittles, then it is not acceptable for the poster to arbitrarily decide that 50% is the cut-off threshold they they would personally accept. It is the claimant that gets to define what the claim means. We normally look for context clues to determine what they meant by the claim. (If it is impossible to tell, then the claim doesn't have much meaning.)

In general, science and statistics gives us a default meaning for these words. We provisionally accept that the skittles are disproportionately distributed if we can reject the null hypothesis (that they are randomly distributed with each skittle independently having equal probably for each skittle) at a statistically significant level.

For larger and larger sample sizes, we can detect smaller and smaller biases.

• You're ignoring here that "disproportionate" can mean various things. In general, one can give an answer on the pure correlation and possibly another if (some) confouders are excluded (i.e controlled for). A typical example of confounder with race is poverty... although that gets to the inevitable discussion of structural racism etc., i.e. why is poverty more correlated with one race than another in the first place. So I think such questions can be bad in the sense of not admitting a clear "best" answer unless "disproportionate" is spelled out what it means in the claim. – Fizz Jan 27 at 3:24
• Frankly users here seem only satisfied by the simplest answers, judging by the "meh" reception my answer (unlike the q) got here skeptics.stackexchange.com/questions/12258/… – Fizz Jan 27 at 3:29
• @Fizz: I would suggest some other factors are relevant for your answer's reception, but this comment thread isn't the place to discuss that. – Oddthinking Jan 27 at 4:00
• @Fizz: It's not that I am ignoring it, but I am focussing on the proposed solutions being inappropriate. I explained that we look for what the claimant meant. If the claimant was ignoring confounding factors, then go ahead and mention why their analysis is very shallow - in an answer that first shows whether the actual claim is true or not. – Oddthinking Jan 27 at 4:04

Stack exchange used to, and still officially does, support a notion of "good subjective". For this site with such questions I feel like the answer can leave the definition however it wants, point to an authority that defines it, or forget that all together and just point to the numbers. In other words, this is a problem that good answers should solve.

In my personal opinion, proportionate means relative to the whole. If X is Y percent of Z then we should expect that X is represented Y percent of the time in event A, where Z (not just X) is subject to the causes and effects of A.

To be more precise, the 2nd question you linked to, while having the "disproportionate" in title, actually has a more narrow/precise claim:

Unarmed black people were killed at 5x the rate of unarmed whites in 2015

(The title however should probably be edited to match the claim.)

In contrast, a question that more squarely fits the profile of "disproportionate" vagueness is one that I alas bothered to answer. And of course I got endless arguments in comments that the studies I found were not accounting for confounders well enough (although they were trying to). So I'm not answering those kinds of questions anymore.

The first q you've linked to is probably even worse as it's a claim-by-linked-video (which you'd have to watch to figure out what they mean exactly). DV, VTC and move on. (As it happens, the video was removed by the source [youtube], so it's basically impossible to fix that question, assuming the claim was even clear in the source, which might be too much of an assumption. Knowing a bit about right-wing arguments on the matter, I suspect the claim may have been relative to crime or something like that, seeing who the claimant was and what else they were saying on the same topic [probably in the same video].)

Yep, if you want to be strictly technical about it

The term “disproportionality” refers to the ratio between the percentage of persons in a particular racial or ethnic group at a particular decision point or experiencing an event (maltreatment, incarceration, school dropouts) compared to the percentage of the same racial or ethnic group in the overall population.

So anyone (conservative, racist etc.) using the term incorrectly can be easily slapped here with this (def) and then statistics that back it. There's nothing subjective about that definition, except some people may not know it. I don't know if this really makes interesting question(s) if the claimant was really trying to say something else, if you read a tiny bit more of their claims, usually in the same piece.

(Aside, but related, others use the term "systemic racism" incorrectly. There was a recent q here centering on that but was deleted.)

Of course, there are conservatives who use the "disproportionality" term correctly, but that also makes rather boring questions if you just want to focus on the data, e.g. (to reuse a source here):

The real problem is that for whatever reason blacks commit a disproportionate share of the crimes.

Should a questions only inquire about the data claim in the latter part/half of this statement? (Maybe it should do just that?)

And that's a disproportionality that's within the Overton window for the mainstream-ish media to print. I can think of some which are probably much less kosher and would result in questions that probably aren't answerable with hard data, even though the question may be/seem "objective". (Most ethnic/racial stereotypes out there, I suspect.) If one can find a reasonably scientific formulation of these some other (e.g. using the proper d-word terminology, or just "X are more Z than Y") are they on-topic? Food for thought.

To wit, here's one along those lines that's still within the aforementioned Overton window (and actually answerable):

Do Greeks think of themselves as the most hardworking in Europe, even though they are disproportionately unproductive?

True-true. HNQ?

• "endless arguments"? There's one person with whom you engaged, resulting in a 10 comment exchange. I'm not sure I understand the complaint, other than perhaps you found that one person grating or something. I guess my point is that you make it sound like you're describing the community at large based off a short comment exchange with a single user. – fredsbend Jan 28 at 4:38
• When you ask "Should a questions only inquire about the data claim in the latter part/half of this statement?" Perhaps you need to edit and clarify if you mean to say that individual questions need to pick what they will fixate on, or whether all questions on the site must be only "the latter part". The "a questions" typo leaves far too much ambiguity. – fredsbend Jan 28 at 4:47