Is the scope of this site unlimited?

Question

I have a difficulty with trying to understand which questions would be off-topic on this site. I understand questions have been closed because they were written in a way that would degenerate into endless discussion, but that doesn't limit the what can be discussed here, only how it can be discussed.

Half-jokingly: If my question "Is A X?" is off-topic I can magically make it on-topic by rewriting it as "Has it been proven that A is X?", like you can make any question on-topic for Stack Overflow by writing it as "How to do Y (programmatically)?"

The current FAQ states:

If you have a question about the accuracy of public claims made in the media or elsewhere, if you're interested in the evidence behind what you hear or read, then you are in the right place.

The first part of the sentences seems to be included in the second part of the sentence, leaving us with "If you want to know whether anything you heard or read is true, come to us." That's great, but doesn't that require experts in every field to be able to answer these questions? The Gadgets site was closed because it was impossible to answer questions about any random electronic gadget, most (all?) questions were migrated to the more specialized Apple or Android sites. Won't this also happen with the Skeptics SE site, with the questions going to Physics and other science-related sites? After all, I would think they aren't interested in unproved answers either.

I am not a participant in the Skeptics site (yet?), and I wish you all the best with making it a great place to find good information - I'm just worried that this is a huge task and that it will be impossible to handle the unlimited amount of possible on-scope questions.

Answer 1

I'm sure it won't surprised you before, but we've discussed this problem in early private beta. Here's what I had to say then:

We'll need some criteria other guidelines to determine what is or isn't on-topic. Otherwise, the scope of this site would be, well, everything.

As I said in What should be in our FAQ, I believe Skeptics.SE should be to science what factcheck.org is to politics.

That is, I think we should cover the basic questions on science that are propagated by non-scientists (Al Gore, Jenny McCarthy, mainstream media, etc.) and hearsay ("you get a cold from being cold" and the like), but we leave the real questions to the real pros (i.e. Physics.SE, Biology.SE, Chemistry.SE, etc.)

By those guidelines, "Is it dangerous to have several vaccines at the same time?" is on-topic, but not all medical questions will be.

On the other hand, question around the lines of "Is there a substantial difference in rehabilitation therapy and expected outcome for a hemorrhagic stroke patient versus an ischemic stroke patient?" would belong elsewhere. They're too technical and thus beyond the scope of this site.

In other words, we limit ourselves to claims addressed to laymen. That has the benefit of usually, but not always, allowing us to easily fact-check the answer without necessarily being experts on the topic. In a sense, Skeptics rewards greatly people with good web search skills.

We also require referencing of all significant claims, so we can verify the information:

The voting system of Stack Exchange is largely meant to relax any need for specific policy regarding what constitutes a valid answer (and by and large it accomplishes this) — however, due to the nature of Skeptics, the community needs to enforce the idea of no original research to encourage healthy voting.

Users are required to reference all significant claims they make in their answers.

Those two policies reduce the need for experts in all topics.

While it require at least some basic understanding of science, and while being an expert allows you to write better answers, those two policies are helpful at preventing bad answers from being upvoted.

Answer 2

The scope of this site is not really unlimited. What questions should be allowed has been discussed extensively here on meta. Questions here must have the following characteristics:

• There must be a relevant claim (well known or documented)
• The question must be about the claim and not about the subject. We are expert in analyzing what the known facts are:
  • OK: "Jeff says 23 monitors are better than 5 for coding. Is there any research on the subject?"
  • Not OK: "What is the optimal number of monitors for coding?"
• We don't allow anything philosophical or which is not testable through scientific research (statistics, peer-reviewed papers, government data, etc.)
• The claim must be addressed to laymen

Surely more constraints will be needed and become apparent as we grow as a site - but so far, so good: we have way more questions closed as off-topic than migrated out to any other site.

Answer 3

Possibly. Good question.

A few distinctions I can think of, off the top of my head.

1. Ideally, questions here should be based on actual problems you face, not just blue sky daydreaming about "is this {completely random thing} scientifically valid?" This is basically covered by the standard "don't ask" part of the /faq.

2. If, for whatever reason, it is impossible to come up with any kind of scientific evidence to affirm or refute the question, then it's probably off topic.

Important distinction: you don't actually have to be an expert in the topic to locate and share scientific evidence supporting or refuting a given question. If anything, the skeptics site is optimizing for researchers who are highly skilled at producing evidence and citations, preferably of the "strong science" type.

None of that means the site is guaranteed to work, or that the scope is correct -- it's still quite early in the public beta -- but those are my thoughts so far.