0

This is part of discussion brought up during 2013 election, and it have been a hot topic before. Rather than keeping discussion in comments of OddThinking's nomination I feel this is a better place for it.

By keeping a conservative and hard stance on references, some great answers are lost, the community gets a bit more close minded, and it alienates some people. The goal of the site should be to create as good answers as possible, and to minimize the amount of misinformation.

On the other side: by having a more liberal stance on references and being more open for original research, the amount of misinformation is increased. It risks lowering the credibility of the whole community.

Both brings good things, and both bring bad things. References ARE the core of the site, to maintain its credibility. That said, I would like to see the page open up more for original research. that said. If someone uses good deductive logic and makes proper answers based off that, it should be accepted as an answer.

That said, even with flawless logic, if what you base the logic off is wrong, your conclusion is wrong. This is not an easy thing to solve.

1

5 Answers 5

Reset to default
7

(I am giving a response out of obligation, because the question was originally directed at me. I don't feel I have a special insight here.)

There are two issues combined here.

  • Should all answers require empirical evidence to support them?

I think the community agrees with this, when worded this way. Does anyone object?

  • Is original research an acceptable form of empirical evidence, or do we only accept externally referenced sources?

I think this is where we are not all in alignment.

I don't think we have a clear and shared understanding of what original research is. I wonder if that is leading to disagreements.

Original research might mean introducing new empirical facts:

I keep ducks. I fed half of my ducks rye bread and half of my ducks white bread. I counted how often they quacked. Ducks fed rye bread quacked 75 times per hour. Ducks fed white bread quacked 25 times per hour.

Original research might mean synthesizing new conclusions from existing facts.

Research shows that 75% of bread sold in Germany is rye bread, compared to 2% in the UK. Research shows ducks in city parks quack 2.36 times faster in Germany. This shows a link between rye bread and duck quacking speed.

Original research might mean applying conclusions to a particular case.

Research shows rye bread increases ducking quacking speeds by a factor of three. The fast quacking duck featured on the OP's YouTube video is quacking at a speed consistent with being fed rye-bread. Therefore, it is reasonable to conclude that, rather than the video being fake, it is just showing a pure-rye fed duck, which is unusual for the OP who lives in the UK, but is not uncommon here in Germany.

In my view:

  • The last one seems acceptable; we always have to make step where we apply science results to the real world. The smaller the step is, the more reliable it is.

  • The middle one is unsafe speculation, and shouldn't be allowed. However, it is a blurry line between the last one and the middle one.

  • The first one is legitimate science, but this just isn't the forum for it. We have no way of verifying that the experiment actually happened and isn't just a troll. We have no mechanism for review by experts (in ducks, baking, statistics, geography), ethics committees, retractions, etc. Basically there is a blurry line between this and an anecdote. (Do I need to mention that anecdotes are unacceptable?)

I am aware that I could probably give examples of each of these forms of arguments just from the answers I myself have posted. I am not equally proud of all 300+ answers I have posted, and the rules aren't rigid. I just want to make sure that I am the first person to accuse me of hypocrisy!

3
  • You really put the word on the problem at hand, and what divides the community. Also the second example isn't good answer because of a logical fallacy :P A speculation of that type could be interesting to the discussion about the topic but it should not be an answer. It's more like a hypothesis that requires to be proven.
    – Wertilq
    Commented Mar 27, 2013 at 8:08
  • I offer my recent estimate of children of same-sex couples in California as a possible borderline case. Is multiplying four numbers together original research? I honestly don't know what the community reaction will be, and I look forward to learning from it.
    – Oddthinking Mod
    Commented Mar 27, 2013 at 13:14
  • 1
    I think it's a reasonable calculations. You clearly SAY it's an estimate, and show exactly what you do, and where you estimate. I would not call that borderline. You are making an estimate from a clear references, and saying it's an estimate, and from that saying 'the number is plausible'. To me that is a perfect example of how original research should be done. That is how science is done, use other science to make new science.
    – Wertilq
    Commented Mar 27, 2013 at 13:27
5

I'm not certain that I agree that being strict on requiring references translates into a close-minded community.

Regarding original research... I think in theory, the idea of allowing answers based off of logic and deduction sounds good.

In practice, though, I think there are some serious downsides and potential pitfalls.

Let's take some actual questions on the site as examples.

In the question Does torture work well as an interrogation, it is entirely possible to construct a coherent, logically sound answer explaining how it must be effective, citing countless historical precedents.

It is similarly possible to construct a coherent, logically sound answer explaining how it isn't effective, again citing countless historical precedents.

One of those answers, however, would be wrong.

Some of the questions asked here have answers that are patently counterintuitive.

In fact, we have some examples of answers based on logical deduction and original research that appear to be patently wrong.

We have to keep in mind the audience we want to attract, and the audience we don't want to attract.

Given the nature of many of the topics here, as well as human nature, the reality is that we naturally draw attention from people looking to promote agendas or who have heavy emotional attachment to certain social or political positions. I've seen this site get a fair number of answers, or even questions, that are nothing more than people explaining their pet theories. Almost invariably, these posts tend to be argumentative in tone, and lean heavily towards discussion, which is not what this site is about.

This site is successful because answers here tend to be well-documented and authoritative. If we become "more welcoming" to answers that sound good (are logically sound, well-reasoned, but have no reputable sources beyond that), the overall utility of our site will be diminished.

10
  • You have a good point there, but at the same time as you can use logic so can you use references in your example. You can take examples of when they have used torture and gotten out information, and reference to situation when it happened. While I partially agree with you, problem with weak references is there as well, as weak speculations.
    – Wertilq
    Commented Mar 26, 2013 at 14:27
  • I don't think the fact that references can be weak is really an argument in favor of speculation. Both are bad, but the gulf between weak references and strong references is significantly different than the difference between weak speculation and strong speculation (if there is such a thing).
    – Beofett
    Commented Mar 26, 2013 at 14:33
  • Calling it a speculation is kinda downtalking it a bit, but whatever. And while I agree nothing really beats a strong reference, and that speculations is kinda bad, there is more types of references than a reference to a scientific paper. And the site is so broad that it need to have room for other type of experts than only the science versed ones, or so I believe.
    – Wertilq
    Commented Mar 26, 2013 at 14:58
  • Speculation is the word you introduced to the discussion, not me.
    – Beofett
    Commented Mar 26, 2013 at 15:01
  • I did, I don't like the word still though, because it partially implies it's done without knowledge at all. That it's not truth based at all. That is off topic though. I should not have raised my voice over my own choice of words. The main topic is references.
    – Wertilq
    Commented Mar 26, 2013 at 15:04
  • I take speculation in a slightly more neutral fashion, in that I feel it indicates extending upon and beyond what is known by fact, but I take your point. There are certainly more types of references than just scientific papers. I just feel that they should be more notable than just rationalization and logic.
    – Beofett
    Commented Mar 26, 2013 at 15:15
  • To make an example. A guy have worked with pigs his whole life, have observed many things about his pigs, about things they eat and things they do. Someone makes a claim from PETA where they say pigs are friendly vegetarian creatures. He then tells the story about his nephew that got eaten by one of his pigs. Now this was a bit stupid example, since you can just reference any zoological homepage/book about pigs saying they are omnivores, but still the guy working with pigs provides expertise, and first hand experience that should be collected, SOMEHOW. It's information going to waste.
    – Wertilq
    Commented Mar 26, 2013 at 15:20
  • 3
    Unfortunately, the issue with your example is how do we know that guy is telling the truth? The sad reality is that people do come to SE to troll. Anyone can come and say they work with pigs, and we have no way to check.
    – Beofett
    Commented Mar 26, 2013 at 15:32
  • Yes I agree that IS a problem, but it's not impossible to prove, if they was interested in it, but it's quite a bit of work. That is the problem, and that goes back to what you originally brought up in your post.
    – Wertilq
    Commented Mar 26, 2013 at 16:20
  • 1
    To be clear, logic is not forbidden at the moment. Answers which do not rely on proven facts are. Our FAQ specifically says that this site is about looking at facts behind claims. Answers that do not do that, have nothing to do with us.
    – Sklivvz Mod
    Commented Mar 26, 2013 at 20:21
2

This is where I get to wave my "it depends" flag because it really does depend on the quality of the answer that is written and in some ways this might go back to the status of Mythbusters on the site.

The way I see it is this, if I want to publish my own original research I need to document the process and how I arrived at my conclusions. That process generally involved providing sources that lay a groundwork for what lead me in a given direction and my own work is how I justify my conclusions. If I follow that process and peer review then my research is published in a high profile location and I say high profile since I can always just write it up in a blog without peer review.

We already have a bit of a peer review system on the site through the up-down votes coupled with the commenting system so someone could be challenged to provide more justification for their answer if it looks like original research.

However, for this site the problem with original research is verifiability. The odds of having enough experts on a given topic on this site that could verify that the research appears to be valid is going to be an issue and a "unpopular" conclusion could be down-voted simply because it goes against the grain.

1
  • You provide a good point against original research, that I didn't think of, but that is a general weakness of the whole page. Just as you need to verify original research, you need to verify references, and while it's generally easier, that can also be a bit of a trouble if you're not expert on the topic. The type of references accepted tend to be fairly weak at times.
    – Wertilq
    Commented Mar 26, 2013 at 15:00
0

peer reviewed sources should be the gold standard of reliability, but that becomes problematic for many questions, because peer reviewed research answer a very limited set of questions. The status of original research should be evaluated on a question by question basis and original research should be discouraged on questions that cite peer reviewed studies or have existing answers with peer reviewed references, but otherwise acceptable.

5
  • So an answer based on original research is okay if it is the first answer, but the moment an answer that cites peer reviewed studies is posted, original research answers are no longer okay?
    – Beofett
    Commented Mar 26, 2013 at 13:32
  • @Beofett: I have been challenged before, in esoteric areas (e.g. McDonald hamburgers rotting) because my sources haven't been peer-reviewed. My response has been "This is the best we can hope for; but if a peer-reviewed article is found, an answer citing it should be voted up over mine." IS that the same?
    – Oddthinking Mod
    Commented Mar 26, 2013 at 14:00
  • (Aside: Peer-reviewed is high on the scale of reliability, but not the gold standard. Properly conducted meta-analyses, for example, would rate higher.)
    – Oddthinking Mod
    Commented Mar 26, 2013 at 14:02
  • @Oddthinking I agree that there is a hierarchy of acceptable references, with peer-reviewed studies being near the top, but by no means the only option. I was looking to clarify Ryathal's answer, since it does seem to imply that the appropriateness of original research is dependent upon whether or not peer reviewed studies have been cited yet. Is voting the proposed means by which such answers should be discouraged or encouraged? Or would the existing system of a comment requesting citations be maintained should peer-reviewed answers exist?
    – Beofett
    Commented Mar 26, 2013 at 14:14
  • To clarify yes original research is ok until someone finds something better, there could be a few specific topics where original research would never be acceptable due to the wealth better data (i.e. cancer). voting and comments would be the vehicle to affect this change deletion should be in exceptional cases.
    – Ryathal
    Commented Mar 26, 2013 at 17:28
-2

My belief on how it should be solved is that it should be different partially topic to topic. Some questions is hard to give proper references to, and a slightly weaker answer should be accepted then. If it was possible to signal this, it would be great. Anyone can edit any post on the page, and if weaker answers could be marked as that, it would be great.

Second is, avoid deleting Questions and Answers with positive score. If the community believes in the question/answer, it should be kept in comments instead to say what you feel is wrong about it. If it's not fixed then it should be closed/removed etc or marked for 'weak answer' or something along those lines.

Maintain the credibility, while open up for original research, and less scientific people.

Not everyone is used to scientific methods, but instead of pushing those aside, make things easier for them. Take from them what can possibly be taken, and improve the answers. SE allows editing of answers, that should be utilized at those times, to improve and put things at a proper level, instead of shooting them down because they doesn't follow the jargon and terminology.

9
  • and who is going to choose whether a particular question needs referenced answers? Based on what objective criteria? You can't have a rule based on subjective choice...
    – Sklivvz Mod
    Commented Mar 27, 2013 at 0:13
  • Yes you can have rules based on subjective choice. I have seen it done more than once before. Usually on games though, but still. This is a community which allows you to vote things up and down. That means everyone have their vote on if it's a proper question/answer. Not every question is one where you give reference to a scientific paper. And first of all you can REQUEST references using comments. Deleting things based on whatever you feel like, is NOT the way to go.
    – Wertilq
    Commented Mar 27, 2013 at 7:58
  • We do request and wait months. The problem is that certain answers are not, and will never, be based on fact due to their speculative nature. Such answers are not fixable.
    – Sklivvz Mod
    Commented Mar 27, 2013 at 8:00
  • See: meta.skeptics.stackexchange.com/questions/2136/…
    – Sklivvz Mod
    Commented Mar 27, 2013 at 8:04
  • But if an answer have a positive score like 7-10, is the accepted answer, why is it then removed, without even commenting on it first? And if something CAN'T be fixed is also entirely subjective. Wasn't it getting away from being subjective, you tried to do.
    – Wertilq
    Commented Mar 27, 2013 at 8:10
  • The score is irrelevant if the answer is not an answer - and answers which do not rely on facts (and will never be able to) are not answers here, per our FAQ. They are flagged as "not an answer" and are acted upon by community and mods. The fact is, this is a community of experts in evaluating facts, not experts in evaluating rational thought, speculation and philosophy. That is why this site is about applied skepticism.
    – Sklivvz Mod
    Commented Mar 27, 2013 at 10:20
  • Yes sure it's about facts, but that doesn't change that "can't be fixed" is subjective. And you said you wanted rules that wasn't subjective at all, while at the moment it's still done in a subjective way.
    – Wertilq
    Commented Mar 27, 2013 at 10:49
  • Questions that can't be fixed are questions that contain exactly no facts and propose a theoretical solution to a practical problem, for example. If you feel any question has been deleted in error, open a question about it. However, outright deletion of unsalvageable answers is very rare.
    – Sklivvz Mod
    Commented Mar 27, 2013 at 10:55
  • meta.skeptics.stackexchange.com/questions/1080/…
    – Sklivvz Mod
    Commented Mar 27, 2013 at 10:59

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .