5

This is similar to THIS, "Vote early, vote often." I'd like to ask something a bit different.

What, exactly, are the criteria for upvoting questions?

The FAQ doesn't specify. Hovering over voting options with the cursor reveals these tool tips:

  • Upvote: This question shows research effort; it is useful and clear
  • Downvote: This question does not show any research effort; it is unclear or not useful

If you sort by votes, it's difficult, at least to me, to see what criteria are actually being used to upvote. Here's to top ten (as of 2011-06-15) with some comments about their listed sources, usefulness/clarity, apparent level of research, and my own comments:

  • Does torture work well as an interrogation technique?

    • Source: friends say that...
    • Clarity/usefulness: simple question, high usefulness
    • Apparent research: little
    • Comments: very important question with high potential impact
  • Is it true that hot water freezes faster than cold water?

    • Source: none given
    • Clarity/usefulness: simple question, not sure if the answer is useful
    • Apparent research: none
    • Comments: well-known, shown in movies (water on roof above door technique), popular
  • Do cats always land on their feet?

    • Source: none given
    • Clarity/usefulness: simple question, not sure if the answer is useful (except when throwing a cat out of a building from the optimum height)
    • Apparent research: none
    • Comments: popular saying/belief, perhaps has "interesting factoid" appeal?
  • Does more monitor real estate increase productivity?

    • Source: very reputable programmers
    • Clarity/usefulness: very high
    • Apparent research: quite a bit
    • Comments: This struck me as perhaps the highest quality question asked, could be quite useful depending on one's job/typical projects, and was very well stated/formulated with linked references and emphasized important points.
  • How much energy does displaying a webpage with a black background actually save?

    • Source: Blackle.com
    • Clarity/usefulness: simple question, moderate usefulness (depending on the power savings, if any provided in the answer)
    • Apparent research: some
    • Comments: the question definitely has the "interesting factoid" appeal; the answer(s) revealed that if you use an LCD and don't want to use an extra 1.5W, you should just stick to white-backgrounded google
  • Is popping your knuckles bad for you?

    • Source: friends say that...
    • Clarity/usefulness: simple questions, could have profound usefulness if the answer was that knuckle cracking leads to health issues
    • Apparent research: little
    • Comments: the answer ended up being that negative effects are negligible, but it definitely would have been useful to know if this were not the case! This is also one of those "everyone knows that" questions, so it's helpful to have the answer referenced.
  • Is there any verifiable historical and/or scientific evidence that Jesus lived?

    • Source: none given
    • Clarity/usefulness: clear, concise, useful
    • Apparent research: some
    • Comments: I know that in the realm of apologetics, there are definite sources who say that Jesus might never have existed, so perhaps it's good to have an answer here.
  • Security in open source vs. closed source software

    • Source: teacher
    • Clarity/usefulness: very high
    • Apparent research: little
    • Comments: This could have definite implications on software application choices, at least if one factors security into such decisions. I somewhat doubt the average user does -- most probably just assume it's safe and that developers are always working on fixing holes.
  • Will a mother bird abandon her young if touched by a human?

    • Source: friends say that...
    • Clarity/usefulness: simple question, not sure how useful an answer would be
    • Apparent research: little
    • Comments: the answer ended up being "No" -- this is useful in that one can now touch birds at will
  • Has man walked on the Moon?

    • Source: none given
    • Clarity/usefulness: simple question, useful for curiosity's sake
    • Apparent research: none
    • Comments: age-old conspiracy theory/contraversy, popular question

I'm not sure that it's readily apparent to me why these questions were so highly upvoted. In looking them over, it seems that some possible factors might include:

  • Is it in the "everyone knows that..." category?
  • Is it a well-known conspiracy theory?
  • Is it in the "neat tidbit" category?
  • Does it already have high upvotes or many answers (> traffic = > votes)?
  • Is it worded cleverly/wittily?

For that last question, I just recently searched for "cholesterol" and found these two questions as the top hits:

My interpretation of the second question was "is cholesterol bad for me?", since "eating eggs" was translated into "1 egg contains 70% of my daily cholesterol recommendation" + "I frequently eat 2-3 eggs per day." In other words, the eggs scenario is somewhat of a subset of the first question (if one allows that "is bad for me" is fulfilled by "significant factor in heart attacks"). I personally found the first question more useful in terms of it's phrasing/specificity, and the potential answer it might provide... but the second had much higher numbers for both the question and answer upvotes.

In some recent questions, I've seen downvotes, or at least comments requesting sources if none are given. I don't see these requests in the questions above, even though sources for the claims are not given. This may be fine, due to the meta discussion, "My friend says...," which points out that the circumstances may dictate. In any case, "my teacher says" can't be that different from "an old professor of medicine," but one was quite hounded for sources (including by me), and the software one was not.


I know that was somewhat long; much of it was spent on summarizing some top voted questions to examine. I just thought I'd do the legwork of giving this snapshot in order to ask about the community's thoughts on upvoting/downvoting.

  • Is upvoting clearly defined for everyone?
  • What are your takes on the current/practiced (not intended) reasons for upvoting?
  • Does anyone think that criteria should be made clearer, perhaps in the tooltip or in the FAQ?
  • Similarly, is it functioning as it should already?

I personally appreciate interesting claims which show the asker exercised critical thinking and was prompted to look for more data. Whether it's obscure or common knowledge doesn't impact me that much; if it's well worded and strikes me as well-applied skepticism, I upvote. I'm interested in other common criteria used by those here.

4

I'm only acting partially as intended, I fear.

I judge, whether I find the question interesting. And is it in an answerable state (clear).

If the headline gets my interest, but the body of the message distracts me, because it is taking a different way than I thougt, I still might upvote it, if I see good research at work, or citings.

Questions, where the headline doesn't wake my interest (cholesterin? who cares!) don't get visited by me, so no upvote.

Questions which pick my interest, but are too poorly made, get downvoted by me.

Often I see poorly produced questions which have already many upvotes - but I don't downvote them, if they are highly above the 10 upvote level. My downvote will not stop it, so I spare the -2 for questions where I'm early.

  • 2
    I tend to be a hyper up voter. :o I enjoy giving people reputation points and I find anyone asking a QUESTION to be doing something of value versus just accepting any old thing they heard! – Larian LeQuella Jun 16 '11 at 3:38
  • 3
    My experience is, that 90% of the questions is just rubbish. Debunked long ago, boring myths, repeated over and over again, and silly prejudices, where I often have the feeling, the asker is very aware of the answers, and is just collecting the points. The questions are totally vague (do black people jump higher than white? Who is a black person? Who is white? It's racism and rubbish. Do we have to explain racism here? – user unknown Jun 16 '11 at 13:46
  • +1 pretty exactly what i wrote. But 90% seems a bit high. Is there any way to turn off the rep gaining if a question gets closed? Now, imo you can still earn further rep, even if the question was closed. This is a serious problem here, as newbies can best gain rep by asking high voted qestions. Thats self-affirmating group process hunting more for attention rather than critical questions. On other SE sites, qestion are due to topic anyway mostly pretty specific and ok. But YOU should really downvote qestions despite they have many upvotes to draw back attention – Werner Schmitt Jun 16 '11 at 14:43
  • Well - a closed question produces rep, as long as it isn't deleted, no? I'm not sure on the procedure. Closing stops new answers at least. But habits will grow with the number of users, which earned upvotes with their behaviour. But I'm not so sure, what the intention of Skeptics.SE is. I often have the impression, it's a search service for scientific papers. You ask 'Is the earth flat? My mum told me!' and get 3 papers back which prove your mom wrong. Hm. If you can explain it with everyday observations, you get downvoted. Think for yourself: unwanted. – user unknown Jun 16 '11 at 20:02
  • 1
    Full ACK. The intention/scope seems much too wide and the avg. user knowledge too low for it. The philsophy of the SE System is to overcome this issues by the rating system. But currently the user number seems to be to low to make this work reliably. Downvoting doesnt work really, questions highly speculative too answer but popular by votes dont get closed. Many-links answers get per se more votes than reasonable explanations. No one really checks the sources of the links or how old they are. Googling gets synonym to verificating. Searchin for falsificating sources is much harder... – Werner Schmitt Jun 17 '11 at 1:59
4

The major determining factor, in my experience, are mass appeal and high quality answers. If the answers is interesting, then the question will ride the answer's coattails.

Does hot water freeze faster than cold water? is perhaps the poster child of that phenomenon, by having 95 upvotes yet being an incredibly poor question. In this case, it's the answer that makes the page worthwhile. My reaction was the same as Kit Sundle:

Consider my mind blown. I very nearly voted down the question thinking it's just plain absurd.

As for sources, they do play a role, but only in downvoting and closure.

If a claim is improbable, and cannot be attributed to source of relative importance, then the question will either get heavily downvoted or closed. It's our policy and for a reason: we cannot, and should not, waste our time refuting every drunken man' theory. But that does not mean we require a source for every claim. "Everyone knows that you get a cold from being cold." is a common statement, at least in North America. Requiring a source would be pedantry, in this case. We'd alienate new users more than anything.

  • 1
    water freeze likely would have been closed immediately on physics.SE. But as a physicist myself i was not really surprised (anomaly of water is also very unintuitive). But you are right, the occasional "surprising factor" is IMO one of the unique selling propositions of skpetics answers. Otherwise alot of generalizations on gender, behaviour, race surprise me negatively and i dont understand why not really specified question dont get closed. This is not psychology.SE and never will. Obviously psychology is a higly popular topic, but also that most (hobby)programmer/nerds know much about it – Werner Schmitt Jun 17 '11 at 1:59
  • 1
    I'm the author of the "freezing water" question and I'm... a bit skeptic about your definition of "incredibly poor question". It's indeed a question positing a really improbable prospective, with no source whatsoever other than the author (me), yet just this straight to the claim approach, that made it seem outrageous, is what brought it attention and traffic. And sites need traffic. Especially young ones, that attract fewer visitors than many of its other Stackexchange peers. – luvieere Jun 17 '11 at 5:47
  • @luv: The problem with the question is that it has absolutely no context. Zero. We need a bit of context to distinguish good questions from time wasters. I wouldn't bother refuting "Is there a pink tiger on Mars?" unless it had serious proponents. The same goes for your question. – Borror0 Jun 17 '11 at 21:37
  • Questions with no context demand answers that supply different contexts with their different particularities. In the case of my question, the situations when water behaves like I claim it does and the situations where it does not. I believe that a passionate skeptic is a curious fellow, who wouldn't mind doing some research of his own, not relying on the OP to have done this. Absurd claims would quickly turn out no results upon trying to research them, so there is no real danger here of wasting vast amounts of time for nothing. – luvieere Jun 18 '11 at 4:44
  • On top of that, what's the difference if I claim some absurd theory and ask about it here, or I find someone else on the internet that makes some absurd statements and I reference my question with a copious amount of links? The nature of the claim is the same: bullsit will remain bullsit, no matter how referenced it is, good, interesting facts with curious outcomes will still be interesting to research and discuss, even when not referenced. The rest, is just the digital equivalent of bureaucracy. – luvieere Jun 18 '11 at 4:48
  • @luvieere: If that's how you feel, express so in an answer on this question: meta.skeptics.stackexchange.com/questions/806 You'll notice that the move upvoted question disagree with you. – Borror0 Jun 18 '11 at 5:27
  • @Borror0 What do you mean by "the move upvoted question"? The most upvoted answer perhaps? If this is the case, well, Jeff's answer specifically bashes marginally on-topic questions, not un-referenced ones. There's even a comment that underlines this omission. Either case, thanks for the pointer, I will leave an answer to it as well. – luvieere Jun 18 '11 at 6:06
2

I think that many the questions are silly; and I tend to vote on answers more than questions (80% of my votes are for answers).

I don't see a way to list which half-dozen or so questions I felt strongly enough to vote on (in about the last month), so I can't analyse that list for what they have in common.

The question I up-voted most recently was "Is DRM effective?":

  • Not a ridiculous idea: i.e. DRM is, so far as I know, actually being implemented by serious companies
  • Controversial: I've often read, in the (new) media, people alleging that it doesn't work
  • Answerable: it seems like an objective answer might be possible (it isn't nothing but a matter of subjective opinion)

The content of the question text is fairly brief, and unobjectionable in every way.

It was more the question itself than the way it was asked, the title rather than the text, that got my up-vote.

When I say "I think that many the questions are silly", I probably won't up-vote any question that I'm pretty sure I know the answer to already, or whose answer is of little consequence to anyone, or whose topic is unreasonably subjective or unclear.

  • 2
    Interesting -- I'm interested if the majority follows your methodology as well, particularly "I probably won't up-vote any question that I'm pretty sure I know the answer to already." That struck me as possibly unique. – Hendy Jun 16 '11 at 13:38
  • +1 last question i voted too and what this site should be about imo. As the email spam question its mainly solvable by logical reasoning. Data can support the answer, but i dont feel its necessary, as the company decision to implement DRM is also mainly based on reasoning to controll the market, building monopoly. But what to do with all the other questions. Just dont vote up questions of no personal interest is imo not a good solution considering the SE system. – Werner Schmitt Jun 16 '11 at 13:43
  • @Hendy - If I already know the answer then how could I find it a useful question? – ChrisW Jun 17 '11 at 1:50
1

Thx for the overview, interesting to study. Believe me, i really understand your qestion. But the whole point of the stackexchange system compared to discussion boards (there you also have a 5 star thread rating) is facilitation of moderator work by user rating (privileges gaining and carrying responsibility for evolution of site) of most POPULAR topics/question. Its not aimed on voting up the "best asked" qestions. In fact its absolutely democratic, every one has one vote/voice. Whereas in a scientific community, the single peer-reviewer decides on his own, which paper/question will be publishded and placed on site #1 of the journal. That difference is crucially to understand. Popularity goes before quality (which doesnt mean quality has to be low!)

The SE system works quite well with other topics. The main problem IMO here is you get upvote points despite the question is closed afterwards. So people will use buzzwords and formulate questions as much as possible in a general way so that average joe can relate to it. On stackoverflow this wouldnt work as people come there to get specific problems of their specific known programming language solved. Then of course non-linear effects will add (highly upvoted questions get upvoted even more - avalanche effect)

Dont try to analyze or force single user rule of thumbs of voting, they have to adapt to community and type of questions by time and learning. Obviously this rating system isnt so perfect as it was for original stackoverflow, as this is more science related and stackoverflow about practical problems. Peer-Review on Nature/Science has some big advantages against Arxiv. It not fun to browse arxiv titles. Look this

A FAQ "reglementing" voting would misplace the idea of defining "what is a popular question" by the avg. user here and thus, limit the further growth! Thats the backside. If you can get very good sophisticated experienced mods for a site, the choice should be clear. But look at AREA51, the idea is to outsource as much as possible to the user by a clever rating system. On physics/stackoverflow.stackexchange it works quite well due to avg. user level and topic (its easy to decide if questions makes sense), here a lot of inexact questions/hoaxes get voted up.

Its interesting to watch how this works out, still pretty young community here, probably pretty hobby programmer heavy. But probably i will leave after some time, as its gets boring and more & more answers seem to be clones of former scurrilous reputation hunting ones. Its sometimes here like having a small child trying to jar upon fathers nerve with unanswerable questions. Its worse than a good & clever entertaining troll/keyboard-warrior. But maybe much more kids are postin here than on an average adult board with adult discussion & language. I think there is more kindergarten here than on other stackexchange sites, so voting rules, etc will not have huge effect as kids are used to hunting gaming/reputation points. I dont care alot, too old for this virtual karma stuff.

Due to the system here IMO the only working thing to force "better asked" questions getting more upvotes is turning off reputation gathering for closed questions (as privileged experienced user will define this). But afaik this doesnt work cause of the SE philosophy. But i also saw some over 10 upvoted questions with only 1 answer with 1-2 votes. So many seem to read & vote questions, but not the answers :)

  • I don't think this answers the question: "What questions should receive up-votes?" – ChrisW Jun 16 '11 at 13:12
  • @ChrisW: Well, I think it offers at least one answer to that question: whatever any particular person wants to upvote should receive up-votes. Perhaps tautological, but my take away from this was that @Werner offered a take on what makes SE unique and better than alternatives, and one of those things was just letting the questions receive the votes they get for whatever reason (perhaps except closed questions). I may be misunderstanding him, though! – Hendy Jun 16 '11 at 13:36
  • @chrisW i know my answers often dont look like a direct simple one to the question and draw a bigger picture. But esp. in meta i will not concentrate on this form function, but give my input/view in a discussion (which is imo more the function of the META concept in SE). You can analyse a scientific topic completely by asking very few good questions or lots of bad too simple ones dont giving you really a understanding of whats happening. A famous quote i like goes "you dont will see a clever mind giving many good answers but asking good questions" Thats my definition of a skeptical view – Werner Schmitt Jun 16 '11 at 13:53
  • @Werner - Well that's fair enough (if you know what I mean: I mean, that your comment is OK by me). Still I thought that this was a good question, and I'd kind of like to see a good (clear, semi-objective) answer or set of guidelines. – ChrisW Jun 16 '11 at 13:57
  • @Hendy you got it! But as i wrote in Pros & Cons thread, this has some downsides, which imo the experienced users here/mods have to manage to keep the quality high. But animating newbies here to rule by given criterions is imo pointless dead end, they have to learn what good questions and answers are like learing a programming language – Werner Schmitt Jun 16 '11 at 13:58
  • It's one thing to learn a programming language by reading the programmer's instruction manual: it's another to experience your programs being up-voted or down-voted by the computer. – ChrisW Jun 16 '11 at 14:18
1

The highest voted questions are not a good example to discuss how to vote, the voting for these questions is dominated by users coming from the Supercollider (the list of hot questions in the top left). Almost all of those questions were featured on that list for a significant amount of time and thus got a lot of voting.

Whether a question got 10 upvotes or 100 doesn't tell you much about the quality of the question, only about the popularity and the amount of traffic (from users that can vote) the question got. But the difference between a question that has zero or a negative amount of votes and one that has 5-10 upvotes is quite useful to filter out potentially interesting questions.

Your question is mainly about upvotes, but I think it is easier to talk about reasons to downvote. If I see no reason to downvote, and I think the topic is mildly interesting that is good enough for an upvote for me.

Reasons for upvoting for me are

  • The subject is interesting
  • The question is well-written

Reasons for not voting or downvoting a question are

  • The subject is trivial and not interesting
  • The question is not detailed enough
  • It shows a lack of effort in writing it

Generally, if I'm interested in the answer to the question or if I know the answer and consider the topic interesting I'll upvote the question, if it has no other serious issues. If the question is really well-written, but in an area that does not interest me personally, I also might upvote it to reward the effort.

If a user has performed some initial research into the topic and writes about that I consider it a plus, but I don't require that for upvoting a question.

The most important reason to downvote for me is a lack of effort in writing the question on behalf of the author. I don't expect users to perform significant research into possible answers before asking here, but I do expect them to make some effort in asking the question. That means I expect a summary of the claim, if I have to go to another site just to find out what the question is really about, that's a non-vote or downvote from me. If the question is formally on-topic, but just really uninteresting I also might downvote.

There are a lot of other reasons to downvote, but the very bad questions usually fulfill one of the close-criteria. I personally don't downvote here if I also vote to close, but as my votes are binding this is somewhat different from a normal user.

  • 1
    as far as i understand the SE system the downvotes should signal the high-rep guys a question is problematic/speculative. What im missing in your downvote list is "out of scope". There are currently alot of hoax questions with a "is there a study" in it, trying to legitimate a pointless question with this term. I know you have to make this site grow. The broken window theory seems esp. on skeptics.SE quite crucial IMO. Or the quality will nonlinear decrease with growth – Werner Schmitt Jun 17 '11 at 1:58
  • @Werner "Out of scope" questions should get closed, so I personally don't dowmvote them as my closing vote is binding so I can just directly close them. If you're bothered by certain questions I suggest to compile a list of examples and open a new meta post, discussing without examples does not lead to much. – Mad Scientist Jun 17 '11 at 5:43
0

The tooltip for the up-vote arrow says,

This question shows research effort; it is useful and clear.

The tooltip for the down-vote is,

This question does not show any research effort; it is unclear or not useful.

So, those are the desired criteria, presumably:

  • Research effort: you done some research and found (I'm not sure what: that many people believe it; that the answer is difficult to determine; that it's definitely a pseudo-scientific falsehood; ...?)

  • Useful: you reckon that that the question is useful because (I'm not sure why: you want to know the answer yourself; you want other people to know the answer; the answer affects some genuinely important decisions; ...?)

  • Clear: you reckon that that the question is clear (I'm not sure in what way: I'd use 'clear' to mean 'not subjective', not the kind of question to which I'd want to reply with another question).

  • my point is, you dont learn asking good questions and giving good answers by tooltips, but only by practice of answering and asking. Often as you wrote people will not downvote if a qestion already got upvoted a lot. These nonlinearities cannot be managed by clever tooltips, FAQs, manuals. Yes, many questions here are very unuseful and show no research effort, as the information given by answer has no practical value and topic is too complex to introduce yourself (See e.g. "do women feel more emotion") YOU have to downvote the question according to the SE system, there is not really a side way – Werner Schmitt Jun 16 '11 at 14:31
  • Well, I listed the same quotes in my question, and the entire point of listing the top 10 most upvoted questions is that these definitely aren't the criteria being used. So... my question is 1) should there be better information about upvote criteria to help steer upvotes where they are "supposed" to go, or 2) is the current consensus on what votes actually mean (as observed) a better path to follow? – Hendy Jun 16 '11 at 14:35
  • @Werner The question which I'm trying to answer, first and foremost, is, "What are the criteria for voting, i.e. what standards do you use to assess the value/worth/goodness of a question, and are those criteria clear?" I'm not trying to answer, "Do other people follow the criteria when they ask questions and/or when they vote on question, and if not what could we do to change that?" (And it was someone else, not me, who said they don't bother to down-vote any already-highly-up-voted question). – ChrisW Jun 16 '11 at 14:39
  • @Hendy - I've tried to demonstrate that I find the tool-tips unclear: especially about what kind of "research effort" is expected. The FAQ somewhat begs the question too, by using "pseudoscience" in the first sentence (as if the topics you're expected to ask about are not just topics whose evidence you're unsure of but actually mostly assertions which are false/wrong/ridiculous). Also the "practical, answerable questions based on actual problems that you face" boilerplate may not be appropriate for this site. IMO that main FAQ page should be rewritten or copy-edited. – ChrisW Jun 16 '11 at 14:52
  • @chrisW but your bigger aim IS to get better questions?!Hendy question is what answers SHOULD receive vote, so im ontopic here.Thats why imo your question is quite hypothetical as you cant bypass the SE rating system. You try to find common criterion standards, but this will not happen, as there is much more different user level than on other SE sites, from unexperienced ignorant - genius. Its like asking what criterions should one choose voting for a party, its pointless, thats not how democracy will ever work. You are imo thinking in the end of a aristocratic System, which SE never will be. – Werner Schmitt Jun 16 '11 at 15:03
  • @Werner - There are different user levels on other forums too (e.g. the Bicycles forum has experts and novices). It may be that some people don't know what the rules are, or don't obey them even if they do know; but I'm especially focussed on whether there are rules, on whether I know what they are, and on whether they're clear to me. – ChrisW Jun 16 '11 at 15:17
  • @chris We can't adjust that part of the FAQ (yet), we can only change the first part of it. There is an FAQ overhaul in the works, so that might change in the future. – Mad Scientist Jun 16 '11 at 15:53
  • @Fabian - And the tool-tips on the voting arrows are apparently, also, the same as on other forums and not specific to this forum. – ChrisW Jun 18 '11 at 1:03
0

Perhaps every question that doesn't receive a down-vote should receive an up-vote (or vice versa): questions should get more votes.

  • And, I've just either up-voted or down-voted (with a corresponding comment) every question on the main page. – ChrisW Jun 17 '11 at 4:11
  • Maybe I'll up-or-down-vote every question for a while, with a corresponding comment, from now on. – ChrisW Jun 17 '11 at 4:17
0

A question that's implausible but true.

Someone's +1 to this question implies a +1 to a question that's implausible although true-ish.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .