I want to start with a dark spectre of the future I am worried about - a vision from the Ghost of Christmas Yet to Come.
There is a recent [perfectly innocent] question that asks about a terrible "study". I wanted to answer it, so I looked to see what the response in the literature, from peers and experts. Unfortunately, it was a little too soon for that (a few days later, and that is already changing).
What I found instead was Reddit threads from subreddits that don't have our referencing rules.
On one, people were confidently stating that the study's sample size was too small. And confidently stating that a larger sample size wouldn't help because of confounding factors. And confidently stating that vaccines doesn't reduce infections. And confidently stating that the data was cherry-picked. And confidently stating it was because of "risk compensation" (despite that theory having only limited support.
Worst of all, none of them hit on the Pure Honest Truth (my nickname for my personal unsupported conjecture about the biggest problem with this study.)
Meanwhile, another Reddit thread was substantially people trusting the terrible study.
These discussions were not converging to a single, best answer. These discussions were not converging at all.
In the marketplace of ideas these are the ones of unmerchantable quality.
In my opinion, these lightly-moderated, laissez-faire subreddits are swamps, and the signal:noise ratio is way to low to be useful.
But the good news is you don't have to agree with me. You can enjoy the cut-and-thrust of speculative opinions being argued without substance and without any way of checking them. You exercise the freedom of other users not having expectations about what you write. You can do that by going to Reddit (or any of its dozens of forum competitors) any time you like and join in.
Meanwhile, the unique selling proposition of Skeptics.SE is that we don't tolerate that nonsense. We expect all significant claims to be referenced, we expect questions and answers to be openly edited for clarity and accuracy, and we have little time for opinions and point-scoring against political opponents. As a result, the signal:noise ratio is much higher, and the site is more useful and trustworthy.
So, broadly, I don't think we should change to emulate Reddit. I disagree with that the main site is about sharing "ideas". It is about sharing substantiated facts. I, for one, don't come here to read half-baked ideas and opinions.
But that's the big picture. I think having these boundaries in the sand is important - but there is still plenty of room for discussion in Meta about exactly where these boundaries should be.
In @Fredsbend answer he argues a big distinction between pseudo-answers and reasonable comments is those that include references and those that don't.
I am using a very different definition.
Question comments, unfortunately, are placed higher on the page than even the highest ranked answer. If we get a reader of a question who is interested in an answer, and who then reads a comment and feels that they have their answer and need read no further, then that is a pseudo-answer in a comment.
Such a situation does a disservice to the people who post full answers. It doesn't allow people to correct errors (even their own), fix broken links, downvote bad comments.
It also dissuades good answers - why would I bother going to the effort of posting an answer if a comment already has dozens of upvotes?
Now, I, too, have posted comments that are references to help the next person answer it properly. (e.g. "here is a near miss" reference that does not answer the question, but maybe it can help as a template for someone to find a reference that does.") The idea of giving the next researcher a leg up makes good sense.
But a comment that is basically "The answer is 'No'. Here's a link. I don't have time to write a full answer. Someone else can do it." isn't acceptable. (It is okay not to have time to answer a question. It isn't okay to use that as an excuse to leave a half-baked pseudo-answer instead.)
Nor is a drive-by "[Link to Wikipedia]" comment.
In @Fredsbend's answer, he argues he is fine with half-answers in comments... until a point, when they become a problem that needs to be converted to an answer.
I think there are two things to be noted here.
- We seem to agree on the spectrum and we are arguing for different points on the spectrum.
- Moderators still seem to be involved in policing the line - but it is worse, because the line is vaguer and gets crossed later when then attention is no longer on the question. I think our best-case process - the "happy path" - should not have to involve moderators. We are here to clean up messes; we can't scale to guide every discussion. Can we have a process that doesn't need mods to step in?
@JustSomeOldMan's answer is basically the argument used by a speeder who is pulled over: "Everyone else is speeding. Why are you picking on me?" This isn't a double standard. There are limited moderator resources. You can help by flagging problem comments. They don't appear in any feed, so it is difficult to notice them all. You can help even more by not posting such comments; a mod deleting a comment is a lose-lose to everyone involved.
JustSomeOldMan accuses me (or someone who looks like me) of being stubborn, but his sole argument for being changing approaches is it is better to be free-for-all than only catch some infringers. Perhaps, then, a better description is "consistently and diligently doing their job in an inevitably under-resourced environment".
JustSomeOldMan should be less happy with Fredsbend's plan, because it is inevitably going to be less evenly policed.
So what do we do?
If we had free control over the software, we might take a leaf out of Wikipedia's book - every page has a discussion page where issues can be best thrashed out. (Wikipedia has to use an approach like this because their main pages are consensus based, where we can have competing answers.)
Perhaps the equivalent in the world of StackExchange is a chat session. Go off, away from the casual readers and build on your ideas, and come back and actually post an answer when you are ready to do so? Should we promote using the chat for this more?
- Creating a chat session is a little tricky in the existing software - and impossible for low-rep users.
- We could shove stray pseudo-answers off into a chat (understanding most will go there to die; they are just drive-by comments from people who don't care enough to form an actual cohesive answer).
- Further pseudo-comments in the same comment section would have to be deleted; they can't be pushed to an existing chat session.
- I remain uncomfortable that this will require mod attention for a happy path.
Is this a feasible plan?