This details my first and second impressions as both a lurker and asker. These are obviously opinions, and they are pretty harsh, so be warned. (Also, I didn't really proofread this because I had to bug out immediately after finishing. If you see a typo, go ahead and fix it.)
"This site is crap." I kept seeing junk appearing on the super-collider asking things about Hitler and the Moon Landings. I came over from EL&U and their chats had nothing nice to say about the questions or answers on this site. Not being satisfied with hearsay, I browsed around through the top questions, looked at the active questions and read comments, answers, meta, chat.
The big problem I immediately saw is that a large portion of the questions were about urban legends, current events, religion and conspiracy theories. While those are "interesting", they don't have anything to do with Skepticism as written in the FAQ:
Skeptics - Stack Exchange is for skeptics, rationalists, free thinkers, or anyone who questions pseudoscience. Skeptics is about applying skepticism — it's for researching the evidence behind the claims you hear or read. It is not for philosophical discussions about skepticism itself.
As much as debunking urban legends is fun and, admittedly, is related to Skepticism, the questions feel like a reputation farm. Step 1: Browse Snopes. Step 2: Ask question. Step 3: Quote Snopes.
In addition, questions aren't generally formed in a way that can really be answered or are overly... flashy. "Is this really what happened?" "Can XYZ be proven?" And I've already weighed in on the excessive amount of unnecessary images scattered over the site.
But the silver lining was there. The list of all-time top-voted questions has some awesome questions and answers. I learned some cool shit. Over a large enough section of questions and answers, the general community response is solid. People do recognize a good question when they see it and are more than happy to upvote them.
In addition, answers with zero references are handled very well. The moderators seem to be on top of guiding the quality of answers. My first impressions could be boiled down into this generalization: Questions are crap; answers are awesome.
To compare to EL&U, the questions and answers have a different quality scale. The answers are significantly less references and the fuzzy nature of language makes this okay. The questions are generally more on-topic and focused due to the easy to qualify topic of English Language and Usage. The on-topic debates there are valid debates about subject inclusion.
Here, nothing seems to be explicitly off-topic except for religious questions in the form of, "Is God real?"
So, I saw enough interesting material here to start asking questions. I wasn't comfortable answering for a while but gauging the responses to my own questions seemed like a good way to understand this community better. The result: This site isn't actually interested in Skepticism.
This period is when I started challenging other questions' on-topicness or formats. I challenged the image thing; I added comments asking for details on questions. And I asked questions. I was mostly satisfied with the answers I got and the topics I chose were intended to gauge the deviations of interest. Things worked out dandy.
But as for the Skepticism... I don't get the feeling that anyone cares. People are more interested in growing reputation or finding a cute image to upvote. Comments contain jabs at people who hold wrong beliefs or one-liners that get severely upvoted. Blatant bias is overlooked for the sake of an agreeable conclusion. Questions don't form a real question that can be definitively answered.
None of these things are an inherent problem with this site; these things are just how humanity works. But I thought this site was pandering to Skeptics, not cynics. Instead of attracting experts on Skepticism, it feels like this site is attracting angsty teenagers, rep farmers and people with axes to grind. The feeling I get from questions is that someone is wrong, dammit, and we are here to prove it. But that isn't Skepticism. And it isn't a friendly site were people can come to get answers to tough questions or challenge the assumptions that society shoves at us.
The big problem I see from this "second impression" phase is that Skeptics.SE is an exclusionary community. If you don't agree with "us" you are dumb and you are wrong. The status quo is god and the hivemind upvotes cheap-shots at people outside the club. When a troll rears its head, the community responds in a feeding frenzy for the biggest smackdown which is rewarded with heaps of reputation.
Can I point to examples of these things? Eh, probably. I didn't keep links to the sources of these feelings because I wasn't expecting to write this all out. I came to Meta today to ask for details about what is or is not on-topic. I now have the ability to close-vote and was surprised to discover I was the first one to vote on most of the questions I thought were obviously inappropriate for this site. But really, all of what I just described is a feeling. They are opinions birthed from my usage of the site. They are very likely biased by my desires for this site. Since opinions were asked, here are mine. Even if you disagree with it or can essentially say, "Yeah, that is a very skewed perspective", hopefully you can use this information somehow.
What are the actual problems?
Aside from the peeving above, here is a list of what I consider to be very large roadblocks on Skeptics.SE. Again, these are opinions.
Unprofessionalism. The images-in-questions thing is a perfect example of what I consider to be a barrier of entry for serious Skeptics. Pictures of Spock or Harrison Ford work on kids or people who are entering from the blogsphere where Oatmeal and xkcd are kings. Instead of trying to flash puppies and kitties at people, we should be attracting visitors with damn good content.
Fuzzy terms. This is slightly related to the previous bullet-point, but most questions I see while browsing don't present a testable claim or question. Questions with fuzzy terms will get fuzzy answers unless the term is specific jargon for a specific field. Even if it is a specific term, most people reading the question and answers will know not that and could do with an explanation. Somewhere in here is an annoying subjective line between asking a question in laymen's terms and asking a question such that a relevant expert can provide a straight up answer.
Angst. This is mostly related to religious topics. The site guidelines and the moderator response seems to be excellent in this regards but the comments and general community could use a big dose of tact. Being right doesn't matter if it is off-topic. Clean it up and do better than your average internet debate forum. This is again related to acting professional.
On-topicness. This site will always have trouble with what is or is not on-topic but I don't think the grab bag style is going to work in the long run. If a question is about math, send it over to Math.SE. If the question is about the English language, send it over there. The same goes for Physics, Biology, Cooking, Cycling. Don't try to syphon off topics from other SE sites. But even more than this, don't accept questions just because there isn't an SE for it yet. If an SE did exist and you would migrate it, the question should probably be off-topic. This leaves the question about what is on-topic and I think the answer to that should be Skepticism. You can have questions about all of the above topics that would be on-topic for Skeptics. The point isn't to send everything over to the other SE sites. The point is to ask, "Why does this belong here and not there?"
Closing. Related to the previous point, Skeptics needs to start closing bad questions. Not just questions related to being off-topic; you need to start getting pickier about how questions are asked. The point to closing shouldn't be to get them out of the way but to encourage editing such that the question is more relevant to the site. Answering questions on Skeptics is tougher than on other SE sites; poorly formed questions are making that even more difficult. Part of the solution to this may be to encourage significantly more community editing than is typical (but that is just a guess/theory.)
The super-collider. The top SE questions list will probably pull in a lot of visitors curious about Hilter, sex, the moon landing, Michael Jackson. That is because these are globally interesting topics. Unfortunately, "globally interesting" doesn't mean the same as "interesting Skeptics question". SE as a whole seems to have this problem but I suspect Skeptics will have it tougher because of the loose topicality. Phrased correctly, you can ask about any subject here. If the subject (and timing) is right, it will pull in the upvoters from the super-collider. If you let these people define your community, you will not have a community about Skepticism. You will have a community driven by catchy phrases, punny titles and pretty pictures.
All of the above is conjecture and opinion. I have no great study to back it up. I cannot reference anything to help me. Take it as food for thought and a quick challenge. If you consider my points irrelevant, great, at least you thought about it. If you actually want a site full of pretty pictures, okay. Your call.
As for me, I am here for a bit. I will keep fighting for the valuable site I can see under all of the crap. I don't think Skeptics is really much worse off than, say, EL&U where I see much of the same issues. But I think Skeptics has a very tough battle to justify itself against the SE model. Why are you here? What is this site supposed to be doing? The answer shouldn't be "generating traffic" or "Snopes.SE." You can be more than that. It just takes a hell of a lot of work.
But if no one else wants that, let me know. I can go work on other SE sites. I like the idea of a Skeptics.SE but have little interest in what I see as a low-quality mirage of what Skepticism should be. Beta is tough; beta is especially difficult because you have to resolve all of the conflicting goals for the site.
In any case, there are my thoughts. All discussions welcome; feel free to hit me up in chat or the comments with questions/complaints/thoughts.