On a recent answer, which was quite poor, I observed the following statistics:
My comment stating that the answer is bad and I am downvoting it because of $reasons: 19 upvotes
Answer itself: 8 downvotes, 7 if you subtract mine
Only slightly over 1/3  of people who obviously agreed that the answer needed downvoting, actually went on to vote the answer down.
 in all fairness, this assumes that all people upvoted my comment agreed with my assessment of "-1", whereas some people may have liked my critique/reasoning but not the "-1" conclusion I arrived at based on it.
Presumably, the -1 reputation cost is one of the reasons for this glaring discrepancy, though I don't know how to test if it's the main/only reason.
Downvoting bad answers is one of the most important mechanics of what makes SE sites in general, and especially Skeptics, produce good quality content.