I'm so torn over this issue.
Firstly, you are right.
You are absolutely right that science doesn't pivot around a single study. The meaning of confidence limits and statistical significance means we should expect incorrect results to be published occasionally, and that is only one possible source of error.
Where there is more than one study available, we should list those too. If there are contradictory studies, they should get some visibility too. Well-run meta-analysis should get an even higher priority.
We want our answers to be definitive, and this is how you get definitive answers.
So, yes, so far, I am right behind you.
On the other hand, this raises the bar very, very high. If we make every answer require a full literature review, we are going to quickly run out of people willing to submit an answer.
This site is frequently criticised for demanding too high a standard. I'm happy to defend against that view. By rejecting many of the weak non-answers that are tolerated elsewhere, the quality of the answers we do get is exceptionally good. However, if we raise the bar so high that only scientists actually working in the field can respond, then perhaps we would be better off discarding the Stack Exchange software and becoming some form of a peer-reviewed journal.
(Just pausing a moment, to let myself daydream about that.)
There's also a spectrum of how much evidence we should require, depending on (a) how extraordinary the claim is, and (b) how sure we really need to be.
For example: I would never swallow a cancer-curing drug that had been only tested with the skills of Jamie Hyneman and Adam Savage - their tests are far from rigorous, the idea that a single drug could cure all cancer is outside of our current understanding of the many and varied diseases that fall under that umbrella, and the downsides are potentially fatal.
On the other hand, if Mythbusters says vodka can remove foot odour, then... well, okay, sure, whatever. If I ever becomes an issue, I can always give it a go and see what happens (although antibacterial soap would be a preferred choice). I can provisionally accept that hypothesis until further evidence is provided.
So, I remain torn: I want the definitiveness, but I am not sure we can afford it.
I suspect the compromise answer is to rely on the comments and voting system to ensure that answers that invoke a single study to support extraordinary claims are challenged and better answers are voted up.
I'm going to try to keep more vigilant about this.