As a thought, there seems to be some complication from claims about toy models meant only to highlight a single point.
For example, the primary claim asked about in this question is:
Drug regulators’ acceptance of any statistically significant improvement shown in a single randomized trial and lofty drug prices has created a situation where it is now, hypothetically, profitable for a company to run a clinical trials portfolio of chemically inert compounds. While the current cancer drug pipeline is certainly superior to inert drugs, we must rethink market incentives to encourage transformational drug development.
– "Low-value approvals and high prices might incentivize ineffective drug development" (2018)
In short, the authors are pointing out that anti-cancer drugs are frequently costly for benefits that may be questionable both in extent and statistical reliability – a situation that the market allows because people who're dying tend to be desperate for any solution, even costly, unpleasant solutions with a low chance of success. The authors seem to be of the opinion that this is an undesirable phenomena, and they argue that it can be somewhat reduced by requiring further testing for regulatory approval.
As the authors point out in their comment's only figure,
,
requiring another p<=0.05 test would be expected to reject 95% of false positives that would've otherwise received approval, increasing the break-even cost of an approved product 1/(1-0.95)=20 times from 440-million USD to 8.8-billion USD, assuming various simplifications that they lay out in their "thought experiment".
So, given that they're basically just pointing out a conceptual framework, do we fact-check them as though they were making a real-world claim? And if we do, then what do we fact check? I mean, the paper does reference real-world figures, but only as motivation; it doesn't actually rely on any of those real-world figures to be correct to make its argument – their "Fig. 1" pictured above was meant to highlight the results that they would obtain for different data, as the authors explicitly point out that they're using average/median estimates, basically to show some hypothetical results in their toy model, rather than comment on real drugs.
Then more generally beyond that particular question, how do we address abstract mechanistic claims in general?
I mean, all models are wrong, including the most fundamental laws of physics known today. So merely pointing out that a model isn't perfectly correct is a meaningless exercise; of course all economic models that don't reduce to pure logic are wrong! (And, under incompleteness, we can even say that purely logical claims can be wrong – but that's way off-topic.)
In practice, what we'd do isn't qualify a model as wrong/right, but rather discuss its practicality and merits/flaws. So, would an appropriate tact be to respond to an economics claim by launching into a discussion of economic theory? And if so, what would be the target endpoint: would we be trying to qualify the various merits of an economic idea, or what?
Finally, SE.Skeptics still doesn't have TeX enabled. Above, I managed
increasing the break-even cost of an approved product 1/(1-0.95)=20 times from 440-million USD to 8.8-billion USD
, but beyond such utterly trivial content, it's simply not practical to format math or data here.
Conclusion
I'm not really sure how to address hypothetical economic models as though they were factual claims about the real world, even when they're motivated by real-world scenarios and used as mechanistic explanations for real-world policy advocacy.
That said, I certainly wouldn't object SE.Skeptics expanding its scope to include critical analyses of theoretical frameworks (that's awesome fun!), though it'd be even more important to enable TeX if we'd like to go in that direction.
