3

I was reading up on some of the chronic medicine that I take daily and according to the Cochrane review at least one of the medicines is not effective for what it is supposed to be treating. The other one I am using is listed as being effective, so it could be that I am taking one that is ineffective with potentially damaging side effects for nothing.

My question is this: Is the Cochrane method a reliable method of determining the efficacy of treatments or are there instances when it can be wrong?

  • 1
    Please consult with your doctor, or get a second opinion before changing medications based on your own searching. – Oddthinking May 11 '15 at 15:07
  • Thanks for your concern. Of course that is important. – Tjaart May 11 '15 at 22:00
1
  • Cochrane formed an official relationship in January 2011 with the World Health Organization (WHO) as a partner non-governmental organization with a seat on the World Health Assembly to provide input into WHO resolutions. The collaboration is active in providing evidence for good practice during disaster relief and humanitarian crisis through a partnership with Evidence Aid. [src][src]

  • Cochrane reviews are peer reviewed:

    Cochrane Reviews are peer reviewed and dynamic; we update them regularly to incorporate new research. This ensures that you can base treatment decisions on the most up-to-date and reliable evidence.

  • Cochrane reviews are authoritative, relevant, and reliable evidence and they're published in http://www.cochranelibrary.com/.
  • For 20 years, Cochrane reviews has produced systematic reviews of primary research in human health care and health policy, and are internationally recognized as the highest standard in evidence-based health care resources.
  • The most popular Cochrane evidence can be found here:

    enter image description here

    If you go for example to "Screening for breast cancer with mammography", you will see an AM Score 245 which measures article level metrics. As you notice, it is the top 5% of all articles scored by Altmetric. As you see, their evidence has had the highest impact in the scientific community.

  • John Wiley & Sons, Ltd. publishes most if not all of the evidence found for cochrane (for example: here. It could be that publications by John Wiley & Sons are regarded as definitive source of information.


But, all of the above doesn't really show anything. Some studies have been written just to check if Quality of Cochrane reviews are reliable or not.

A study published in 13 October 2001 cited as BMJ 2001;323:829 entitled Quality of Cochrane reviews: assessment of sample from 1998 found some inconsistencies in Cochrane reviews:

The British Guideline on the Management of Asthma, 2003 (2) contains the statement, “sodium cromoglycate is ineffective in children” (para 4.2.4). This statement was based on the conclusion of a systematic review by Tasche et al. in 2000 (3). We had criticised this review and its conclusions when first published (4, 5) on the basis that the review did provide statistically significant evidence in favour of the efficacy of sodium cromoglycate compared to placebo.

And:

We found this review also to be seriously flawed. We published detailed criticisms on the Cochrane website (8). These included inappropriate statistical analyses and incorrect interpretation of analyses presented; lack of homogeneity in the patient groups, which included the whole age range from new born babies to adolescents; the choice of a primary outcome measure for which data was available in only 4 of the 24 studies; and exclusion of trials without justification.

And concluded:

The implications for the reputation of Cochrane Reviews are serious. Criticisms of Cochrane reviews have been made previously (11, 12). Given its role in the promotion of Evidence-Based Medicine including providing evidence for the preparation of guidelines, the Cochrane Collaboration needs to clarify [explain] how it discharges its responsibilities for the quality of reviews published under its imprimatur, and how it responds when they are shown to have come to unjustified conclusions.

Another study also found inconsistencies:

The Cochrane Collaboration: A leading role in producing reliable evidence to inform healthcare decisions in musculoskeletal trauma and disorders

In this article, we have described the work of The Cochrane Collaboration, many of whose reviews, freely available on the web in numerous countries including India, will be relevant to the reader's practice. But there are many gaps in the evidence, some reflecting missing topics, and others missing perspectives. Thus, as well as using evidence to inform practice, the readers of this article should consider how they can address the deficiencies in the evidence needed to inform orthopedic practice. To do this they should take heed of the national calls for change16,17 and the establishment of the Clinical Trials Registry - India (www.ctri.in) and the South Asian Cochrane Network.

Cochrane reviews can be regarded as a definitive reliable source of information.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .