Comment on blog by Edzard Ernst on systematic review

Harald J. Hamre, Klaus von Ammon, Anja Glockmann, Helmut Kiene


--> PDF


Our publication [1] of a systematic review (SR) of meta-analyses (MA) of randomised placebo-controlled homoeopathy trials for any indication was addressed in a blog [2] by Edzard Ernst (also reprinted at the end of this document). We are happy to clarify and correct a number of wrong, misleading or unsubstantiated statements in this blog: 

Wrong, misleading or unsubstantiated statements

“…who would want to do an SR of MAs (a most peculiar exercise)… bizarre approach”

Wrong: SRs of MAs are neither peculiar nor bizarre, there are plenty of them. A Google Scholar search for the exact phrase "systematic review of meta-analyses" yields 7,510 hits, a full Google search ca 835,000 hits.

“…by pooling the MAs, they generated a positive result… this strategy (which in effect multiplies the results of many primary studies by factor 6)”

Wrong: We have not pooled any results. Pooling of several results into one effect estimate is done in MAs, this was an SR of existing MAs without any new meta-analytic pooling.

Wrong: There was no multiplication of any results whatsoever. We have followed standard procedures for SRs, grouping results and other features of the 6 MA together, and assessing the confidence in the cumulative evidence according to the detailed GRADE framework [3] (presented in Additional file 3).

“…about the ‘efficacy’ (actually it should be effectiveness)”

Wrong: The scope of our SR was MAs of placebo-controlled trials, which evaluate efficacy not effectiveness.

“…the 6 MA included more or less almost the same primary studies”

Wrong: As described in the Trial Characteristics section of our SR, the 6 MA comprised 310 trials or trial comparisons, thereof 182 different trials. Thus only 41.3% ((310-182)/310) of trials overlapped.

“The 6 included MAs are marginally positive…”

Misleading: In the GRADE framework, the term “marginally positive” corresponds to “imprecision” (significantly positive effect estimates with confidence intervals close to the threshold for 'no significant difference' [4]). The blog comment suggests all six MA had imprecise results, which was not the case: The primary outcome of our SR comprised 9 effect estimates from the six MA, of which 6 showed a significant and “more than marginally” positive effect of homeopathy, compared to placebo; 2 estimates showed a significant and “marginally” positive effect (see Additional file 3, section 1.4); and 1 estimate showed a positive but not significant effect.

“…(mainly due to publication bias and other artefacts)”

Unsubstantiated: This is essentially a reiteration of the assumption of the Shang 2005 MA [5] (“effects… could be explained by a combination of methodological deficiencies and biased reporting”, p.730) without further substantiation. We have assessed the possible impact of publication/nonreporting bias as well as other forms of bias and confounding in all six MA (see Additional file 3, sections 1.1-1.8 and Additional file 2). Further information can be found in our assessment of risk of bias in the Shang 2005 MA in Additional file 1, pp. 11-15 and our comment on the circular logic underlying the above-mentioned assumption in the Discussion (p. 21).

One of the two MAs by Mathie et al excluded primary studies that reported positive findings (i.e. mine and the one by Walach et al)”

Wrong statement: Contrary to the statement by Ernst, the two trials in question (White 2003 [6] with Ernst as last author, Walach 2000 [7]) did not report “positive findings”, i.e. significant positive effects of homeopathy, compared to placebo. For both trials, no significant nor relevant between-group differences in main outcomes were reported.

Wrong trial: Walach 2000 was an open-label long-term follow-up analysis of the primary double-blind trial Walach 1997 [8]. Hence, Walach 1997 not Walach 2000 was the primary candidate for the Mathie 2014 MA [9], which is in question here.

Unsubstantiated: Three of the six MA, including Mathie 2014, presented data on trials included in the SR section of their analysis but excluded from the MA, because they did not have results extractable for meta-analytic data pooling. In Mathie 2014, Walach 1997 and White 2003 as well as eight other trials were included in the SR section but excluded from the MA for this reason. Ernst does not substantiate any other reason for his disapproval of the two exclusions apart from the trial results.

Citations without credit to the original authors and source

This blog contains only 15 full sentences by the author, the rest are verbatim citations from our SR, only in part marked as such (cf. the CC BY 4.0 license). Thus the reader might think the first five paragraphs summarising the SR to have been written by the blogger, while they were actually copied from our abstract.


The 15 original sentences of this blog by Edzard Ernst contain a number of statements that are wrong (7 times), misleading (1x) and unsubstantiated (2x).


  1. Hamre HJ, Glockmann A, von Ammon K, Riley DS, Kiene H. Efficacy of homeopathic treatment: Systematic review of meta-analyses of randomised placebo-controlled homeopathy trials for any indication. Syst Rev 2023; 12: 191
  2. Ernst E. A new systematic review of homoeopathy reported a positive result – but I can’t take it seriously! 2023. (accessed 13.11.2023). 
  3. Guyatt G, Oxman AD, Akl EA, et al. GRADE guidelines: 1. Introduction - GRADE evidence profiles and summary of findings tables. J Clin Epidemiol 2011; 64(4): 383-94
  4. Guyatt GH, Oxman AD, Kunz R, et al. GRADE guidelines: 6. Rating the quality of evidence - imprecision. J Clin Epidemiol 2011; 64(12): 1283-93
  5. Shang A, Huwiler-Muntener K, Nartey L, et al. Are the clinical effects of homoeopathy placebo effects? Comparative study of placebo-controlled trials of homoeopathy and allopathy. Lancet 2005; 366(9487): 726-32
  6. White A, Slade P, Hunt C, Hart A, Ernst E. Individualised homeopathy as an adjunct in the treatment of childhood asthma: a randomised placebo controlled trial. Thorax 2003; 58(4): 317-21
  7. Walach H, Lowes T, Mussbach D, et al. The long-term effects of homeopathic treatment of chronic headaches: 1 year follow up. Cephalalgia 2000; 20(9): 835-7
  8. Walach H, Haeusler W, Lowes T, et al. Classical homeopathic treatment of chronic headaches. Cephalalgia 1997; 17(2): 119-26
  9. Mathie RT, Lloyd SM, Legg LA, et al. Randomised placebo-controlled trials of individualised homeopathic treatment: systematic review and meta-analysis. Syst Rev 2014; 3: 142

Wir benötigen Ihre Zustimmung zum Laden der Übersetzungen

Wir nutzen einen Drittanbieter-Service, um den Inhalt der Website zu übersetzen, der möglicherweise Daten über Ihre Aktivitäten sammelt. Bitte überprüfen Sie die Details in der Datenschutzerklärung und akzeptieren Sie den Dienst, um die Übersetzungen zu sehen.