HOMEOPATHY IS EFFECTIVE - STUDY PROVES
Evidence for the effectiveness of homoeopathic treatments is as strong as conventional treatments.
A new analysis, published in BMC Systematic Reviews, reviewed six meta-analyses of placebo-controlled randomised efficacy trials of homeopathy for any indication. It found, that contrary to frequent claims, homeopathy has significant positive effects beyond that expected from placebo.
The evidence provided by the new study further undermines the continued attacks on homeopathy and shows there's no justification for regulatory or political measures against the practice of homeopathy in mainstream healthcare.
QUOTE: "...Download PDF
Downloa
Methods
The inclusion criteria were as follows: MAs of PRETHAIs in humans; all ages, countries, settings, publication languages; and MAs published from 1 Jan. 1990 to 30 Apr. 2023. The exclusion criteria were as follows: systematic reviews without MAs; MAs restricted to age or gender groups, specific indications, or specific homoeopathic treatments; and MAs that did not assess efficacy. We searched 8 electronic databases up to 14 Dec. 2020, with an update search in 6 databases up to 30 April 2023.
The primary outcome was the effect estimate for all included trials in each MA and after restricting the sample to trials with high methodological quality, according to predefined criteria. The risk of bias for each MA was assessed by the ROBIS (Risk Of Bias In Systematic reviews) tool. The quality of evidence was assessed by the GRADE framework. Statistical analyses were performed to determine the proportion of MAs showing a significant positive effect of homoeopathy vs. no significant difference.
Results
Six MAs were included, covering individualised homoeopathy (I-HOM, n = 2), nonindividualised homoeopathy (NI-HOM, n = 1) and all homoeopathy types (ALL-HOM = I-HOM + NI-HOM, n = 3). The MAs comprised between 16 and 110 trials, and the included trials were published from 1943–2014. The median trial sample size ranged from 45 to 97 patients. The risk of bias (low/unclear/high) was rated as low for three MAs and high for three MAs.
Effect estimates for all trials in each MA showed a significant positive effect of homoeopathy compared to placebo (5 of 5 MAs, no data in 1 MA). Sensitivity analyses with sample restriction to high-quality trials were available from 4 MAs; the effect remained significant in 3 of the MAs (2 MAs assessed ALL-HOM, 1 MA assessed I-HOM) and was no longer significant in 1 MA (which assessed NI-HOM).
Discussion
The quality of evidence for positive effects of homoeopathy beyond placebo (high/moderate/low/very low) was high for I-HOM and moderate for ALL-HOM and NI-HOM. There was no support for the alternative hypothesis of no outcome difference between homoeopathy and placebo.
The available MAs of PRETHAIs reveal significant positive effects of homoeopathy beyond placebo. This is in accordance with laboratory experiments showing partially replicable effects of homoeopathically potentised preparations in physico-chemical, in vitro, plant-based and animal-based test systems.
Systematic review registration
PROSPERO CRD42020209661. The protocol for this SR was finalised and submitted on 25 Nov. 2020 and registered on 26 Dec. 2020.
Background and rationale
Homoeopathy is a therapy system widely used in Europe, India and other countries [1]. Core features of homoeopathy include drug provings (observation of symptoms occurring in healthy persons exposed to substances of mineral, botanical or zoological origin), simile principle (similarity between symptom patterns in drug provings and the symptoms to be treated with the same substance) and potentization (successive dilution of the homoeopathic substance, with each dilution step involving repeated shaking of liquids or grinding of solids into lactose) [2].
The clinical effects of homoeopathic treatment have been investigated in several hundred randomised controlled trials [3] and in systematic reviews (SRs). Among the SRs, two contrasting approaches can be discerned.
One approach is to focus on a specific indication (e.g., depression [4], acute respiratory tract infections in children [5]) while often including open-label trials and observational studies. In this approach, data synthesis is grouped by design, thus yielding information about homoeopathy in patient care.
The opposite approach is to include all indications while restricting study designs to placebo-controlled trials and aggregating results in an MAs, thus yielding information about the specific effects of homoeopathy beyond those of placebo. A major reason for using this approach has been the claim that ‘homoeopathy violates natural laws and thus any effect must be a placebo effect’ [6].
Since 1997, at least six MAs of placebo-controlled homoeopathy trials for any condition have been published [6,7,8,9,10,11]. These MAs have differed in their methods for trial inclusion, data synthesis and assessment of risk of bias; furthermore, their results and conclusions have been inconsistent. During this period, there have been substantial advancements in methodology and quality standards for MAs and other SRs [12,13,14,15], including SRs of SRs (also called overviews or umbrella reviews) [16,17,18]. To our knowledge, a formal SR of MAs of randomised placebo-controlled homoeopathy trials for any condition has not been performed. Herein, we report such an SR.
Objectives
Research questions
1.
2.
Methods
Eligibility criteria for meta-analyses (MAs)
The eligibility criteria are presented in Table 1.
Table 1 Eligibility criteria for meta-analyses
Full size table
Information sources and search strategy
Databases
We searched eight online databases, including four databases largely or totally restricted to SRs (A–D), two generic databases (E–F) and two databases focused on complementary or alternative therapies (G–H) (Table 2). In addition, one private database (author HJH) was searched.
Table 2 Online databases and search strategies
Full size table
Other sources
A list of included MAs was sent to experts in the field to identify any missing eligible MAs or additional analyses of the included MAs.
Selection process
Screening
Two reviewers (HJH, AG) independently searched the online literature databases and screened the titles and abstracts to identify potentially eligible MAs. The reviewers compared their screening results, and discrepancies were resolved by discussion (HJH, AG).
Eligibility
For the potentially eligible MA records, full-text reports were obtained. Two reviewers (HJH, AG) independently read the full texts and assessed their eligibility in accordance with the eligibility criteria (Table 1). The reviewers compared their eligibility assessments, and discrepancies were resolved by discussion (HJH, AG).
Data collection process
Two reviewers independently extracted data from the full-text reports into Excel files (HJH + [GSK, HK or AG]) using a piloted data extraction form. Reviewer AG compared the two sets of extracted data. Discrepancies were resolved by discussion (HJH + [GSK, HK or AG]).
We extracted and summarised trial-level data from tables of the MAs but did not inspect original trial publications (with one exception, cf. Additional file 2, Section 2.3.1). Indications/diagnoses in individual trials were coded according to the International Classification of Diseases, 10th Edition (ICD-10). If more than one diagnosis was listed, the first listed diagnosis was coded. If two trials or trial comparisons were analysed separately in one MA and analysed together in another MA, they were counted as 3 trials or trial comparisons, respectively. If more than one trial report for the same trial was listed, only one trial report was extracted.
Data items
All outcomes in the following subsections refer to the combined effect estimate with a measure of precision for the primary clinical outcome reported in each MA (henceforth ‘effect estimate’).
Primary outcome
Effect estimates for.
1.
2.
trials of higher methodological quality (or lower risk of bias), as stated and defined by the authors of the MA
based on an assessment of at least three specified components of methodological quality (e.g. concealment of allocation sequence, blinding of outcome assessors)
maximum one single high-quality category defined for the respective MA
Sensitivity analyses
Effect estimates in sensitivity analyses, calculated after restricting the sample based on the methodological quality (risk of bias) of individual trials, as assessed by:
individual quality (risk of bias) components such as concealment of allocation sequence, double blinding [blinding of participants, study personnel and outcome assessors], risk of outcome reporting bias, peer-reviewed trial publication
the criterion ‘high-quality trials’ (as in Item 2 above) + one or several additional quality components
other combination of quality components, grouped by total number of components in the respective analysis: 2–4 or ≥ 5
cumulative MAs with stepwise removal of trials by risk-of-bias ratings, conceptualised in a hierarchical order by the authors of the respective MA (e.g. ascending numbers in a numeric scale or ‘poor’, ‘fair’, ‘good’)
Supplementary analyses addressing meta-bias
Effect estimates in supplementary analyses based on assumed risk of bias across trials (meta-bias):
Statistical adjustment for possible publication bias/small study bias
Sensitivity analyses, with restrictions of included trials, based on trial sample size
Analyses addressing possible outcome reporting bias
Combined analyses
Effect estimates in analyses combining features of Sections 'Sensitivity analyses' and 'Supplementary analyses addressing meta-bias' above.
Subgroup analyses
With regard to research question 2, five types of trial subgroups in the respective MAs (A.1–5) were examined. The subgroup analyses had four types of results (B.1–4), and they were grouped by the timing of the analysis (C.1–2):
A.
1
a
b
c
d
e
2
3
4
5
a
b
B.
1
2
3
4
C.
1
2
Other variables
Other variables collected from the MAs are listed in Suppl. Table 1.
Assessment of risk of bias in the included MAs
Risk of bias/methodological quality of the MA was assessed using the ROBIS tool (Risk of Bias in Systematic Reviews) [13], supplemented with items 7, 10 and 16 from the AMSTAR-2 tool (A MeaSurement Tool to Assess systematic Reviews) [14], which are not addressed in ROBIS. Assessments were performed independently by two reviewers (HJH, GSK); discrepancies were resolved by discussion between the reviewers.
The outcome of these assessments was the composite body of reports, comprising.
1
2
3
Effect measures
Effect estimates of each MA (cf. Section 'Outcomes', above) were reported using the metric reported in the MA (e.g., odds ratio [OR], standardised mean difference [SMD]). Standardised mean differences for homoeopathy vs. placebo were reported with point estimates > 0 indicating a benefit of homoeopathy.
Synthesis methods
Effect estimates were summarised in table format and classified as follows:
1
2
3
If both fixed effects and random effects models had been used for the same analysis, the results from random effects models were used for the data synthesis herein.
Meta-bias assessment
See Sections 'Supplementary analyses addressing meta-bias' and 'Combined analyses', above.
Confidence in cumulative evidence/certainty assessment
Confidence in cumulative evidence for the two research questions (Sect. Research questions) was assessed.
For question 1, the conceptual framework of the Grading of Recommendations Assessment, Development and Evaluation (GRADE) group [20] was used, with a focus on six issues: risk of bias of individual trials [21], inconsistency/heterogeneity [22], risk of publication bias/small study bias [23], imprecision [24], indirectness [25] and occasions for rating up the quality of evidence [26].
For question 2, results of subgroup and heterogeneity [22] analyses were used.
Results
Identification, screening and inclusion of meta-analyses
From the eight online databases, we identified 293 literature records of potentially eligible meta-analyses (search completed on 14 Dec. 2020). After the removal of 82 duplicates, 211 records were screened, of which 191 were excluded and 20 were further assessed for eligibility. In addition, searches in the database of reviewer HH (20 Jan. 2021 + addition of Gartlehner 2022 on 04 July 2022, cf. Section 'Additional data: Gartlehner 2022') and letters to experts (sent 10 Feb. 2021) yielded a total of 9 nonduplicate records that were also assessed for eligibility. Thus, 29 full-text reports were assessed for eligibility, of which 13 were excluded. Thus, 16 reports of 6 different MAs were included (PRISMA 2020 [27] flow diagram, cf. Fig. 1).
Fig. 1
PRISMA 2020 flow diagram for new systematic review which included searches of databases, registers and other sources
Full size image
By 30 April 2023, a period of 30 months had passed after the end of the report time frame according to the original eligibility criteria (reports published up to 31 Oct. 2020). We therefore conducted an updated search of reports published in the period from 01 Nov. 2020 to 30 April 2023. We searched databases A–C, E, G–H (Table 2; D was no longer available, and F was omitted for budget reasons, having yielded no nonduplicate records in the primary search) and the database of reviewer HJH. The updated search yielded 13 records, of which 11 were excluded and 2 were assessed for eligibility. Of these, 1 report had already been included on 04 July 2022 (Gartlehner 2022 cf. Section 'Additional data: Gartlehner 2022'), and 1 was excluded (PRISMA 2020 flow diagram for the update in Additional file 4).
A list of the 14 excluded publications (original search: n = 13, update n = 1) with reasons for exclusions is presented in Suppl. Table 2.
The 16 reports consisted of 6 primary publications of one [6,7,8, 10, 11] or two [9] MAs, 2 published MA protocols [28, 29], 7 publications of additional analyses [3, 30,31,32,33,34] and 1 error correction [35] (Table 3).
Table 3 Overview of included meta-analyses and publications
Full size table
Description of meta-analyses
Chronological overview
The six MAs were published in the period 1997–2017. The two first (Linde 1997 [6] and 1998 [7]) and the two most recent (Mathie 2014 [10] and 2017 [11]) MAs were MA ‘pairs’, i.e. they were conducted and published by the same first author with overlapping co-authorships. The other two MAs (Cucherat 2000 [8], Shang 2005 [9]) were published by different author groups.
The MA conducted by Linde (1997) [6] was the first MA of placebo-controlled homoeopathy trials for any condition worldwide. The primary publication was followed by a detailed assessment of the relation between study quality (risk of bias) and effect estimates (Linde 1999) [30]. The MA conducted by Linde (1998) [7] was an updated subgroup analysis of Linde (1997) [6], restricted to I-HOM.
The MA conducted by Cucherat (2000) [8] originated from a homoeopathy report prepared for the European Parliament by the Homoeopathic Medicine Research Group (Boissel 1996) [31]. Compared to the Boissel report, the MA conducted by Cucherat [8] had modifications in some analyses. We considered this MA the definitive work, but we also consulted the Boissel report as an additional source of details on the methods and conduct of the MA.
The MA conducted by Shang [9] was designed as a prospective comparison of two MAs of placebo-controlled trials: one MA of any type of homoeopathic treatment for any disorder and one MA with matched trials on conventional treatment. According to the protocol for the present SR [37], the results of the latter MA were beyond the scope of this SR. However, the authors of the MA conducted by Shang [9] used the results of the MA on conventional treatment to draw inferences about the homoeopathy MA results. We therefore included comparative data on the two MAs (presented in Additional file 2).
The MAs conducted by Mathie (2014, 2017) [10, 11] were part of a comprehensive MA program (Mathie 2013) [3], covering placebo-controlled trials of individualised [10] and nonindividualised [11] homoeopathy, respectively.
Methods of the meta-analyses
Research objective or hypothesis
The main research objective concerned the efficacy of homoeopathic products vs. placebo in all six MAs: generally stated [7, 8] or in terms of outcome difference between homoeopathy and placebo [6, 10, 11] (full text excerpts in Suppl. Table 3). In the MA conducted by Shang [9], the research hypothesis was further specified: ‘We assumed that the effects observed in placebo-controlled trials of homoeopathy could be explained by a combination of methodological deficiencies and biased reporting’ (Discussion, p.730).
Eligibility criteria
Design, publication types
In all six MAs, parallel group randomised trials were included, while crossover trials were excluded from four MAs [6, 9,10,11], included in the MA conducted by Linde (1998) [7] and not mentioned in the MA conducted by Cucherat [8]. Four MAs had no restrictions regarding publication format, while two (Mathie 2014 and 2017) [10, 11] were restricted to peer-reviewed journal articles of at least 500 words (Suppl. Table 4).
Patients and indications
Restriction to disease groups as such was not applied in any MA (Suppl. Table 5). Notably, in the MA conducted by Shang [9], the homoeopathy trials were compared to placebo-controlled trials of interventions used in conventional medicine, matched for indication. For 94.0% (n = 110/117) of otherwise eligible homoeopathy trials, a trial of conventional medicine for the respective indication could be found, while 7 unmatchable homoeopathy trials were excluded.
Interventions, comparators
In the MAs conducted by Mathie (2014 and 2017) [10, 11], the homoeopathic intervention types were restricted as follows: radionically prepared medicines, anthroposophic medicine, homotoxicology, and homoeopathy combined with other (complementary or conventional) treatments were excluded (Suppl. Table 6).
Other
In the meta-analysis conducted by Cucherat [8], ‘only trials with a clearly defined primary outcome’ were included (Suppl. Table 7).
Literature search and inclusion, data extraction and analysis
For all six MAs, previously published MAs or SRs [38] were consulted. Between 4 [6] and 19 [9] online databases were researched. For all MAs, experts in the field were contacted for information on additional trials; manual searches of reference lists were used in five MAs but not in the MA conducted by Linde (1998) [7], which was largely an update on their previous MA from 1997 (Suppl. Table 8). Screening of titles and abstracts was performed independently by two reviewers in the MA conducted by Linde (1997) [6] and by one reviewer in the MA conducted by Cucherat [8]. The screening approach was not reported in the four other MAs. Full-text assessments were performed independently by two persons in the MA conducted by Linde (1997) [6]; by one person and checked in part by another person in the MA conducted by Cucherat [8]; and by one person in the MA conducted by Linde (1998) [7]. The full text assessment approach was not reported in three MAs.
Data extraction was performed independently by two persons in five MAs and by one person in the MA conducted by Linde (1998 [7]). Risk of bias assessments were performed independently by two persons in three MAs [6, 10, 11] and by one person in the MA conducted by Linde (1998 [7]). The number of persons performing risk of bias assessment was not reported in two MAs. Lists of excluded trials were available in three MAs [9,10,11]. The reasons for exclusion of trials were provided in all MAs except the one conducted by Linde (1998) [7] (Table 4).
Table 4 Quality of trial data handling
Full size table
All six MAs used one main clinical outcome for each trial or trial comparison. For the MA conducted by Cucherat [8], this was the primary outcome as reported in the trials (cf. Section 'Eligibility criteria', above); for the other MAs, a predefined hierarchical list of criteria for extraction of the main outcome was used (Suppl. Table 9).
Protocol
For two MAs (Mathie 2014 and 2017) [10, 11], a prepublished protocol was available; for two MAs (Linde 1997. Cucherat [6, 8]), a protocol was referred to in the publication; and for two MAs (Linde 1998, Shang 2005 [7, 9]), a protocol was not mentioned in the publication, while one single design criterion (outcome extraction in both cases) was explicitly stated as predefined.
Risk of bias assessment, heterogeneity, meta-bias
High-quality trials
High-quality trials according to our criteria (cf. Section 'Data items' / 'Primary outcome', above) were performed in four MAs [6, 9,10,11]. The criteria for high-quality trials were described as predefined (Linde 1997) [6] or fully (Mathie 2017) [11] or partially (Mathie 2014) [10] defined in a prepublished protocol. One MA did not mention this aspect (Shang [9]). The criteria for high-quality trials were as follows:
The MA conducted by Linde (1997) [6] used a combination of two score-based instruments:
Jadad score [39] (range 0–5 points, thereof 0, 1 or 2 points each for items no. 1 and 3 and 0–1 point for item 11 in Table 5): ≥ 3 points
Internal validity scale [30] (range 0–7 points, thereof 0, 0.5 or 1 point each for items 1–2, 4–7 and 11 in Table 5): ≥ 5 points
Table 5 Criteria for high-quality trials
Full size table
The instruments used in the following MAs consisted of sets of mandatory criteria, all of which were to be fulfilled.
The MAs conducted by Mathie (2014 and 2017) [10, 11] used the Cochrane risk-of-bias tool (RoB, version 2011) [40]: low risk of bias for items 1–2 and 4–5 in Table 5, low risk for two of the three items 8 and 12–13 and low or uncertain risk for one of the latter four items.
In the MA conducted by Shang [9], the number of quality components used was variously described as 3 or 4, corresponding to fulfilment of items (1–3) or (1–3 + 10) in Table 5. Lüdtke [32] interpreted Shang [9] as having used 3 components (Suppl. Table 29). Details in support of either 3 or 4 components are presented in Suppl. Table 11.
The high-quality criteria were based on 8 [6], 7 [10, 11] and either 3 or 4 quality components [9] (Table 5).
Risk of bias (methodological quality) otherwise
The total number of methodological quality components assessed in each MA (including components of high-quality criteria as well as other components) ranged from 3 [8] to 10 [6, 7], details in Suppl. Table 12.
Associations between quality components and outcome were analysed with hypothesis testing in four MAs (not in the MA conducted by Linde (1998) [7] and Cucherat [8]).
Cumulative MA with stepwise removal of trials according to increasing quality categories was performed in four MAs using interval-scaled [7, 10, 11] or rank-ordered [8] categories. Of the two other MAs, one [7] had outcome analysis in 4 ranked quality subgroups instead of cumulative MA.
Statistical heterogeneity testing was performed in four MAs (not in the MAs conducted by Linde (1998) [7] and Cucherat [8]); all but one MA [7] included an assessment of publication bias/small study bias (Suppl. Table 14).
Potential conflicts of interest were stated and explained for at least one author in two MAs (Mathie 2014 and 2017) [10, 11]; a statement of no conflicts of interest for any author was included in one MA (Shang) [9], while this issue was not addressed in the three other MAs.
Trial characteristics
Number of trials, trial comparisons and trial reports
For each MA, between 150 and 359 full-text records were assessed for eligibility (data available for four MAs) and between 16 and 119 trials were eligible for SR, including 16–110 trials with extractable data for MA. Altogether, 182 different trials (or in some cases, trial comparisons) reported in 165 different publications or other trial reports were included in the 6 MAs. Of these, n = 88 trials were included in 1 MA, 65 trials in 2 MA, 24 trials in 3 MA and 5 trials in 4 MA, with a total of 310 trials or trial comparisons (Suppl. Table 15). All following descriptions refer to these 310 trials.