Expectation Bias-the Main Culprit for Large Number of Inconclusive Randomized Controlled Trials in Hematological Malignancies

Background: When randomized clinical trials (RCTs) are conducted there should be some reasonable expectations that the trials will provide the answer to the questions they were design to address. Failure to do so i.e. completing trials that will result in inconclusive results represents an avoidable waste of precious resources and ethical breach of contracts with patients who expect that definitive answers will help future patients. Two main factors are thought to be responsible for generating inconclusive findings in clinical research: poor patient accrual and expectation bias or ‘optimism bias’ - an unwarranted belief in the efficacy of new therapies. (

Objectives: Our objective was to determine which of the two factors, poor patient accrual or expectation bias, is the main culprit for producing inconclusive clinical research results in the trials focusing on hematological cancers.

Methods: We extracted data from the original protocols and publications related to 210 hematological cancer trials conducted by the 8 NCI sponsored Cooperative Groups conducted and published from year 1955 to 1998. A priori effect size used in the power calculation for the trial’s pre-defined primary outcome (expected differences) was compared with those observed in the final reports. All treatment effects were expressed as either HR [hazard ratio] or OR [odds ratio]. We also analyzed the planned accrual vs. actual patient accrual. The question whether “negative”/inconclusive i.e. statistically non-significant results were due to poor accrual or due to “optimism” bias was addressed. We defined inconclusive results as those in which treatment was consistent with no difference between treatment effects, or experimental treatment being superior to the standard treatments, or vice verse. Operationally, this was defined as the point estimate of the treatment effect and its 95% confidence intervals (CI) crossing the lines of equivalence on either side of the no-treatment effect from 0.77 to 1.3.

Results: Data on primary outcomes were available for 105 out of 210 total hematological cancer studies. 71% of the studies had non-significant results (75/105). The vast majority of these trials (92%, 69/75) were inconclusive i.e. 95% CI of their treatments had crossed both limits of equivalence. Overall, the investigators enrolled more patients than actually planned [Nplanned: median: 220, mean: 280, range: 40–1100; Nactual: median: 250, mean: 336, range: 40–1606; p=0.1203]. However, the expected effect size was considerably greater than actually observed differences in treatment effects: [Expected: mean=47% (range: 17% to 74%) vs. Observed: mean=12% (range: −88% to 83%; p=0.000). The median ratio between the expected and the observed HR/OR was 1.77 [range: 0.34–6.41], meaning that observed HR/OR fell short of the expected HR/OR values by 1.77 times on average.

Conclusion: Unrealistic expectations in treatment effects appear to be a major culprit for continuing generation of large number of “negative”/inconclusive results in the field of hematological malignancies.