Network meta-analysis of randomized trials in multiple myeloma: efficacy and safety in relapsed/refractory patients

Multiple myeloma (MM) is among the most common hematologic malignancies. Current milestones of MM therapy include either a triple- or double-drug combination, based on proteasome inhibitors (PIs) and/or immunomodulatory imide drugs (IMiDs) plus dexamethasone with or without chemotherapy. Eligible patients further undergo autologous stem cell transplantation and consolidation therapy,^1,2  and patients not eligible for autologous stem cell transplantation enter follow-up.^3,4  However, virtually all patients still relapse and require further treatment. Thus, relapsed/refractory MM (RRMM) currently represents the main focus of intensive clinical research. Indeed, a plethora of new agents, including second-generation PIs, histone deacetylase (HDAC) inhibitors, and monoclonal antibodies (mAbs), have shown consistent activity in prospective phase 2/3 clinical trials.^5,6  On the basis of these premises, we undertook a systematic review and 2 pairwise meta-analyses of phase 2/3 randomized trials in the RRMM setting to evaluate the impact and safety of new therapeutic strategies compared with bortezomib with or without dexamethasone or IMiDs + dexamethasone regimens. Because each trial usually compares only 2 regimens, it was challenging to evaluate the relative efficacy of all the regimens we investigated. To overcome these limitations, we further performed a network meta-analysis (NMA), which is a recently introduced Bayesian statistical approach that allows combining direct and indirect evidence to rank the different treatments according to their efficacy and safety.

An electronic search for relevant publications was performed by using PubMed, Embase, Ovid, the Cochrane collaboration database, and proceedings from the major international meetings in hematology and oncology. Only prospective studies were allowed in this analysis.⁷  The following search headings were used: “multiple myeloma”, “relapse”, “refractory”, “randomized clinical trial”, “management”, “bortezomib”, “lenalidomide”, “thalidomide”, and “therapy” in different combinations (eg, “relapsed/refractory multiple myeloma”). All titles were screened and abstracts were reviewed. The related articles function, article references, and Google Scholar were also screened for other applicable publications and were used for searching related studies, abstracts, and citations. Only articles in English were considered. The last date of the search was June 24, 2016. A systematic review was performed according to the guidelines and recommendations from the preferred reporting items for systematic reviews and meta‐analyses (PRISMA) checklist.^8,9 

To be included in the analysis, the studies had to meet several criteria. They had to involve only patients with a diagnosis of RRMM (including patients in second-line treatment), be randomized controlled trials (RCTs) with or without blinding, be abstracts or unpublished data (to be included only if they had sufficient information on study design, characteristics of participants, interventions, and outcomes), have patients who received an unconventional or new regimen in the experimental arm, and have patients who received a standard regimen for RRMM in the control arm.

Studies were excluded from the analysis if they were not comparative or not prospective, if outcomes of interest were not reported, if the methodology was not clearly reported, and if only patients after the second line of treatment were included.

Two reviewers (C.B. and N.S.) independently reviewed the literature according to the above predefined strategy and criteria. Each reviewer extracted the following data: title and reference details (first author, year), study population characteristics (number of patients in study, number treated by each approach, number of treatment lines, previous exposure to bortezomib or IMiDs), type of interventions, and outcome data. For each trial, we evaluated hazard ratios (HRs) of progression-free survival (PFS) and overall survival (OS); odds ratio (OR) of overall response rate (ORR), very good partial response rate (VGPR), and complete response (CR); and risk ratio (RR) for safety (evaluation of common toxicities). If the HR of survival curves was not reported,^10,11  it was derived from the graph by using the method of Tierney et al¹²  (see supplemental Data).

All data were recorded independently in separate databases and were compared at the end of the reviewing process to limit selection bias. The database was also reviewed by 2 additional investigators (D.C. and M.C.). Duplicates were removed and any disparities were clarified.

Selected studies were assessed for quality according to the CochraneHandbook for Systematic Reviews of Interventions, as described elsewhere,¹³  by assigning 1 point for each of the following 5 requirements: method of randomization, allocation concealment, blindness, withdrawal or dropout, and adequacy of follow-up. Studies were ranked as A if they had 4 to 5 points, B if they had 2 to 3 points, or C if they had 0 to 1 point of a total of 5 points.¹⁴  The presence of publication bias was investigated by using Egger’s test¹⁵  or Begg’s test¹⁶  and by visual inspection of funnel plots.

Traditional pairwise meta-analyses were carried out as described elsewhere.^17-19  The following comparisons were used: experimental therapy vs conventional bortezomib-based therapy or experimental therapy vs conventional IMiD-based therapy (additional details are provided in the supplemental Data). Survival data were extracted as HRs of OS and PFS with relative confidence intervals (95% CIs); response rates and toxicity rate were calculated by using the method for dichotomous data (OR and RR assessment with 95% CIs). Cochrane’s Q test and I² statistics were used to assess heterogeneity between studies, and both fixed-effects (FE) and random-effects (RE) models were used for the analysis. Pooled data analysis was performed according to the DerSimonian and Laird test. P < .05 was considered statistically significant. Statistical analysis was conducted by using STATA software (Version 14.1, STATA, College Station, TX) and R (meta package).

We performed an NMA by using both Bayesian and frequentist approaches to compare the different therapeutic regimens simultaneously. The Bayesian NMA was performed with STATA software by using the mvmeta package, and the frequentist NMA was performed with R software by using the netmeta package.²⁰  NMA synthesizes data from a network of trials that involve multiple interventions and therefore has the potential to rank the treatments according to the outcome. This method integrates direct and indirect comparisons. Within the framework of NMA, we ranked the evaluated regimens based on survival outcomes (OS and PFS), treatment efficacy (ORR and CR), and safety (the most frequent grade 3-4 adverse event in each trial). For each outcome, we performed a Bayesian NMA with an (RE) model by using a Markov chain Monte Carlo simulation technique with up to 30 000 iterations. Loop inconsistency and heterogeneity were assessed by evaluating the logarithm of the ratio of 2 odds ratios (RoR) from direct and indirect evidence in the loop with the ifplot command in STATA.^21,22  RoR values close to 0 indicate that both direct and indirect evidence are in agreement. Heterogeneity of the loop was then assessed through the restricted maximum likelihood method.^21,22 

Relative effects of treatments are reported as HRs for survival outcomes (PFS and OS) and OR or RR for binary outcomes (ORR, CR, and toxicity) along with corresponding 95% credible intervals, the Bayesian equivalent of 95% CIs. Ranking probabilities and surface under the cumulative ranking curve (SUCRA), which provides a numerical summary of the rank distribution of each treatment schedule on the different end points, were used to provide hierarchy probabilities.^21-23  The larger the SUCRA value (ie, closer to 1), the better the rank of the intervention. The frequentist equivalent of the SUCRA, the P-score, was further used to confirm the results of the NMA.²⁰ 

Figure 1 shows a PRISMA chart of RCT selection and search strategy. In the time frame covered by this systematic review, a total of 1679 studies, including full articles or meeting abstracts, were retrieved. Nineteen trials with a total of 8997 patients were finally selected and included in the analysis (Table 1).^10,11,24-42  Two trials^41,42  did not report data or provide a graph showing OS and were therefore excluded from all OS analyses. At least 1 data comparison in terms of survival, ORR, or toxicity was reported in all selected RCTs, which were therefore deemed eligible for the analysis. A summary of the 19 trials included in the final analyses showed that 17 trials were eligible for OS analysis; 19 were eligible for PFS, ORR, and safety analyses; 16 were included in the pairwise meta-analyses; and 18 were included in the NMA.

As depicted in Table 2, 15 trials were scored as A (low risk of bias), and 4 trials were scored as B (intermediate risk of bias). None of the trials evaluated was scored as C (high risk of bias).

Our analyses included 16 trials for pairwise meta-analyses (involving 7500 patients). In particular, there were 10 trials (4371 patients)^{10,11,33-37,39-41}  in which an experimental treatment was compared with a bortezomib-based conventional treatment and 6 trials (3129 patients)^11,24-27,42  in which an experimental treatment was compared with an IMiD-based conventional treatment. Funnel plot visual inspection (supplemental Figure 1), Egger’s test (t = 0.57; P = .58), and Begg’s test (z = 0.80; P = .42) showed no evidence of publication bias for bortezomib comparison. For IMiD comparison (supplemental Figure 1), only funnel plot visual inspection was performed because of the low number of studies (6) included in the analysis, again suggesting no evidence for publication bias. All these tests should be considered to have low power because of the small number of studies involved in our analysis.

The experimental treatment, when compared with bortezomib-based regimens, significantly improved PFS by using either an FE or RE model (pooled HRs for RE, 0.67; 95% CI, 0.58-0.78; P < .001; Figure 2A). By using subgroup analysis, we demonstrated significant PFS benefit in doublet therapy subgroups and in the subgroup for therapy with all classes of drugs except IMiDs (pooled HRs, 0.97; 95% CI, 0.65-1.44; P = .880; Figure 2A; supplemental Figure 2A).

The experimental treatment, when compared with IMiD-based regimens, significantly improved PFS (pooled HRs for RE, 0.66; 95% CI, 0.54-0.81; P < .001; Figure 2B). Subgroup analysis demonstrated significant PFS benefit in the doublet therapy subgroup and in the subgroup for therapy with all classes of drugs except bortezomib (pooled HRs, 0.78; 95% CI, 0.47-1.30; P = .341; Figure 2B; supplemental Figure 2B).

In our analyses, both models showed high heterogeneity, and the prediction intervals suggest an overall nonsignificant result. Specifically, the highest level of heterogeneity was mainly a result of the study by Hjorth et al.¹¹  However, this study had a low weight in both FE and RE models, and we can conclude that the addition of another drug to either bortezomib or the IMiD backbone or the use of carfilzomib (only in the case of bortezomib) is significantly better than the backbone alone.

In our analysis, the experimental treatment showed a trend toward benefit in terms of OS compared with conventional treatment (pooled HRs for RE, 0.90; 95% CI, 0.80-1.01; P = .06). This result was confirmed in subgroup analyses for all classes of drugs and for doublet therapy and single-drug therapy (Figure 2C; supplemental Figure 2C).

In our analysis, the experimental treatment showed a trend toward benefit in terms of OS (pooled HRs for RE, 0.83; 95% CI, 0.66-1.44; P = .101). The models showed high heterogeneity, and the very wide prediction intervals suggest overall uncertain results. Subgroup analyses suggest again that heterogeneity is mainly dependent on the study by Hjorth et al.¹¹  Indeed, each subgroup analysis showed no heterogeneity, and we observed a significant advantage for doublet, IMiD + mAb, and PI + IMiD groups (Figure 2D; supplemental Figure 2D). Note that the doublet group included all studies except the study by Hjorth et al,¹¹  thus further underscoring the relevance of this study in increasing the overall heterogeneity of the model.

From the previous findings, we can conclude that the addition of a second drug to a bortezomib backbone or the use of carfilzomib does not improve patients’ survival compared with the bortezomib backbone alone, whereas the addition of another drug to the IMiD backbone significantly improves patients’ OS.

In terms of ORR, we found a significant advantage for the experimental treatment (OR, 1.45; 95% CI, 1.16-1.82; P = .001). This advantage was clearly evident with HDAC inhibitor, carfilzomib, and doublet subgroups (supplemental Figure 3A-B). Differences in terms of VGPR are shown in supplemental Figure 3C-D. In terms of CR, we demonstrated a significant advantage for the experimental treatment (OR, 1.84; 95% CI, 1.50-2.26; P < .001). This advantage was evident in all subgroups except BORT + OTHER (supplemental Figure 3E). As for doublet therapy and single-drug treatment subgroups, a significant CR benefit was observed for doublet therapy (supplemental Figure 3F).

In terms of ORR, we demonstrated a significant advantage for the experimental treatment (OR, 2.23; 95% CI, 1.56-3.20; P < .001) across all subgroup analyses except for bortezomib (supplemental Figure 4A-B). In terms of VGPR, a significant difference was found in favor of the experimental arm (pooled VGPRs, 2.49; 95% CI, 1.70-3.64; P < .001) as confirmed for all subgroups except for the IMiD + mAb group (supplemental Figure 4C-D). In terms of CR, we found a significant advantage for the experimental treatment (OR for CR, 2.30; 95% CI, 1.29-4.10; P = .005; supplemental Figure 4E-F).

Altogether, our results suggest that the addition of a second drug to the IMiD backbone or bortezomib backbone (or the use of carfilzomib) significantly increased the probability of obtaining a highest ORR, VGPR, or CR and that this result is more evident in patients treated with HDAC inhibitors, anti-MM mAbs, or PIs + IMiDs.

Common adverse events were similar in both meta-analyses, as detailed in Table 3. We observed a significant difference for thrombocytopenia in the experimental arm of IMiD meta-analysis and a major risk of neutropenia, fatigue, and serious adverse events in the experimental arm of bortezomib meta-analysis.

All treatments analyzed were divided into 9 groups: dexamethasone, bortezomib, IMiDs, bortezomib + HDACs, bortezomib + mAbs, bortezomib + other drugs, carfilzomib, PI + IMiDs, and IMiDs + mAbs according to drug class (details are provided in supplemental Table 1) or as independent treatments (except for bortezomib or bortezomib + dexamethasone regimens grouped into the “bortezomib with or without dexamethasone” group; for thalidomide, thalidomide + dexamethasone and lenalidomide + dexamethasone were grouped into the “IMiDs with or without dexamethasone” group). All regimens or groups were evaluated for differences in PFS, OS, ORR, CR, and safety (network plots are shown in Figure 3A; supplemental Figure 5A-B).

Each group was simultaneously compared against all other groups through a Bayesian NMA, and efficacy results for PFS and safety are shown in Figure 3B in terms of HRs with 95% credible intervals (efficacy results in terms of OS, ORR, and CR are shown in supplemental Figure 6). All groups evaluated had a significantly lower risk for progression compared with dexamethasone, whereas dexamethasone was found to be significantly safer than any other group. Bortezomib + mAbs and carfilzomib groups were significantly better than the bortezomib alone group, whereas the PI + IMiD group was significantly better than IMiDs alone. Interestingly, IMiDs + mAbs was the only group significantly better than both IMiDs and bortezomib. Regarding safety, the only significant value (except for dexamethasone) was reached by the IMiDs group, which shows a significantly lower RR for toxicity compared with the bortezomib + HDAC group.

We determined which group had the highest probability of being the most effective in term of PFS, OS, ORR, CR, and safety (supplemental Figure 7A). The IMiD + mAB group achieved the highest probability of being the most effective in term of PFS, OS, and ORR, whereas the carfilzomib group reported the highest probability of achieving CR. The bortezomib + HDAC group had the highest probability of inducing a grade 3 to 4 toxicity in MM patients. However, ranking the treatment solely on the basis of the probability of being the best does not account for the uncertainty in the relative treatment effects and may lead to overinterpretation of the results. Thus, we evaluated the probability of each group being at each rank (rank probabilities) and summarized them in a rankogram together with the median rank. We also evaluated the cumulative probability of each group being among the k ranks (k ranges from 1 to 9 in our analysis) together with the SUCRA.^22,23,43 

Figure 4A-B clearly shows that, regarding PFS, the IMiD + mAb group has the highest probability of being the most effective and that almost certainly it is among the 4 best treatments. It should be noted, however, that the bortezomib + mAb and carfilzomib groups are only slightly inferior to the best-ranked treatment and that the magnitude of this difference is very difficult to estimate.

In term of OS, the scenario is slightly different (Figure 4C-D). The IMiD + mAb group is again the highest ranked treatment and has a more than 90% probability of being among the 4 best treatments. In this case, however, the PI + IMiD group performed similarly, followed by the bortezomib + mAb group. Thus, there is considerable uncertainty regarding the ranking of the carfilzomib group, which has similar probabilities of being at each of the 9 ranks. We then estimated SUCRA values for PFS, OS, ORR, CR, and safety and calculated an average value to rank all the groups included in the analysis by using a multiparametric approach (supplemental Figure 7B). This analysis revealed IMiD + mAb, PI + IMiD, bortezomib + mAb, and carfilzomib as the best 4 options for RRMM patients, but we should take into account that the carfilzomib group, while achieving the highest score in term of CR, seems to be inferior to other groups in term of OS and toxicity. Moreover, when examining the rank probabilities (supplemental Figure 7C), the addition of a second drug to an IMiD backbone (IMiD + mAb and PI + IMiD together) accounts for an impressive 70% probability of being the best regimen in term of OS.

Finally, we investigated which regimen, among all regimens included in the NMA, scores as the overall best regimen. To find the answer, we determined the probability of being the best, cumulative rank probabilities, and SUCRA values for PFS, OS, ORR, CR, and safety and estimated an average value to rank all the treatment options included in the analysis (Figure 4E; supplemental Figure 8). According to average SUCRA values, the daratumumab + lenalidomide-dexamethasone regimen (within the IMiD + mAB group) achieved the highest score (average SUCRA, 0.7198) followed by carfilzomib + lenalidomide-dexamethasone (average SUCRA, 0.6334) and daratumumab + bortezomib-dexamethasone (average SUCRA, 0.6154). These results were further confirmed by the fact that the daratumumab + lenalidomide-dexamethasone regimen reported the highest probability of being the best regimen in terms of PFS (44.56%), OS (27.57%), and ORR (22.26%), with a very low probability of being the worst regimen in terms of grade 3 to 4 toxicities (3.70%) (supplemental Figure 8A). Again, it is relevant that 4 of the 5 best regimens in our ranking include the addition of a second drug to an IMiD backbone, and that 3 of the 5 best regimens include the new anti-MM mAbs daratumumab and elotuzumab.

No significant inconsistency or loop-specific heterogeneity were found in our NMA as shown by the inconsistency factor plots in supplemental Figure 9A (based on the 95% CI of the logRoR crossing the null value of 0).

For exploratory purposes, we performed the same NMA by using a frequentist approach. All regimens were ranked by using the P-score according to the method recently described by Rücker et al.20 We observed similar results in terms of all variables (supplemental Figure 9B) and the daratumumab + lenalidomide-dexamethasone regimen was again found to be the highest scoring regimen (average P-score, 0.733).

The aim of our study was to systematically review and compare all therapeutic options available for RRMM, taking into account the high activity of regimens that included novel agents such as HDAC inhibitors or mAbs. Indeed, although most of these agents have undergone randomized clinical trials, a clear understanding of their relative efficacy is still lacking, and in the absence of predictive biomarkers, treatment choice is exclusively based on patient characteristics (prior treatment, absolute refractoriness of the disease, comorbidities, and the patient’s ability to tolerate the regimen) and tumor-specific characteristics (disease burden, risk status, cytogenetics, and score according to the International Scoring System [ISS] and Revised ISS) characteristics. Thus, ranking the value of all these opportunities might be of help in designing therapeutic algorithms to be validated in prospective clinical trials.

In this study, we performed 2 traditional pairwise meta-analyses and an NMA. Indeed, although pairwise meta-analysis can offer a robust tool for evaluating the strength of evidence derived from direct comparative studies, Bayesian NMA seems to be the best tool for investigating the strength of evidence for regimens that have not undergone direct comparison.

The pairwise meta-analyses showed that a 3-drug combination that includes bortezomib or lenalidomide reduces the risk of disease progression and increases the probability of achieving at least a partial response, even at the cost of greater toxicity compared with doublet therapy. However, only IMiD-based regimens plus mAbs or PIs demonstrated a clear advantage in terms of OS. To overcome the limitation of the lack of direct comparisons, we used the NMA approach and grouped all available regimens into 9 subgroups. In our analysis, the IMiD + mAb group achieved the highest probability of being the most effective regimen for RRMM in term of PFS followed by the bortezomib + mAb and the carfilzomib group with very close SUCRA scores; OS followed closely with the PI + IMiD and bortezomib + mAb groups. When considering specific regimens, we observed that the daratumumab + lenalidomide-dexamethasone regimen reached the highest SUCRA values in term of PFS, OS, and ORR with a good safety profile, ranking as the average best treatment. We also observed that a regimen that adds a second drug to an IMiD backbone has a 70% probability of being the best treatment option and that, among the 5 regimens (of 16 evaluated) scoring the highest average SUCRA, 4 included an IMiD backbone.

Thus, the results of our pairwise comparison indicate that whenever possible, a 3-drug regimen must be preferred over a 2-drug regimen, and our NMA further suggests that this regimen should have an IMiD backbone (preferentially combined with a new anti-MM mAb or alternatively with a PI). Furthermore, we observed that the combination of an HDAC inhibitor or pegylated liposomal doxorubicin with bortezomib significantly increases the toxicity of the treatment without adding additional benefit compared with regimens based on PIs + IMiDs, carfilzomib, or mAbs. It is clear that these regimens should be reserved for selected very fit MM patients. Furthermore, a correlation between activity and efficacy outcomes was not clearly demonstrated. Although the highest CR rates were attained with carfilzomib or PI-based 3-drug regimens, the 4 regimens that included elotuzumab and daratumumab ranked highest in terms of OS. These data are consistent with clinical outcomes observed in other tumors undergoing immunotherapy-based regimens^44-48  and are supported by recent experimental findings on MM pathogenesis. Indeed, the encouraging results of immunotherapy are, to a degree, not surprising when taking into account that MM is a malignancy strongly dependent on microenvironment features, including inflammation and immunosuppression events.^49-51  In such a scenario, it is likely that new immunotherapeutic agents will quickly become available for RRMM patients. Immune checkpoint inhibitors such as programmed cell death protein 1 and programmed death-ligand 1 blocking antibodies have shown promising preliminary results in small phase 1/2 trials that demonstrate the benefit of IMiDs in combination with pembrolizumab; several phase 2/3 trials with agents of this class are currently ongoing.^52,53 

Unfortunately, these novel drugs are costly, thus leading to sustainability concerns for health care systems. It is therefore imperative to prioritize treatment schedules that have demonstrated improvement in survival and/or quality of life and to develop and implement new methodologies of comparative analysis such as NMA for constructing decision-making therapeutic algorithms. This information might benefit health technology assessment by providing a novel benchmark (NMA) ranking for cost-effectiveness. One of the major strengths of our analysis is the simultaneous comparison of the results from all published trials, thus achieving information regarding hierarchy even in the absence of direct comparisons.

However, our work has limitations. First, data were retrieved from published studies rather than from individual patients’ records. Second, and most important, potential biases can be produced by the heterogeneity of the agents and patient populations included in the analysis. An optimal meta-analysis in this setting should compare studies that are similar in terms of patient characteristics (such as number of prior regimens, percentages of IMiD and PI refractoriness, age, MM risks) and treated with the same regimens in the same treatment line, which cannot be done with current evidence. In addition, no information is provided on the best sequencing of the combinations that were investigated. Finally, this work is intrinsically heuristic in nature and should therefore be considered a snapshot of current evidence, taking into account that some of these regimens are candidates for first-line therapy.

To the best of our knowledge, this is the first NMA on RRMM that includes all regimens currently evaluated in randomized trials in MM. Our findings suggest that a 3-drug regimen containing the lenalidomide-dexamethasone backbone, preferentially combined with anti-MM mAbs daratumumab or elotuzumab, has the highest probability of being ranked as the best treatment in this setting, underlying the role of immunotherapy in MM. Prospective randomized trials are eagerly awaited to clarify the optimal sequencing of treatments for MM.

This work was supported by the Italian Association for Cancer Research (AIRC) with “Special Program for Molecular Clinical Oncology–5 per mille” No. 9980, 2010/15 (Principal Investigator [PI]: P. Tassone); “5 per 1000 Molecular Clinical Oncology Extension Program” No. 9980, 2016/18 (PI: P. Tassone); and partially by “Innovative Immunotherapeutic Treatments of Human Cancer” Multi Unit Regional No. 16695 (cofinanced by AIRC and the CARICAL foundation), 2015/18 (PI: P. Tassone).

Contribution: C.B. and D.C. designed the research, conducted the literature search, performed data extraction, conducted the statistical analysis, interpreted the results, and wrote the manuscript; N.S. reviewed the analysis, assessed methodologic quality, and wrote the manuscript; M.C. and T.G. reviewed the analysis; M.R., P. Tagliaferri and P. Tassone reviewed the protocol, assessed methodologic quality, reviewed and edited the manuscript; and all authors approved the final version of the manuscript.

Conflict-of-interest disclosure: The authors declare no competing financial interests.

Correspondence: Pierfrancesco Tassone, Department of Experimental and Clinical Medicine, Magna Graecia University, Salvatore Venuta University Campus, Viale Europa, 88100 Catanzaro, Italy; e-mail: tassone@unicz.it.