Association of Measurable Residual Disease With Survival Outcomes in Patients With Acute Myeloid Leukemia: A Systematic Review and Meta-analysis | Hematology | JAMA Oncology | JAMA Network
[Skip to Navigation]
Sign In
Figure 1.  Flow Diagram of the Study Selection Process
Flow Diagram of the Study Selection Process

ALL indicates acute lymphoblastic leukemia; MRD, measurable residual disease.

Figure 2.  Estimated Survival Curves, Stratified by Measurable Residual Disease (MRD) Status
Estimated Survival Curves, Stratified by Measurable Residual Disease (MRD) Status

Overall survival (OS) (A) and disease-free survival (DFS) (B). The curves show the posterior means of survival distribution in the bayesian hierarchical analysis. The shadings of each curve show the 95% bayesian credible intervals (CrIs) for the survival proportion at the corresponding point in time of follow-up. The 5-year OS was 68% (95% CrI, 63%-73%) for the MRD-negative group and 34% (95% CrI, 28%-40%) for the MRD-positive group. The average hazard ratio for OS was 0.36 (95% CrI, 0.33-0.39), with a 5-year restricted mean survival time difference of 15.37 months (95% CrI, 13.58-17.19 months). The 5-year DFS was 64% (95% CrI, 59%-70%) for the MRD-negative group and 25% (95% CrI, 20%-32%) for the MRD-positive group. The average hazard ratio for DFS was 0.37 (95% CrI, 0.34-0.40), with a 5-year restricted mean survival time difference of 19.61 months (95% CrI, 17.33-21.92 months).

Figure 3.  Hazard Ratios (HRs) for Subgroups
Hazard Ratios (HRs) for Subgroups

Overall survival (A) and disease-free survival (B). Each square represents the mean HR from bayesian hierarchical analysis, and the horizontal lines represent the 95% bayesian credible interval (CrI) for the subgroup’s HR. AML indicates acute myeloid leukemia; CBF, core-binding factor; FISH, fluorescence in situ hybridization; MA, meta-analysis; MFC, multiparameter flow cytometry; MRD, measurable residual disease; NGS, next-generation sequencing; PCR, polymerase chain reaction; and WT1, Wilms tumor 1.

Table.  Included Studies by Subgroup
Included Studies by Subgroup
1.
Döhner  H, Weisdorf  DJ, Bloomfield  CD.  Acute myeloid leukemia.   N Engl J Med. 2015;373(12):1136-1152. doi:10.1056/NEJMra1406184 PubMedGoogle Scholar
2.
Short  NJ, Rytting  ME, Cortes  JE.  Acute myeloid leukaemia.   Lancet. 2018;392(10147):593-606. doi:10.1016/S0140-6736(18)31041-9 PubMedGoogle Scholar
3.
Papaemmanuil  E, Gerstung  M, Bullinger  L,  et al.  Genomic classification and prognosis in acute myeloid leukemia.   N Engl J Med. 2016;374(23):2209-2221. doi:10.1056/NEJMoa1516192 PubMedGoogle Scholar
4.
Grimwade  D, Hills  RK, Moorman  AV,  et al; National Cancer Research Institute Adult Leukaemia Working Group.  Refinement of cytogenetic classification in acute myeloid leukemia: determination of prognostic significance of rare recurring chromosomal abnormalities among 5876 younger adult patients treated in the United Kingdom Medical Research Council trials.   Blood. 2010;116(3):354-365. doi:10.1182/blood-2009-11-254441 PubMedGoogle Scholar
5.
Döhner  H, Estey  E, Grimwade  D,  et al.  Diagnosis and management of AML in adults: 2017 ELN recommendations from an international expert panel.   Blood. 2017;129(4):424-447. doi:10.1182/blood-2016-08-733196 PubMedGoogle Scholar
6.
Hourigan  CS, Karp  JE.  Minimal residual disease in acute myeloid leukaemia.   Nat Rev Clin Oncol. 2013;10(8):460-471. doi:10.1038/nrclinonc.2013.100 PubMedGoogle Scholar
7.
Schuurhuis  GJ, Heuser  M, Freeman  S,  et al.  Minimal/measurable residual disease in AML: a consensus document from the European LeukemiaNet MRD Working Party.   Blood. 2018;131(12):1275-1291. doi:10.1182/blood-2017-09-801498 PubMedGoogle Scholar
8.
Hochhaus  A, Larson  RA, Guilhot  F,  et al; IRIS Investigators.  Long-term outcomes of imatinib treatment for chronic myeloid leukemia.   N Engl J Med. 2017;376(10):917-927. doi:10.1056/NEJMoa1609324 PubMedGoogle Scholar
9.
Böttcher  S, Ritgen  M, Fischer  K,  et al.  Minimal residual disease quantification is an independent predictor of progression-free and overall survival in chronic lymphocytic leukemia: a multivariate analysis from the randomized GCLLSG CLL8 trial.   J Clin Oncol. 2012;30(9):980-988. doi:10.1200/JCO.2011.36.9348 PubMedGoogle Scholar
10.
Brüggemann  M, Raff  T, Flohr  T,  et al; German Multicenter Study Group for Adult Acute Lymphoblastic Leukemia.  Clinical significance of minimal residual disease quantification in adult patients with standard-risk acute lymphoblastic leukemia.   Blood. 2006;107(3):1116-1123. doi:10.1182/blood-2005-07-2708 PubMedGoogle Scholar
11.
Bassan  R, Spinelli  O, Oldani  E,  et al.  Improved risk classification for risk-specific therapy based on the molecular study of minimal residual disease (MRD) in adult acute lymphoblastic leukemia (ALL).   Blood. 2009;113(18):4153-4162. doi:10.1182/blood-2008-11-185132 PubMedGoogle Scholar
12.
Berry  DA, Zhou  S, Higley  H,  et al.  Association of minimal residual disease with clinical outcome in pediatric and adult acute lymphoblastic leukemia: a meta-analysis.   JAMA Oncol. 2017;3(7):e170580. doi:10.1001/jamaoncol.2017.0580 PubMedGoogle Scholar
13.
Pui  CH, Pei  D, Coustan-Smith  E,  et al.  Clinical utility of sequential minimal residual disease measurements in the context of risk-based therapy in childhood acute lymphoblastic leukaemia: a prospective study.   Lancet Oncol. 2015;16(4):465-474. doi:10.1016/S1470-2045(15)70082-3 PubMedGoogle Scholar
14.
Vora  A, Goulden  N, Wade  R,  et al.  Treatment reduction for children and young adults with low-risk acute lymphoblastic leukaemia defined by minimal residual disease (UKALL 2003): a randomised controlled trial.   Lancet Oncol. 2013;14(3):199-209. doi:10.1016/S1470-2045(12)70600-9 PubMedGoogle Scholar
15.
Buckley  SA, Wood  BL, Othus  M,  et al.  Minimal residual disease prior to allogeneic hematopoietic cell transplantation in acute myeloid leukemia: a meta-analysis.   Haematologica. 2017;102(5):865-873. doi:10.3324/haematol.2016.159343 PubMedGoogle Scholar
16.
Terwijn  M, van Putten  WL, Kelder  A,  et al.  High prognostic impact of flow cytometric minimal residual disease detection in acute myeloid leukemia: data from the HOVON/SAKK AML 42A study.   J Clin Oncol. 2013;31(31):3889-3897. doi:10.1200/JCO.2012.45.9628 PubMedGoogle Scholar
17.
Freeman  SD, Virgo  P, Couzens  S,  et al.  Prognostic relevance of treatment response measured by flow cytometric residual disease detection in older patients with acute myeloid leukemia.   J Clin Oncol. 2013;31(32):4123-4131. doi:10.1200/JCO.2013.49.1753 PubMedGoogle Scholar
18.
Ivey  A, Hills  RK, Simpson  MA,  et al; UK National Cancer Research Institute AML Working Group.  Assessment of minimal residual disease in standard-risk AML.   N Engl J Med. 2016;374(5):422-433. doi:10.1056/NEJMoa1507471 PubMedGoogle Scholar
19.
Ravandi  F, Jorgensen  J, Borthakur  G,  et al.  Persistence of minimal residual disease assessed by multiparameter flow cytometry is highly prognostic in younger patients with acute myeloid leukemia.   Cancer. 2017;123(3):426-435. doi:10.1002/cncr.30361 PubMedGoogle Scholar
20.
Balsat  M, Renneville  A, Thomas  X,  et al.  Postinduction minimal residual disease predicts outcome and benefit from allogeneic stem cell transplantation in acute myeloid leukemia with NPM1 mutation: a study by the Acute Leukemia French Association Group.   J Clin Oncol. 2017;35(2):185-193. doi:10.1200/JCO.2016.67.1875 PubMedGoogle Scholar
21.
Freeman  SD, Hills  RK, Virgo  P,  et al.  Measurable residual disease at induction redefines partial response in acute myeloid leukemia and stratifies outcomes in patients at standard risk without NPM1 mutations.   J Clin Oncol. 2018;36(15):1486-1497. doi:10.1200/JCO.2017.76.3425 PubMedGoogle Scholar
22.
Jongen-Lavrencic  M, Grob  T, Hanekamp  D,  et al.  Molecular minimal residual disease in acute myeloid leukemia.   N Engl J Med. 2018;378(13):1189-1199. doi:10.1056/NEJMoa1716863 PubMedGoogle Scholar
23.
Short  NJ, Ravandi  F.  How close are we to incorporating measurable residual disease into clinical practice for acute myeloid leukemia?   Haematologica. 2019;104(8):1532-1541. doi:10.3324/haematol.2018.208454 PubMedGoogle Scholar
24.
Moher  D, Liberati  A, Tetzlaff  J, Altman  DG; PRISMA Group.  Preferred Reporting Items for Systematic Reviews and Meta-analyses: the PRISMA statement.   PLoS Med. 2009;6(7):e1000097. doi:10.1371/journal.pmed.1000097 PubMedGoogle Scholar
25.
Parkin  B, Londoño-Joshi  A, Kang  Q, Tewari  M, Rhim  AD, Malek  SN.  Ultrasensitive mutation detection identifies rare residual cells causing acute myelogenous leukemia relapse.   J Clin Invest. 2017;127(9):3484-3495. doi:10.1172/JCI91964 PubMedGoogle Scholar
26.
Weber  S, Alpermann  T, Dicker  F,  et al.  BAALC expression: a suitable marker for prognostic risk stratification and detection of residual disease in cytogenetically normal acute myeloid leukemia.   Blood Cancer J. 2014;4:e173. doi:10.1038/bcj.2013.71 PubMedGoogle Scholar
27.
Weber  S, Haferlach  T, Alpermann  T,  et al.  Feasibility of BAALC gene expression for detection of minimal residual disease and risk stratification in normal karyotype acute myeloid leukaemia.   Br J Haematol. 2016;175(5):904-916. doi:10.1111/bjh.14343 PubMedGoogle Scholar
28.
Steinbach  D, Bader  P, Willasch  A,  et al.  Prospective validation of a new method of monitoring minimal residual disease in childhood acute myelogenous leukemia.   Clin Cancer Res. 2015;21(6):1353-1359. doi:10.1158/1078-0432.CCR-14-1999 PubMedGoogle Scholar
29.
Guyot  P, Ades  AE, Ouwens  MJNM, Welton  NJ.  Enhanced secondary analysis of survival data: reconstructing the data from published Kaplan-Meier survival curves.   BMC Med Res Methodol. 2012;12(1):9. doi:10.1186/1471-2288-12-9 PubMedGoogle Scholar
30.
Berry  DA.  Bayesian clinical trials.   Nat Rev Drug Discov. 2006;5(1):27-36. doi:10.1038/nrd1927 PubMedGoogle Scholar
31.
Gelman  A, Carlin  JB, Stern  HS, Dunson  DB, Vehtari  A, Rubin  DB.  Bayesian Data Analysis. 3rd ed. Text in Statistical Science series. CRC press; 2013. doi:10.1201/b16018
32.
Higgins  J, Thomas  J, Chandler  J,  et al Cochrane handbook for systematic reviews of interventions. Accessed September 1, 2019. http://www.training.cochrane.org/handbook
33.
Borenstein  M, Hedges  LV, Higgins  JP, Rothstein  HR.  Introduction to Meta-analysis. John Wiley & Sons; 2011.
34.
Fu  C, Zhou  S, Berry  DA. Evidence synthesis with reconstructed survival data. Pennsylvania State University Technical Report; 2020.
35.
Royston  P, Parmar  MK.  Restricted mean survival time: an alternative to the hazard ratio for the design and analysis of randomized trials with a time-to-event outcome.   BMC Med Res Methodol. 2013;13:152. doi:10.1186/1471-2288-13-152 PubMedGoogle Scholar
36.
Uno  H, Claggett  B, Tian  L,  et al.  Moving beyond the hazard ratio in quantifying the between-group difference in survival analysis.   J Clin Oncol. 2014;32(22):2380-2385. doi:10.1200/JCO.2014.55.2208 PubMedGoogle Scholar
37.
Trinquart  L, Jacot  J, Conner  SC, Porcher  R.  Comparison of treatment effects measured by the hazard ratio and by the ratio of restricted mean survival times in oncology randomized controlled trials.   J Clin Oncol. 2016;34(15):1813-1819. doi:10.1200/JCO.2015.64.2488 PubMedGoogle Scholar
38.
San Miguel  JF, Vidriales  MB, López-Berges  C,  et al.  Early immunophenotypical evaluation of minimal residual disease in acute myeloid leukemia identifies different patient risk groups and may contribute to postinduction treatment stratification.   Blood. 2001;98(6):1746-1751. doi:10.1182/blood.V98.6.1746 PubMedGoogle Scholar
39.
Sievers  EL, Lange  BJ, Alonzo  TA,  et al.  Immunophenotypic evidence of leukemia after induction therapy predicts relapse: results from a prospective Children’s Cancer Group study of 252 patients with acute myeloid leukemia.   Blood. 2003;101(9):3398-3406. doi:10.1182/blood-2002-10-3064 PubMedGoogle Scholar
40.
Coustan-Smith  E, Ribeiro  RC, Rubnitz  JE,  et al.  Clinical significance of residual disease during treatment in childhood acute myeloid leukaemia.   Br J Haematol. 2003;123(2):243-252. doi:10.1046/j.1365-2141.2003.04610.x PubMedGoogle Scholar
41.
Feller  N, van der Pol  MA, van Stijn  A,  et al.  MRD parameters using immunophenotypic detection methods are highly reliable in predicting survival in acute myeloid leukaemia.   Leukemia. 2004;18(8):1380-1390. doi:10.1038/sj.leu.2403405 PubMedGoogle Scholar
42.
Weisser  M, Kern  W, Rauhut  S,  et al.  Prognostic impact of RT-PCR-based quantification of WT1 gene expression during MRD monitoring of acute myeloid leukemia.   Leukemia. 2005;19(8):1416-1423. doi:10.1038/sj.leu.2403809 PubMedGoogle Scholar
43.
Weisser  M, Kern  W, Schoch  C, Hiddemann  W, Haferlach  T, Schnittger  S.  Risk assessment by monitoring expression levels of partial tandem duplications in the MLL gene in acute myeloid leukemia during therapy.   Haematologica. 2005;90(7):881-889.PubMedGoogle Scholar
44.
Perea  G, Lasa  A, Aventín  A,  et al; Grupo Cooperativo para el Estudio y Tratamiento de las Leucemias Agudas y Miel.  Prognostic value of minimal residual disease (MRD) in acute myeloid leukemia (AML) with favorable cytogenetics [t(8;21) and inv(16)].   Leukemia. 2006;20(1):87-94. doi:10.1038/sj.leu.2404015 PubMedGoogle Scholar
45.
Lapillonne  H, Renneville  A, Auvrignon  A,  et al.  High WT1 expression after induction therapy predicts high risk of relapse and death in pediatric acute myeloid leukemia.   J Clin Oncol. 2006;24(10):1507-1515. doi:10.1200/JCO.2005.03.5303 PubMedGoogle Scholar
46.
Laane  E, Derolf  AR, Björklund  E,  et al.  The effect of allogeneic stem cell transplantation on outcome in younger acute myeloid leukemia patients with minimal residual disease detected by flow cytometry at the end of post-remission chemotherapy.   Haematologica. 2006;91(6):833-836.PubMedGoogle Scholar
47.
Langebrake  C, Creutzig  U, Dworzak  M,  et al; MRD-AML-BFM Study Group.  Residual disease monitoring in childhood acute myeloid leukemia by multiparameter flow cytometry: the MRD-AML-BFM Study Group.   J Clin Oncol. 2006;24(22):3686-3692. doi:10.1200/JCO.2005.05.4312 PubMedGoogle Scholar
48.
Maurillo  L, Buccisano  F, Spagnoli  A,  et al.  Monitoring of minimal residual disease in adult acute myeloid leukemia using peripheral blood as an alternative source to bone marrow.   Haematologica. 2007;92(5):605-611. doi:10.3324/haematol.10432 PubMedGoogle Scholar
49.
Narimatsu  H, Iino  M, Ichihashi  T,  et al.  Clinical significance of minimal residual disease in patients with t(8;21) acute myeloid leukemia in Japan.   Int J Hematol. 2008;88(2):154-158. doi:10.1007/s12185-008-0108-1 PubMedGoogle Scholar
50.
Al-Mawali  A, Gillis  D, Lewis  I.  The use of receiver operating characteristic analysis for detection of minimal residual disease using five-color multiparameter flow cytometry in acute myeloid leukemia identifies patients with high risk of relapse.   Cytometry B Clin Cytom. 2009;76(2):91-101. doi:10.1002/cyto.b.20444 PubMedGoogle Scholar
51.
Hess  CJ, Feller  N, Denkers  F,  et al.  Correlation of minimal residual disease cell frequency with molecular genotype in patients with acute myeloid leukemia.   Haematologica. 2009;94(1):46-53. doi:10.3324/haematol.13110 PubMedGoogle Scholar
52.
Cilloni  D, Renneville  A, Hermitte  F,  et al.  Real-time quantitative polymerase chain reaction detection of minimal residual disease by standardized WT1 assay to enhance risk stratification in acute myeloid leukemia: a European LeukemiaNet study.   J Clin Oncol. 2009;27(31):5195-5201. doi:10.1200/JCO.2009.22.4865 PubMedGoogle Scholar
53.
Guièze  R, Renneville  A, Cayuela  JM,  et al.  Prognostic value of minimal residual disease by real-time quantitative PCR in acute myeloid leukemia with CBFB-MYH11 rearrangement: the French experience.   Leukemia. 2010;24(7):1386-1388. doi:10.1038/leu.2010.112 PubMedGoogle Scholar
54.
Buccisano  F, Maurillo  L, Spagnoli  A,  et al.  Cytogenetic and molecular diagnostic characterization combined to postconsolidation minimal residual disease assessment by flow cytometry improves risk stratification in adult acute myeloid leukemia.   Blood. 2010;116(13):2295-2303. doi:10.1182/blood-2009-12-258178 PubMedGoogle Scholar
55.
Corbacioglu  A, Scholl  C, Schlenk  RF,  et al.  Prognostic impact of minimal residual disease in CBFB-MYH11-positive acute myeloid leukemia.   J Clin Oncol. 2010;28(23):3724-3729. doi:10.1200/JCO.2010.28.6468 PubMedGoogle Scholar
56.
van der Velden  VH, van der Sluijs-Geling  A, Gibson  BE,  et al.  Clinical significance of flow cytometric minimal residual disease detection in pediatric acute myeloid leukemia patients treated according to the DCOG ANLL97/MRC AML12 protocol.   Leukemia. 2010;24(9):1599-1606. doi:10.1038/leu.2010.153 PubMedGoogle Scholar
57.
Chou  WC, Hou  HA, Liu  CY,  et al.  Sensitive measurement of quantity dynamics of FLT3 internal tandem duplication at early time points provides prognostic information.   Ann Oncol. 2011;22(3):696-704. doi:10.1093/annonc/mdq402 PubMedGoogle Scholar
58.
Inoue  D, Maruoka  H, Takahashi  T.  Clinical analysis and optimization of postremission therapy for acute myeloid leukemia patients with minimal residual disease as determined by flow cytometry.   Mediterr J Hematol Infect Dis. 2010;2(2):e2010020. doi:10.4084/mjhid.2010.020 PubMedGoogle Scholar
59.
Krönke  J, Schlenk  RF, Jensen  KO,  et al.  Monitoring of minimal residual disease in NPM1-mutated acute myeloid leukemia: a study from the German-Austrian acute myeloid leukemia study group.   J Clin Oncol. 2011;29(19):2709-2716. doi:10.1200/JCO.2011.35.0371 PubMedGoogle Scholar
60.
Chen  Y, Cortes  J, Estrov  Z,  et al.  Persistence of cytogenetic abnormalities at complete remission after induction in patients with acute myeloid leukemia: prognostic significance and the potential role of allogeneic stem-cell transplantation.   J Clin Oncol. 2011;29(18):2507-2513. doi:10.1200/JCO.2010.34.2873 PubMedGoogle Scholar
61.
Gray  JX, McMillen  L, Mollee  P,  et al.  WT1 expression as a marker of minimal residual disease predicts outcome in acute myeloid leukemia when measured post-consolidation.   Leuk Res. 2012;36(4):453-458. doi:10.1016/j.leukres.2011.09.005 PubMedGoogle Scholar
62.
Terwijn  M, Kelder  A, Snel  AN,  et al.  Minimal residual disease detection defined as the malignant fraction of the total primitive stem cell compartment offers additional prognostic information in acute myeloid leukaemia.   Int J Lab Hematol. 2012;34(4):432-441. doi:10.1111/j.1751-553X.2012.01416.x PubMedGoogle Scholar
63.
Loken  MR, Alonzo  TA, Pardo  L,  et al.  Residual disease detected by multidimensional flow cytometry signifies high relapse risk in patients with de novo acute myeloid leukemia: a report from Children’s Oncology Group.   Blood. 2012;120(8):1581-1588. doi:10.1182/blood-2012-02-408336 PubMedGoogle Scholar
64.
Yin  JA, O’Brien  MA, Hills  RK, Daly  SB, Wheatley  K, Burnett  AK.  Minimal residual disease monitoring by quantitative RT-PCR in core binding factor AML allows risk stratification and predicts relapse: results of the United Kingdom MRC AML-15 trial.   Blood. 2012;120(14):2826-2835. doi:10.1182/blood-2012-06-435669 PubMedGoogle Scholar
65.
Inaba  H, Coustan-Smith  E, Cao  X,  et al.  Comparative analysis of different approaches to measure treatment response in acute myeloid leukemia.   J Clin Oncol. 2012;30(29):3625-3632. doi:10.1200/JCO.2011.41.5323 PubMedGoogle Scholar
66.
Xu  XJ, Feng  JH, Tang  YM,  et al.  Prognostic significance of flow cytometric minimal residual disease assessment after the first induction course in Chinese childhood acute myeloid leukemia.   Leuk Res. 2013;37(2):134-138. doi:10.1016/j.leukres.2012.11.002 PubMedGoogle Scholar
67.
Jourdan  E, Boissel  N, Chevret  S,  et al; French AML Intergroup.  Prospective evaluation of gene mutations and minimal residual disease in patients with core binding factor acute myeloid leukemia.   Blood. 2013;121(12):2213-2223. doi:10.1182/blood-2012-10-462879 PubMedGoogle Scholar
68.
Zhang  L, Li  Q, Li  W,  et al.  Monitoring of minimal residual disease in acute myeloid leukemia with t(8;21)(q22;q22).   Int J Hematol. 2013;97(6):786-792. doi:10.1007/s12185-013-1344-6 PubMedGoogle Scholar
69.
Hoyos  M, Nomdedeu  JF, Esteve  J,  et al.  Core binding factor acute myeloid leukemia: the impact of age, leukocyte count, molecular findings, and minimal residual disease.   Eur J Haematol. 2013;91(3):209-218. doi:10.1111/ejh.12130 PubMedGoogle Scholar
70.
Shayegi  N, Kramer  M, Bornhäuser  M,  et al; Study Alliance Leukemia (SAL).  The level of residual disease based on mutant NPM1 is an independent prognostic factor for relapse and survival in AML.   Blood. 2013;122(1):83-92. doi:10.1182/blood-2012-10-461749 PubMedGoogle Scholar
71.
Marani  C, Clavio  M, Grasso  R,  et al.  Integrating post induction WT1 quantification and flow-cytometry results improves minimal residual disease stratification in acute myeloid leukemia.   Leuk Res. 2013;37(12):1606-1611. doi:10.1016/j.leukres.2013.07.005 PubMedGoogle Scholar
72.
Rossi  G, Minervini  MM, Melillo  L,  et al.  Predictive role of minimal residual disease and log clearance in acute myeloid leukemia: a comparison between multiparameter flow cytometry and Wilm’s tumor 1 levels.   Ann Hematol. 2014;93(7):1149-1157. doi:10.1007/s00277-014-2029-9 PubMedGoogle Scholar
73.
Hubmann  M, Köhnke  T, Hoster  E,  et al.  Molecular response assessment by quantitative real-time polymerase chain reaction after induction therapy in NPM1-mutated patients identifies those at high risk of relapse.   Haematologica. 2014;99(8):1317-1325. doi:10.3324/haematol.2014.104133 PubMedGoogle Scholar
74.
Wang  L, Gao  L, Xu  S,  et al.  High prognostic value of minimal residual disease detected by flow-cytometry-enhanced fluorescence in situ hybridization in core-binding factor acute myeloid leukemia (CBF-AML).   Ann Hematol. 2014;93(10):1685-1694. doi:10.1007/s00277-014-2107-z PubMedGoogle Scholar
75.
Köhnke  T, Sauter  D, Ringel  K,  et al.  Early assessment of minimal residual disease in AML by flow cytometry during aplasia identifies patients at increased risk of relapse.   Leukemia. 2015;29(2):377-386. doi:10.1038/leu.2014.186 PubMedGoogle Scholar
76.
Zhang  L, Cao  Z, Ruan  M,  et al.  Monitoring the AML1/ETO fusion transcript to predict outcome in childhood acute myeloid leukemia.   Pediatr Blood Cancer. 2014;61(10):1761-1766. doi:10.1002/pbc.25109 PubMedGoogle Scholar
77.
Hirsch  P, Labopin  M, Viguié  F,  et al.  Interest of cytogenetic and FISH evaluation for prognosis evaluation in 198 patients with acute myeloid leukemia in first complete remission in a single institution.   Leuk Res. 2014;38(8):907-912. doi:10.1016/j.leukres.2014.05.021 PubMedGoogle Scholar
78.
Lambert  J, Lambert  J, Nibourel  O,  et al.  MRD assessed by WT1 and NPM1 transcript levels identifies distinct outcomes in AML patients and is influenced by gemtuzumab ozogamicin.   Oncotarget. 2014;5(15):6280-6288. doi:10.18632/oncotarget.2196 PubMedGoogle Scholar
79.
Yoon  JH, Kim  HJ, Kim  JW,  et al.  Identification of molecular and cytogenetic risk factors for unfavorable core-binding factor-positive adult AML with post-remission treatment outcome analysis including transplantation.   Bone Marrow Transplant. 2014;49(12):1466-1474. doi:10.1038/bmt.2014.180 PubMedGoogle Scholar
80.
Terwijn  M, Zeijlemaker  W, Kelder  A,  et al.  Leukemic stem cell frequency: a strong biomarker for clinical outcome in acute myeloid leukemia.   PLoS One. 2014;9(9):e107587. doi:10.1371/journal.pone.0107587 PubMedGoogle Scholar
81.
Pigazzi  M, Manara  E, Buldini  B,  et al.  Minimal residual disease monitored after induction therapy by RQ-PCR can contribute to tailor treatment of patients with t(8;21) RUNX1-RUNX1T1 rearrangement.   Haematologica. 2015;100(3):e99-e101. doi:10.3324/haematol.2014.114579 PubMedGoogle Scholar
82.
Chen  X, Xie  H, Wood  BL,  et al.  Relation of clinical response and minimal residual disease and their prognostic impact on outcome in acute myeloid leukemia.   J Clin Oncol. 2015;33(11):1258-1264. doi:10.1200/JCO.2014.58.3518 PubMedGoogle Scholar
83.
Buccisano  F, Maurillo  L, Piciocchi  A,  et al.  Minimal residual disease negativity in elderly patients with acute myeloid leukemia may indicate different postremission strategies than in younger patients.   Ann Hematol. 2015;94(8):1319-1326. doi:10.1007/s00277-015-2364-5 PubMedGoogle Scholar
84.
Shibasaki  Y, Seki  Y, Tanaka  T,  et al.  The association of level of reduction of Wilms’ tumor gene 1 mRNA transcript in bone marrow and outcome in acute myeloid leukemia patients.   Leuk Res. 2015;39(6):667-671. doi:10.1016/j.leukres.2015.03.021 PubMedGoogle Scholar
85.
Klco  JM, Miller  CA, Griffith  M,  et al.  Association between mutation clearance after induction therapy and outcomes in acute myeloid leukemia.   JAMA. 2015;314(8):811-822. doi:10.1001/jama.2015.9643 PubMedGoogle Scholar
86.
Zeijlemaker  W, Kelder  A, Oussoren-Brockhoff  YJ,  et al.  Peripheral blood minimal residual disease may replace bone marrow minimal residual disease as an immunophenotypic biomarker for impending relapse in acute myeloid leukemia.   Leukemia. 2016;30(3):708-715. doi:10.1038/leu.2015.255 PubMedGoogle Scholar
87.
Vidriales  MB, Pérez-López  E, Pegenaute  C,  et al; PETHEMA Programa para el Estudio de la Terapéutica en Hemopatías Malignas Cooperative Study Group.  Minimal residual disease evaluation by flow cytometry is a complementary tool to cytogenetics for treatment decisions in acute myeloid leukaemia.   Leuk Res. 2016;40:1-9. doi:10.1016/j.leukres.2015.10.002 PubMedGoogle Scholar
88.
Willekens  C, Blanchet  O, Renneville  A,  et al; French AML Intergroup.  Prospective long-term minimal residual disease monitoring using RQ-PCR in RUNX1-RUNX1T1-positive acute myeloid leukemia: results of the French CBF-2006 trial.   Haematologica. 2016;101(3):328-335. doi:10.3324/haematol.2015.131946 PubMedGoogle Scholar
89.
Malagola  M, Skert  C, Borlenghi  E,  et al.  Postremission sequential monitoring of minimal residual disease by WT1 Q-PCR and multiparametric flow cytometry assessment predicts relapse and may help to address risk-adapted therapy in acute myeloid leukemia patients.   Cancer Med. 2016;5(2):265-274. doi:10.1002/cam4.593 PubMedGoogle Scholar
90.
Keino  D, Kinoshita  A, Tomizawa  D,  et al.  Residual disease detected by multidimensional flow cytometry shows prognostic significance in childhood acute myeloid leukemia with intermediate cytogenetics and negative FLT3-ITD: a report from the Tokyo Children’s Cancer Study Group.   Int J Hematol. 2016;103(4):416-422. doi:10.1007/s12185-016-1937-y PubMedGoogle Scholar
91.
Kim  Y, Lee  GD, Park  J,  et al.  Quantitative fragment analysis of FLT3-ITD efficiently identifying poor prognostic group with high mutant allele burden or long ITD length.   Blood Cancer J. 2015;5:e336. doi:10.1038/bcj.2015.61 PubMedGoogle Scholar
92.
Tierens  A, Bjørklund  E, Siitonen  S,  et al.  Residual disease detected by flow cytometry is an independent predictor of survival in childhood acute myeloid leukaemia; results of the NOPHO-AML 2004 study.   Br J Haematol. 2016;174(4):600-609. doi:10.1111/bjh.14093 PubMedGoogle Scholar
93.
Othus  M, Wood  BL, Stirewalt  DL,  et al.  Effect of measurable (“minimal”) residual disease (MRD) information on prediction of relapse and survival in adult acute myeloid leukemia.   Leukemia. 2016;30(10):2080-2083. doi:10.1038/leu.2016.120 PubMedGoogle Scholar
94.
Manara  E, Basso  G, Zampini  M,  et al.  Characterization of children with FLT3-ITD acute myeloid leukemia: a report from the AIEOP AML-2002 study group.   Leukemia. 2017;31(1):18-25. doi:10.1038/leu.2016.177 PubMedGoogle Scholar
95.
Huang  S, Yang  H, Li  Y,  et al.  Prognostic significance of mixed-lineage leukemia (MLL) gene detected by real-time fluorescence quantitative PCR assay in acute myeloid leukemia.   Med Sci Monit. 2016;22:3009-3017. doi:10.12659/MSM.900429 PubMedGoogle Scholar
96.
Buldini  B, Rizzati  F, Masetti  R,  et al.  Prognostic significance of flow-cytometry evaluation of minimal residual disease in children with acute myeloid leukaemia treated according to the AIEOP-AML 2002/01 study protocol.   Br J Haematol. 2017;177(1):116-126. doi:10.1111/bjh.14523 PubMedGoogle Scholar
97.
Martínez-Laperche  C, Kwon  M, Franco-Villegas  AC,  et al; Cooperative Group for Molecular Biology in Haematology (GBMH).  Wilms tumor 1 gene expression levels improve risk stratification in AML patients: results of a multicentre study within the Spanish Group for Molecular Biology in Haematology.   Br J Haematol. 2018;181(4):542-546. doi:10.1111/bjh.14635 PubMedGoogle Scholar
98.
Daver  N, Kantarjian  H, Garcia-Manero  G,  et al.  Vosaroxin in combination with decitabine in newly diagnosed older patients with acute myeloid leukemia or high-risk myelodysplastic syndrome.   Haematologica. 2017;102(10):1709-1717. doi:10.3324/haematol.2017.168732 PubMedGoogle Scholar
99.
Frairia  C, Aydin  S, Audisio  E,  et al.  Post-remissional and pre-transplant role of minimal residual disease detected by WT1 in acute myeloid leukemia: a retrospective cohort study.   Leuk Res. 2017;61:10-17. doi:10.1016/j.leukres.2017.08.008 PubMedGoogle Scholar
100.
Boddu  P, Jorgensen  J, Kantarjian  H,  et al.  Achievement of a negative minimal residual disease state after hypomethylating agent therapy in older patients with AML reduces the risk of relapse.   Leukemia. 2018;32(1):241-244. doi:10.1038/leu.2017.285 PubMedGoogle Scholar
101.
Lacombe  F, Campos  L, Allou  K,  et al; Groupe d’Etude Immunologique des Leucémies (GEIL).  Prognostic value of multicenter flow cytometry harmonized assessment of minimal residual disease in acute myeloblastic leukemia.   Hematol Oncol. 2018;36(2):422-428. doi:10.1002/hon.2488 PubMedGoogle Scholar
102.
Ferret  Y, Boissel  N, Helevaut  N,  et al.  Clinical relevance of IDH1/2 mutant allele burden during follow-up in acute myeloid leukemia: a study by the French ALFA group.   Haematologica. 2018;103(5):822-829. doi:10.3324/haematol.2017.183525 PubMedGoogle Scholar
103.
Höllein  A, Jeromin  S, Meggendorfer  M,  et al.  Minimal residual disease (MRD) monitoring and mutational landscape in AML with RUNX1-RUNX1T1: a study on 134 patients.   Leukemia. 2018;32(10):2270-2274. doi:10.1038/s41375-018-0086-0 PubMedGoogle Scholar
104.
Morita  K, Kantarjian  HM, Wang  F,  et al.  Clearance of somatic mutations at remission and the risk of relapse in acute myeloid leukemia.   J Clin Oncol. 2018;36(18):1788-1797. doi:10.1200/JCO.2017.77.6757 PubMedGoogle Scholar
105.
Boddu  P, Gurguis  C, Sanford  D,  et al.  Response kinetics and factors predicting survival in core-binding factor leukemia.   Leukemia. 2018;32(12):2698-2701. doi:10.1038/s41375-018-0158-1 PubMedGoogle Scholar
106.
Onecha  E, Linares  M, Rapado  I,  et al.  A novel deep targeted sequencing method for minimal residual disease monitoring in acute myeloid leukemia.   Haematologica. 2019;104(2):288-296. doi:10.3324/haematol.2018.194712 PubMedGoogle Scholar
107.
Ok  CY, Loghavi  S, Sui  D,  et al.  Persistent IDH1/2 mutations in remission can predict relapse in patients with acute myeloid leukemia.   Haematologica. 2019;104(2):305-311. doi:10.3324/haematol.2018.191148 PubMedGoogle Scholar
108.
Zhu  HH, Zhang  XH, Qin  YZ,  et al.  MRD-directed risk stratification treatment may improve outcomes of t(8;21) AML in the first complete remission: results from the AML05 multicenter trial.   Blood. 2013;121(20):4056-4062. doi:10.1182/blood-2012-11-468348 PubMedGoogle Scholar
109.
Hourigan  CS, Dillon  LW, Gui  G,  et al.  Impact of conditioning intensity of allogeneic transplantation for acute myeloid leukemia with genomic evidence of residual disease.   J Clin Oncol. 2020;38(12):1273-1283. doi:10.1200/JCO.19.03011 PubMedGoogle Scholar
110.
Sockel  K, Wermke  M, Radke  J,  et al.  Minimal residual disease-directed preemptive treatment with azacitidine in patients with NPM1-mutant acute myeloid leukemia and molecular relapse.   Haematologica. 2011;96(10):1568-1570. doi:10.3324/haematol.2011.044388 PubMedGoogle Scholar
111.
Platzbecker  U, Wermke  M, Radke  J,  et al.  Azacitidine for treatment of imminent relapse in MDS or AML patients after allogeneic HSCT: results of the RELAZA trial.   Leukemia. 2012;26(3):381-389. doi:10.1038/leu.2011.234 PubMedGoogle Scholar
112.
Platzbecker  U, Middeke  JM, Sockel  K,  et al.  Measurable residual disease-guided treatment with azacitidine to prevent haematological relapse in patients with myelodysplastic syndrome and acute myeloid leukaemia (RELAZA2): an open-label, multicentre, phase 2 trial.   Lancet Oncol. 2018;19(12):1668-1679. doi:10.1016/S1470-2045(18)30580-1 PubMedGoogle Scholar
113.
Ragon  BK, Daver  N, Garcia-Manero  G,  et al.  Minimal residual disease eradication with epigenetic therapy in core binding factor acute myeloid leukemia.   Am J Hematol. 2017;92(9):845-850. doi:10.1002/ajh.24782 PubMedGoogle Scholar
114.
Gökbuget  N, Dombret  H, Bonifacio  M,  et al.  Blinatumomab for minimal residual disease in adults with B-cell precursor acute lymphoblastic leukemia.   Blood. 2018;131(14):1522-1531. doi:10.1182/blood-2017-08-798322 PubMedGoogle Scholar
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    Original Investigation
    October 8, 2020

    Association of Measurable Residual Disease With Survival Outcomes in Patients With Acute Myeloid Leukemia: A Systematic Review and Meta-analysis

    Author Affiliations
    • 1Department of Leukemia, The University of Texas MD Anderson Cancer Center, Houston
    • 2Department of Public Health Sciences, Penn State College of Medicine, Hershey, Pennsylvania
    • 3Department of Biostatistics, The University of Texas MD Anderson Cancer Center, Houston
    • 4Clinical Research Division, Fred Hutchinson Cancer Research Center, Seattle, Washington
    • 5Institute of Infection and Immunity, University of Birmingham, Birmingham, United Kingdom
    • 6Laboratory of Myeloid Malignancies, National Heart, Lung, and Blood Institute, National Institutes of Health, Bethesda, Maryland
    JAMA Oncol. 2020;6(12):1890-1899. doi:10.1001/jamaoncol.2020.4600
    Key Points

    Question  What is the association between measurable residual disease (MRD) and survival outcomes in patients with acute myeloid leukemia?

    Findings  In a systematic review and meta-analysis of 81 publications reporting on 11 151 patients with acute myeloid leukemia, the estimated 5-year disease-free survival was 64% for patients without MRD and 25% for those with MRD. The estimated overall survival was 68% for patients without MRD and 34% for those with MRD.

    Meaning  The findings of this study suggest that, in patients with acute myeloid leukemia, achievement of MRD negativity is associated with superior long-term survival and warrants consideration as a clinical trial end point that may allow for more rapid evaluation of the efficacy of novel therapies.

    Abstract

    Importance  Measurable residual disease (MRD) refers to neoplastic cells that cannot be detected by standard cytomorphologic analysis. In patients with acute myeloid leukemia (AML), determining the association of MRD with survival may improve prognostication and inform selection of efficient clinical trial end points.

    Objective  To examine the association between MRD status and disease-free survival (DFS) and overall survival (OS) in patients with AML using scientific literature.

    Data Sources  Clinical studies on AML published between January 1, 2000, and October 1, 2018, were identified via searches of PubMed, Embase, and MEDLINE.

    Study Selection  Literature search and study screening were performed according to the Preferred Reporting Items for Systematic Reviews and Meta-analyses guidelines. Studies that assessed DFS or OS by MRD status in patients with AML were included. Reviews, non–English-language articles, and studies reporting only outcomes after hematopoietic cell transplantation or those with insufficient description of MRD information were excluded.

    Data Extraction and Synthesis  Study sample size, median patient age, median follow-up time, MRD detection method, MRD assessment time points, AML subtype, specimen source, and survival outcomes were extracted. Meta-analyses were performed separately for DFS and OS using bayesian hierarchical modeling.

    Main Outcomes and Measures  Meta-analyses of survival probabilities and hazard ratios (HRs) were conducted for OS and DFS according to MRD status.

    Results  Eighty-one publications reporting on 11 151 patients were included. The average HR for achieving MRD negativity was 0.36 (95% bayesian credible interval [CrI], 0.33-0.39) for OS and 0.37 (95% CrI, 0.34-0.40) for DFS. The estimated 5-year DFS was 64% for patients without MRD and 25% for those with MRD, and the estimated OS was 68% for patients without MRD and 34% for those with MRD. The association of MRD negativity with DFS and OS was significant for all subgroups, with the exception of MRD assessed by cytogenetics or fluorescent in situ hybridization.

    Conclusions and Relevance  The findings of this meta-analysis suggest that achievement of MRD negativity is associated with superior DFS and OS in patients with AML. The value of MRD negativity appears to be consistent across age groups, AML subtypes, time of MRD assessment, specimen source, and MRD detection methods. These results support MRD status as an end point that may allow for accelerated evaluation of novel therapies in AML.

    Introduction

    Acute myeloid leukemia (AML) is a heterogeneous disease with a widely variable prognosis and likelihood of cure that is dependent on both patient- and disease-related characteristics.1,2Quiz Ref ID Pretreatment characteristics, such as age, karyotype, and genomic alterations, are well established factors associated with clinical outcomes of patients with newly diagnosed AML receiving first-line therapy.3,4 Assessment of early response to therapy by conventional morphologic analysis also provides important information about the chemosensitivity of leukemia in an individual that cannot necessarily be estimated by pretreatment factors.5 Further refinement of prognosis may be accomplished through evaluation of measurable residual disease (MRD), also called minimal residual disease, which refers to low levels of residual leukemia that cannot be detected by morphologic assessment alone.6,7

    The persistence of MRD has been shown to be associated with higher rates of relapse and worse disease-free survival (DFS) and overall survival (OS) in several leukemias, including chronic myeloid leukemia, chronic lymphocytic leukemia, and acute lymphoblastic leukemia (ALL).8-11 For example, a meta-analysis of 39 publications on children and adults with ALL reported that the presence of MRD confers worse event-free survival and OS regardless of the subgroups evaluated.12 In pediatric ALL, MRD information is also routinely used to risk stratify patients and determine appropriate consolidation strategies.13,14 A meta-analysis of 19 studies suggested that the presence of MRD before hematopoietic cell transplantation for AML was associated with higher rates of relapse and inferior survival.15 Individual studies have similarly suggested that the detection of MRD is associated with inferior outcomes in patients with AML undergoing frontline therapy. Based on the consistent association of MRD with clinical outcomes across multiple studies, consensus guidelines support the use of complete remission without MRD as an official AML response criterion.5

    Although many publications have suggested substantial clinical value of MRD assessment in AML, the optimal use of MRD information to risk stratify patients and inform clinical decision-making has been limited in part by the heterogeneity of these reports.16-22 Frequently used MRD detection methods include multiparameter flow cytometry (MFC), polymerase chain reaction (PCR) for recurrent gene fusions or other genomic alterations, and next-generation sequencing (NGS), among others. However, these methods vary in their sensitivities and applicability to different patient populations. Studies reporting the association of MRD with outcomes in AML also vary in the patient populations tested (eg, age or AML subtypes), timing of MRD assessment, and the specimen source (eg, bone marrow or peripheral blood). These variances all contribute to uncertainty about the broad use of MRD testing in clinical practice and its potential adoption as a clinical trial end point.23

    To use MRD information to guide clinical decision-making in AML and support its use as a meaningful clinical end point, it is necessary to understand the strength of the association of MRD with survival outcomes and the consistency of this association across patient-related, disease-related, and methodologic variables. To address these issues, we performed a literature-based meta-analysis of published AML studies reporting the association of MRD with DFS or OS. We also assessed the association between achieving MRD negativity and long-term clinical outcomes in relevant subgroups.

    Methods
    Data Sources and Selection

    For this systematic review and meta-analysis, 2 investigators (N.J.S. and F.R.) conducted independent searches of PubMed, MEDLINE, and Embase for articles published between January 1, 2000, and October 1, 2018, that included the key words AML, acute myeloid leukemia, or acute myelogenous leukemia in combination with the key words MRD, minimal residual disease, or measurable residual disease. Disease experts (R.B.W., S.D.F., and C.S.H.) reviewed the resultant list and provided recommendations for additional article inclusions. This project was approved by the University of Texas MD Anderson Cancer Center Institutional Review Board. This study followed the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) reporting guideline.24

    We excluded review articles, non–English-language articles, studies that included non-AML diseases in their analysis, studies that used overlapping data sets, and studies that had insufficient description of MRD information or lacked an association with DFS or OS. We also excluded studies that only reported outcomes after hematopoietic cell transplantation (Figure 1).

    Data Extraction and Synthesis

    We extracted the following information from publications when it was available: study sample size, median age and age range, median follow-up time, MRD detection method (eg, PCR, MFC, NGS, cytogenetics and/or fluorescence in situ hybridization [FISH], or others), MRD assessment time points, AML subtype (eg, core-binding factor [CBF] or others), specimen sources (eg, bone marrow or peripheral blood), and survival outcomes. Polymerase chain reaction–based studies were further subdivided into those evaluating expression of Wilms tumor 1 (WT1) and those evaluating specific gene targets or translocations. All studies used real-time quantitative PCR, with the exception of one study that used digital droplet PCR.25 Other methods of MRD detection included brain and acute leukemia cytoplasmic expression, and multigene expression.26-28

    We used a sequential approach to collect 3 different types of survival information comparing patients who achieved MRD negativity with those who did not: (1) if Kaplan-Meier curves were provided with the number of patients at risk and number of events for both MRD-negative and MRD-positive groups, we used commercial graph digitizer software (DigitizeIt, version 2.1; Bormisoft) to extract coordinates of points on the curves and applied a numeric algorithm to reconstruct survival data for each MRD group29; (2) when Kaplan-Meier curves were not available, we used reported hazard ratios (HRs) and their 95% CIs; and (3) for articles that only provided survival proportions at fixed time points (eg, 3 or 5 years), we used the reported survival rates and their SDs.

    Searches of PubMed and Embase/MEDLINE found 1571 and 1105 articles, respectively, for the date range of January 1, 2000, to October 1, 2018. After removing duplicate studies between the 2 searches, 199 articles remained. Additional criteria were applied to exclude studies with non-AML diseases in their analysis, those reporting only on patients with acute promyelocytic leukemia, those with overlapping data sets, those that only reported posttransplantation survival outcomes, and those that did not report on the association of MRD with survival end points, which reduced the total number to 79 publications. Two articles were subsequently added based on review by disease experts.

    To eliminate duplicate data, only one type of data was used from each study. With the highest priority, Kaplan-Meier curves for both MRD-negative and MRD-positive groups were extracted from 24 studies to reconstruct survival results for OS and from 16 studies for DFS. When Kaplan-Meier curves were unavailable for reconstruction, we used observed HRs and their CIs from 28 studies for OS and 38 studies for DFS. In 9 studies for OS and 12 studies for DFS, neither Kaplan-Meier curves nor HRs were reported, and thus survival rates at specific time points were used.

    Statistical Analysis

    The primary end points were DFS and OS, which were modeled separately. Overall survival was defined from the time of treatment start until death or last follow-up. Disease-free survival was defined as the time of remission until relapse, death, or last follow-up. We included disease-free, event-free, leukemia-free, recurrence-free, and relapse-free survival in the definition of DFS.

    We used 2 different approaches for statistical analysis, both of which allowed for the possibility that MRD status could have different HR effects in different studies. The primary analysis used a bayesian hierarchical model with a time-varying HR effect.30,31 The other approach was a traditional frequentist random-effects model assuming a constant proportional HR over time.32,33

    In the bayesian analysis, we modeled the time-varying effects by assuming that hazards were constant within each 6-month period of follow-up and truncated results of all studies at 11 years. Each 6-month segment has its own study-specific hazard rate and therefore also its own HR for MRD-negative vs MRD-positive groups.12 Studies that reported the observed HRs depending on MRD status and the associated CIs (type 2 data) contribute directly to the probability distributions of the log HRs of the studies considered and, in particular, to inferences about the mean of the studies. The baseline hazard rates within each time segment were estimated by the studies that reported Kaplan-Meier curves (type 1) and survival rates (type 3) for MRD-positive and MRD-negative groups.34 Log-normal distributions were proposed for segments in time-varying baseline hazard rates, the time-varying HRs, and the study-specific HR effects with noninformative previous distributions for the mean and SD parameters.

    We plotted the posterior means of DFS and OS distributions for patients who achieved MRD negativity vs those who did not. We used shading to show the 95% bayesian credible intervals (CrIs) of these Kaplan-Meier curves. The difference in restricted mean survival time (ie, the area under survival curves up to given time points)35-37 was added as a robust estimate of treatment effect. For forest plots, we present the HR average over time for MRD-negative vs MRD-positive groups and the corresponding 95% CrI. Unless indicated otherwise, all results reported, including subgroup analyses, were based on bayesian hierarchical analysis. The comparisons between different groups were conducted using posterior probabilities.

    Because closed forms of the full-conditional distributions are not available for bayesian analyses, we identified the joint posterior distributions of model parameters using Markov chain Monte Carlo methods.31 We used statistical software R, version 3.5.3 (R Project for Statistical Computing, with packages survival_v2.43-3, rjags_v4-8, coda_v0.19-3, lattice_v0.20-38, and ggplot2_v3.2.0) and JAGS, version 4.3.0 statistical software for data analysis.

    Results

    Eighty-one distinct publications including 11 151 patients were included in this analysis (17 studies for OS, 20 studies for DFS, and 44 for both outcomes). These studies formed the basis of our statistical analyses.16-22,25-28,38-107 The characteristics of the individual studies are presented in eTable 1 in the Supplement. The number of studies included in each subgroup analysis are reported in the Table.

    The survival curves for MRD-negative and MRD-positive groups are shown in Figure 2. Quiz Ref IDBoth OS and DFS were better for patients who achieved MRD negativity. At 5 years, the estimated OS was 68% (95% CrI, 63%-73%) for the MRD-negative group vs 34% (95% CrI, 28%-40%) for the MRD-positive group. Similarly, at 5 years, the estimated DFS was 64% (95% CrI, 59%-70%) for the MRD-negative group and 25% (95% CrI, 20%-32%) for the MRD-positive group. The estimation for time-varying HRs for both OS and DFS are shown in eFigure 1 in the Supplement. The relative benefit of achieving MRD negativity was comparable for both OS and DFS (average HR, 0.36; 95% CrI, 0.33-0.39 for OS and average HR, 0.37; 95% CrI, 0.34-0.40 for DFS). The average HR for each study is shown in forest plots (eFigure 2 in the Supplement). The difference of 5-year restricted mean survival time of the MRD-negative and MRD-positive groups was 15.37 months (95% CrI, 13.58-17.19 months) for OS and 19.61 months (95% CrI, 17.33-21.92 months) for DFS.

    Figure 3 presents the results of univariate analysis for different subgroups. In general, the MRD results showed a similar pattern for OS and DFS. Quiz Ref IDAchievement of MRD negativity was associated with superior DFS and OS regardless of age group (adult or pediatric), MRD assessment time point (induction, during consolidation, or after consolidation), AML subgroup (CBF or non-CBF), or specimen source (bone marrow or peripheral blood). While most of the MRD detection methods were able to identify a difference in DFS and OS between groups with MRD negativity vs positivity, the MRD association using cytogenetics/FISH was not significant (average HR, 0.77; 95% CrI, 0.39-1.56 for OS and average HR, 0.65; 95% CrI, 0.34-1.23 for DFS). Quiz Ref IDAmong studies evaluating MRD by MFC, the impact of MRD was similar between studies using less than 6-color assays vs greater than or equal to 6-color assays (difference in HR, −0.02; 95% CrI, −0.54 to 0.49 for OS and −0.09; 95% CrI, −0.70 to 0.52 for DFS).

    For AML subtypes, the association between MRD and survival outcomes was greater in studies reporting outcomes of CBF AML compared with non-CBF AML, with a posterior probability of 0.999 for OS and 0.997 for DFS. Regarding the association of specimen source with survival outcomes, peripheral blood assessment of MRD better distinguished MRD-positive and MRD-negative groups compared with bone marrow assessment of MRD, with a posterior probability of 0.918 for OS and 0.999 for DFS.

    Multivariate analysis results were consistent with univariate analysis results (eFigure 3 in the Supplement). All subgroups showed DFS and OS benefit to achievement of MRD negativity with the exception of MRD detection by cytogenetics/FISH. The differences of survival time by subgroup for 5-year survival rates and 5-year restricted mean survival times are summarized in eTable 2 in the Supplement.

    Discussion

    This meta-analysis of 81 publications suggests that MRD status has prognostic importance in AML and may be a valid surrogate marker for both DFS and OS in AML clinical trials. The magnitude of benefit associated with achieving MRD negativity was substantial, corresponding to a 64% reduction in the risk of death for MRD-negative patients. The results of this meta-analysis thus provide quantitative support for consensus guidelines that consider achievement of complete remission without MRD as the optimal response in AML.5

    The large number of studies considered in this meta-analysis allowed for relevant subgroup analyses. These subgroup analyses suggest that achievement of MRD negativity is associated with clinically relevant improvement in long-term survival outcomes in all subgroups evaluated, with the exception of MRD assessment by cytogenetics/FISH. The lack of significant association of cytogenetics/FISH with DFS and OS may be explained in part because only 2 studies using this method were included in this analysis (527 patients total) and because this MRD detection method has the lowest sensitivity among those considered.23 The consistent impact of MRD across all other subgroups analyzed provides evidence that achievement of MRD negativity is an important end point across clinical contexts, including the most relevant MRD detection methods used in contemporary clinical practice and research (eg, MFC, PCR, and NGS). Multivariate analysis further supported the benefit of MRD negativity across these subgroups. Nonetheless, the optimal MRD assay and timing of assessment should be guided by the specific clinical scenario. For example, while PCR for WT1 expression provided useful MRD information in our analysis, consensus recommendations advise against PCR for WT1 as a routine marker of MRD owing to its low sensitivity and specificity unless no other MRD test is available.7

    These data have several potential implications for both clinical practice and drug development in AML. Quiz Ref IDIn our analysis, the 5-year estimated OS for the MRD-negative group was 68% compared with only 34% for the MRD-positive group. In addition to supporting the use of MRD testing to provide prognostic information, the poor outcomes associated with the presence of MRD support the development of novel therapeutic approaches for these patients. Some studies suggested that the outcomes of MRD-positive patients may be improved with hematopoietic cell transplantation in first remission.20,21,54,108,109 The use of hypomethylating agents (eg, azacitidine or decitabine) may also be beneficial.110-113 In ALL, the CD3-CD19 bispecific T-cell–engaging antibody blinatumomab is effective in eradicating persistent or recurrent MRD and is approved by the US Food and Drug Administration for this use.114 Similarly, in AML, well-designed prospective clinical trials using effective MRD-directed therapies should test whether the outcomes of patients with MRD-positive disease could be improved with such an approach. The robust effect of MRD on both DFS and OS across studies also supports the consideration of MRD as a surrogate end point in clinical trial development that could lead to accelerated drug approval. An important caveat is that accelerated approval of any new drug based on an intermediate end point, such as MRD, would require eventual confirmation using traditional efficacy end points (eg, OS).

    Strengths and Limitations

    One strength of this analysis of pooled data from different publications is the advanced method used to generate the DFS and OS curves. Rather than converting survival information to study-level HR estimates, we reversely reconstructed survival data from published Kaplan-Meier curves to take advantage of the additional survival information, such that the baseline hazard function and time-varying HR can still be modeled without individual patient data. To use all of the available evidence, we applied 3 common types of survival information, including reconstructed survival data, HR estimates, and survival rates at particular points. A 1-stage bayesian hierarchical model to integrate different survival data was introduced to complete the information synthesis. Our model can assess the time-varying hazard rate for each treatment group, examine the proportional hazards assumption after controlling for the interstudy heterogeneity, and generate visual presentation of meta-analysis survival curves.

    The study has several limitations. Inherent in any such meta-analysis is the potential for publication bias in that researchers are less likely to publish negative results. Because our meta-analysis was based on pooled data rather than patient-level data, we also cannot assess the association of MRD with survival outcomes in subgroups not reported within the individual publications that we selected. For example, while studies often restricted their analyses to CBF AML or to non-CBF AML, this latter group is composed of heterogeneous cytogenetic and molecular subgroups in which MRD status could have variable effects on long-term outcomes. Although none of the included studies prospectively altered consolidation strategies based on MRD status, we cannot account for how MRD information may have been used by individual clinicians (eg, to inform the decision to pursue allogeneic hematopoietic cell transplantation), and such real-time use of MRD information to guide treatment decisions may have led to imbalances between the MRD-negative and MRD-positive groups. Furthermore, most publications on MRD in AML included in this meta-analysis evaluated the impact of MRD in the context of first-line intensive chemotherapy with a cytarabine- and anthracycline-based induction regimen. Thus, owing to the similarity of treatment across most of these studies, the generalizability of these findings to nonintensive regimens that do not use conventional cytotoxic chemotherapy is limited.

    Conclusions

    In this large-cohort meta-analysis, achievement of MRD negativity was associated with superior DFS and OS in patients with AML, an association that was observed across ages, disease subtypes, time of assessment, specimen source, and most MRD detection methods. Assessment of MRD in AML in cytomorphologic remission provides important prognostic information. Given the robustness of the association of MRD with long-term outcomes across studies, use of MRD status as an eligibility criterion and/or an end point in clinical trial design could lead to more efficient assessment of the efficacy of new drugs and combination therapies in AML.

    Back to top
    Article Information

    Accepted for Publication: July 20, 2020.

    Corresponding Author: Farhad Ravandi, MD, Department of Leukemia, Unit 428, The University of Texas MD Anderson Cancer Center, 1515 Holcombe Blvd, Houston, TX 77030 (fravandi@mdanderson.org).

    Published Online: October 8, 2020. doi:10.1001/jamaoncol.2020.4600

    Author Contributions: Dr Ravandi had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Drs Short and Zhou contributed equally to this work.

    Concept and design: Short, Zhou, Berry, Freeman, Hourigan, Kantarjian, Ravandi.

    Acquisition, analysis, or interpretation of data: Short, Zhou, Fu, Berry, Walter, Freeman, Huang, Nogueras Gonzalez, Hwang, Qi, Ravandi.

    Drafting of the manuscript: Short, Zhou, Fu, Huang, Hwang, Qi, Ravandi.

    Critical revision of the manuscript for important intellectual content: Short, Berry, Walter, Freeman, Hourigan, Nogueras Gonzalez, Kantarjian, Ravandi.

    Statistical analysis: Short, Zhou, Fu, Berry, Huang, Nogueras Gonzalez, Hwang, Qi, Kantarjian.

    Obtained funding: Ravandi.

    Administrative, technical, or material support: Ravandi.

    Supervision: Kantarjian, Ravandi.

    Conflict of Interest Disclosures: Dr Berry is co-owner of Berry Consultants LLC, a company that designs bayesian and adaptive clinical trials. Dr Freeman reported receiving personal fees from JAZZ outside the submitted work. Dr Hourigan reported receiving research funding from Sellas and Merck during the conduct of the study. No other disclosures were reported.

    Funding/Support: The study was supported by MD Anderson Cancer Center support grant CA016672 and Specialized Programs of Research Excellence. Dr Short is supported by the K12 Paul Calabresi Clinical Oncology Scholar Award and the American Society of Hematology Junior Faculty Scholar Award in Clinical Research. This study was supported in part by the Intramural Research Program of the National Heart, Lung, and Blood Institute of the National Institutes of Health.

    Role of the Funder/Sponsor: The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

    References
    1.
    Döhner  H, Weisdorf  DJ, Bloomfield  CD.  Acute myeloid leukemia.   N Engl J Med. 2015;373(12):1136-1152. doi:10.1056/NEJMra1406184 PubMedGoogle Scholar
    2.
    Short  NJ, Rytting  ME, Cortes  JE.  Acute myeloid leukaemia.   Lancet. 2018;392(10147):593-606. doi:10.1016/S0140-6736(18)31041-9 PubMedGoogle Scholar
    3.
    Papaemmanuil  E, Gerstung  M, Bullinger  L,  et al.  Genomic classification and prognosis in acute myeloid leukemia.   N Engl J Med. 2016;374(23):2209-2221. doi:10.1056/NEJMoa1516192 PubMedGoogle Scholar
    4.
    Grimwade  D, Hills  RK, Moorman  AV,  et al; National Cancer Research Institute Adult Leukaemia Working Group.  Refinement of cytogenetic classification in acute myeloid leukemia: determination of prognostic significance of rare recurring chromosomal abnormalities among 5876 younger adult patients treated in the United Kingdom Medical Research Council trials.   Blood. 2010;116(3):354-365. doi:10.1182/blood-2009-11-254441 PubMedGoogle Scholar
    5.
    Döhner  H, Estey  E, Grimwade  D,  et al.  Diagnosis and management of AML in adults: 2017 ELN recommendations from an international expert panel.   Blood. 2017;129(4):424-447. doi:10.1182/blood-2016-08-733196 PubMedGoogle Scholar
    6.
    Hourigan  CS, Karp  JE.  Minimal residual disease in acute myeloid leukaemia.   Nat Rev Clin Oncol. 2013;10(8):460-471. doi:10.1038/nrclinonc.2013.100 PubMedGoogle Scholar
    7.
    Schuurhuis  GJ, Heuser  M, Freeman  S,  et al.  Minimal/measurable residual disease in AML: a consensus document from the European LeukemiaNet MRD Working Party.   Blood. 2018;131(12):1275-1291. doi:10.1182/blood-2017-09-801498 PubMedGoogle Scholar
    8.
    Hochhaus  A, Larson  RA, Guilhot  F,  et al; IRIS Investigators.  Long-term outcomes of imatinib treatment for chronic myeloid leukemia.   N Engl J Med. 2017;376(10):917-927. doi:10.1056/NEJMoa1609324 PubMedGoogle Scholar
    9.
    Böttcher  S, Ritgen  M, Fischer  K,  et al.  Minimal residual disease quantification is an independent predictor of progression-free and overall survival in chronic lymphocytic leukemia: a multivariate analysis from the randomized GCLLSG CLL8 trial.   J Clin Oncol. 2012;30(9):980-988. doi:10.1200/JCO.2011.36.9348 PubMedGoogle Scholar
    10.
    Brüggemann  M, Raff  T, Flohr  T,  et al; German Multicenter Study Group for Adult Acute Lymphoblastic Leukemia.  Clinical significance of minimal residual disease quantification in adult patients with standard-risk acute lymphoblastic leukemia.   Blood. 2006;107(3):1116-1123. doi:10.1182/blood-2005-07-2708 PubMedGoogle Scholar
    11.
    Bassan  R, Spinelli  O, Oldani  E,  et al.  Improved risk classification for risk-specific therapy based on the molecular study of minimal residual disease (MRD) in adult acute lymphoblastic leukemia (ALL).   Blood. 2009;113(18):4153-4162. doi:10.1182/blood-2008-11-185132 PubMedGoogle Scholar
    12.
    Berry  DA, Zhou  S, Higley  H,  et al.  Association of minimal residual disease with clinical outcome in pediatric and adult acute lymphoblastic leukemia: a meta-analysis.   JAMA Oncol. 2017;3(7):e170580. doi:10.1001/jamaoncol.2017.0580 PubMedGoogle Scholar
    13.
    Pui  CH, Pei  D, Coustan-Smith  E,  et al.  Clinical utility of sequential minimal residual disease measurements in the context of risk-based therapy in childhood acute lymphoblastic leukaemia: a prospective study.   Lancet Oncol. 2015;16(4):465-474. doi:10.1016/S1470-2045(15)70082-3 PubMedGoogle Scholar
    14.
    Vora  A, Goulden  N, Wade  R,  et al.  Treatment reduction for children and young adults with low-risk acute lymphoblastic leukaemia defined by minimal residual disease (UKALL 2003): a randomised controlled trial.   Lancet Oncol. 2013;14(3):199-209. doi:10.1016/S1470-2045(12)70600-9 PubMedGoogle Scholar
    15.
    Buckley  SA, Wood  BL, Othus  M,  et al.  Minimal residual disease prior to allogeneic hematopoietic cell transplantation in acute myeloid leukemia: a meta-analysis.   Haematologica. 2017;102(5):865-873. doi:10.3324/haematol.2016.159343 PubMedGoogle Scholar
    16.
    Terwijn  M, van Putten  WL, Kelder  A,  et al.  High prognostic impact of flow cytometric minimal residual disease detection in acute myeloid leukemia: data from the HOVON/SAKK AML 42A study.   J Clin Oncol. 2013;31(31):3889-3897. doi:10.1200/JCO.2012.45.9628 PubMedGoogle Scholar
    17.
    Freeman  SD, Virgo  P, Couzens  S,  et al.  Prognostic relevance of treatment response measured by flow cytometric residual disease detection in older patients with acute myeloid leukemia.   J Clin Oncol. 2013;31(32):4123-4131. doi:10.1200/JCO.2013.49.1753 PubMedGoogle Scholar
    18.
    Ivey  A, Hills  RK, Simpson  MA,  et al; UK National Cancer Research Institute AML Working Group.  Assessment of minimal residual disease in standard-risk AML.   N Engl J Med. 2016;374(5):422-433. doi:10.1056/NEJMoa1507471 PubMedGoogle Scholar
    19.
    Ravandi  F, Jorgensen  J, Borthakur  G,  et al.  Persistence of minimal residual disease assessed by multiparameter flow cytometry is highly prognostic in younger patients with acute myeloid leukemia.   Cancer. 2017;123(3):426-435. doi:10.1002/cncr.30361 PubMedGoogle Scholar
    20.
    Balsat  M, Renneville  A, Thomas  X,  et al.  Postinduction minimal residual disease predicts outcome and benefit from allogeneic stem cell transplantation in acute myeloid leukemia with NPM1 mutation: a study by the Acute Leukemia French Association Group.   J Clin Oncol. 2017;35(2):185-193. doi:10.1200/JCO.2016.67.1875 PubMedGoogle Scholar
    21.
    Freeman  SD, Hills  RK, Virgo  P,  et al.  Measurable residual disease at induction redefines partial response in acute myeloid leukemia and stratifies outcomes in patients at standard risk without NPM1 mutations.   J Clin Oncol. 2018;36(15):1486-1497. doi:10.1200/JCO.2017.76.3425 PubMedGoogle Scholar
    22.
    Jongen-Lavrencic  M, Grob  T, Hanekamp  D,  et al.  Molecular minimal residual disease in acute myeloid leukemia.   N Engl J Med. 2018;378(13):1189-1199. doi:10.1056/NEJMoa1716863 PubMedGoogle Scholar
    23.
    Short  NJ, Ravandi  F.  How close are we to incorporating measurable residual disease into clinical practice for acute myeloid leukemia?   Haematologica. 2019;104(8):1532-1541. doi:10.3324/haematol.2018.208454 PubMedGoogle Scholar
    24.
    Moher  D, Liberati  A, Tetzlaff  J, Altman  DG; PRISMA Group.  Preferred Reporting Items for Systematic Reviews and Meta-analyses: the PRISMA statement.   PLoS Med. 2009;6(7):e1000097. doi:10.1371/journal.pmed.1000097 PubMedGoogle Scholar
    25.
    Parkin  B, Londoño-Joshi  A, Kang  Q, Tewari  M, Rhim  AD, Malek  SN.  Ultrasensitive mutation detection identifies rare residual cells causing acute myelogenous leukemia relapse.   J Clin Invest. 2017;127(9):3484-3495. doi:10.1172/JCI91964 PubMedGoogle Scholar
    26.
    Weber  S, Alpermann  T, Dicker  F,  et al.  BAALC expression: a suitable marker for prognostic risk stratification and detection of residual disease in cytogenetically normal acute myeloid leukemia.   Blood Cancer J. 2014;4:e173. doi:10.1038/bcj.2013.71 PubMedGoogle Scholar
    27.
    Weber  S, Haferlach  T, Alpermann  T,  et al.  Feasibility of BAALC gene expression for detection of minimal residual disease and risk stratification in normal karyotype acute myeloid leukaemia.   Br J Haematol. 2016;175(5):904-916. doi:10.1111/bjh.14343 PubMedGoogle Scholar
    28.
    Steinbach  D, Bader  P, Willasch  A,  et al.  Prospective validation of a new method of monitoring minimal residual disease in childhood acute myelogenous leukemia.   Clin Cancer Res. 2015;21(6):1353-1359. doi:10.1158/1078-0432.CCR-14-1999 PubMedGoogle Scholar
    29.
    Guyot  P, Ades  AE, Ouwens  MJNM, Welton  NJ.  Enhanced secondary analysis of survival data: reconstructing the data from published Kaplan-Meier survival curves.   BMC Med Res Methodol. 2012;12(1):9. doi:10.1186/1471-2288-12-9 PubMedGoogle Scholar
    30.
    Berry  DA.  Bayesian clinical trials.   Nat Rev Drug Discov. 2006;5(1):27-36. doi:10.1038/nrd1927 PubMedGoogle Scholar
    31.
    Gelman  A, Carlin  JB, Stern  HS, Dunson  DB, Vehtari  A, Rubin  DB.  Bayesian Data Analysis. 3rd ed. Text in Statistical Science series. CRC press; 2013. doi:10.1201/b16018
    32.
    Higgins  J, Thomas  J, Chandler  J,  et al Cochrane handbook for systematic reviews of interventions. Accessed September 1, 2019. http://www.training.cochrane.org/handbook
    33.
    Borenstein  M, Hedges  LV, Higgins  JP, Rothstein  HR.  Introduction to Meta-analysis. John Wiley & Sons; 2011.
    34.
    Fu  C, Zhou  S, Berry  DA. Evidence synthesis with reconstructed survival data. Pennsylvania State University Technical Report; 2020.
    35.
    Royston  P, Parmar  MK.  Restricted mean survival time: an alternative to the hazard ratio for the design and analysis of randomized trials with a time-to-event outcome.   BMC Med Res Methodol. 2013;13:152. doi:10.1186/1471-2288-13-152 PubMedGoogle Scholar
    36.
    Uno  H, Claggett  B, Tian  L,  et al.  Moving beyond the hazard ratio in quantifying the between-group difference in survival analysis.   J Clin Oncol. 2014;32(22):2380-2385. doi:10.1200/JCO.2014.55.2208 PubMedGoogle Scholar
    37.
    Trinquart  L, Jacot  J, Conner  SC, Porcher  R.  Comparison of treatment effects measured by the hazard ratio and by the ratio of restricted mean survival times in oncology randomized controlled trials.   J Clin Oncol. 2016;34(15):1813-1819. doi:10.1200/JCO.2015.64.2488 PubMedGoogle Scholar
    38.
    San Miguel  JF, Vidriales  MB, López-Berges  C,  et al.  Early immunophenotypical evaluation of minimal residual disease in acute myeloid leukemia identifies different patient risk groups and may contribute to postinduction treatment stratification.   Blood. 2001;98(6):1746-1751. doi:10.1182/blood.V98.6.1746 PubMedGoogle Scholar
    39.
    Sievers  EL, Lange  BJ, Alonzo  TA,  et al.  Immunophenotypic evidence of leukemia after induction therapy predicts relapse: results from a prospective Children’s Cancer Group study of 252 patients with acute myeloid leukemia.   Blood. 2003;101(9):3398-3406. doi:10.1182/blood-2002-10-3064 PubMedGoogle Scholar
    40.
    Coustan-Smith  E, Ribeiro  RC, Rubnitz  JE,  et al.  Clinical significance of residual disease during treatment in childhood acute myeloid leukaemia.   Br J Haematol. 2003;123(2):243-252. doi:10.1046/j.1365-2141.2003.04610.x PubMedGoogle Scholar
    41.
    Feller  N, van der Pol  MA, van Stijn  A,  et al.  MRD parameters using immunophenotypic detection methods are highly reliable in predicting survival in acute myeloid leukaemia.   Leukemia. 2004;18(8):1380-1390. doi:10.1038/sj.leu.2403405 PubMedGoogle Scholar
    42.
    Weisser  M, Kern  W, Rauhut  S,  et al.  Prognostic impact of RT-PCR-based quantification of WT1 gene expression during MRD monitoring of acute myeloid leukemia.   Leukemia. 2005;19(8):1416-1423. doi:10.1038/sj.leu.2403809 PubMedGoogle Scholar
    43.
    Weisser  M, Kern  W, Schoch  C, Hiddemann  W, Haferlach  T, Schnittger  S.  Risk assessment by monitoring expression levels of partial tandem duplications in the MLL gene in acute myeloid leukemia during therapy.   Haematologica. 2005;90(7):881-889.PubMedGoogle Scholar
    44.
    Perea  G, Lasa  A, Aventín  A,  et al; Grupo Cooperativo para el Estudio y Tratamiento de las Leucemias Agudas y Miel.  Prognostic value of minimal residual disease (MRD) in acute myeloid leukemia (AML) with favorable cytogenetics [t(8;21) and inv(16)].   Leukemia. 2006;20(1):87-94. doi:10.1038/sj.leu.2404015 PubMedGoogle Scholar
    45.
    Lapillonne  H, Renneville  A, Auvrignon  A,  et al.  High WT1 expression after induction therapy predicts high risk of relapse and death in pediatric acute myeloid leukemia.   J Clin Oncol. 2006;24(10):1507-1515. doi:10.1200/JCO.2005.03.5303 PubMedGoogle Scholar
    46.
    Laane  E, Derolf  AR, Björklund  E,  et al.  The effect of allogeneic stem cell transplantation on outcome in younger acute myeloid leukemia patients with minimal residual disease detected by flow cytometry at the end of post-remission chemotherapy.   Haematologica. 2006;91(6):833-836.PubMedGoogle Scholar
    47.
    Langebrake  C, Creutzig  U, Dworzak  M,  et al; MRD-AML-BFM Study Group.  Residual disease monitoring in childhood acute myeloid leukemia by multiparameter flow cytometry: the MRD-AML-BFM Study Group.   J Clin Oncol. 2006;24(22):3686-3692. doi:10.1200/JCO.2005.05.4312 PubMedGoogle Scholar
    48.
    Maurillo  L, Buccisano  F, Spagnoli  A,  et al.  Monitoring of minimal residual disease in adult acute myeloid leukemia using peripheral blood as an alternative source to bone marrow.   Haematologica. 2007;92(5):605-611. doi:10.3324/haematol.10432 PubMedGoogle Scholar
    49.
    Narimatsu  H, Iino  M, Ichihashi  T,  et al.  Clinical significance of minimal residual disease in patients with t(8;21) acute myeloid leukemia in Japan.   Int J Hematol. 2008;88(2):154-158. doi:10.1007/s12185-008-0108-1 PubMedGoogle Scholar
    50.
    Al-Mawali  A, Gillis  D, Lewis  I.  The use of receiver operating characteristic analysis for detection of minimal residual disease using five-color multiparameter flow cytometry in acute myeloid leukemia identifies patients with high risk of relapse.   Cytometry B Clin Cytom. 2009;76(2):91-101. doi:10.1002/cyto.b.20444 PubMedGoogle Scholar
    51.
    Hess  CJ, Feller  N, Denkers  F,  et al.  Correlation of minimal residual disease cell frequency with molecular genotype in patients with acute myeloid leukemia.   Haematologica. 2009;94(1):46-53. doi:10.3324/haematol.13110 PubMedGoogle Scholar
    52.
    Cilloni  D, Renneville  A, Hermitte  F,  et al.  Real-time quantitative polymerase chain reaction detection of minimal residual disease by standardized WT1 assay to enhance risk stratification in acute myeloid leukemia: a European LeukemiaNet study.   J Clin Oncol. 2009;27(31):5195-5201. doi:10.1200/JCO.2009.22.4865 PubMedGoogle Scholar
    53.
    Guièze  R, Renneville  A, Cayuela  JM,  et al.  Prognostic value of minimal residual disease by real-time quantitative PCR in acute myeloid leukemia with CBFB-MYH11 rearrangement: the French experience.   Leukemia. 2010;24(7):1386-1388. doi:10.1038/leu.2010.112 PubMedGoogle Scholar
    54.
    Buccisano  F, Maurillo  L, Spagnoli  A,  et al.  Cytogenetic and molecular diagnostic characterization combined to postconsolidation minimal residual disease assessment by flow cytometry improves risk stratification in adult acute myeloid leukemia.   Blood. 2010;116(13):2295-2303. doi:10.1182/blood-2009-12-258178 PubMedGoogle Scholar
    55.
    Corbacioglu  A, Scholl  C, Schlenk  RF,  et al.  Prognostic impact of minimal residual disease in CBFB-MYH11-positive acute myeloid leukemia.   J Clin Oncol. 2010;28(23):3724-3729. doi:10.1200/JCO.2010.28.6468 PubMedGoogle Scholar
    56.
    van der Velden  VH, van der Sluijs-Geling  A, Gibson  BE,  et al.  Clinical significance of flow cytometric minimal residual disease detection in pediatric acute myeloid leukemia patients treated according to the DCOG ANLL97/MRC AML12 protocol.   Leukemia. 2010;24(9):1599-1606. doi:10.1038/leu.2010.153 PubMedGoogle Scholar
    57.
    Chou  WC, Hou  HA, Liu  CY,  et al.  Sensitive measurement of quantity dynamics of FLT3 internal tandem duplication at early time points provides prognostic information.   Ann Oncol. 2011;22(3):696-704. doi:10.1093/annonc/mdq402 PubMedGoogle Scholar
    58.
    Inoue  D, Maruoka  H, Takahashi  T.  Clinical analysis and optimization of postremission therapy for acute myeloid leukemia patients with minimal residual disease as determined by flow cytometry.   Mediterr J Hematol Infect Dis. 2010;2(2):e2010020. doi:10.4084/mjhid.2010.020 PubMedGoogle Scholar
    59.
    Krönke  J, Schlenk  RF, Jensen  KO,  et al.  Monitoring of minimal residual disease in NPM1-mutated acute myeloid leukemia: a study from the German-Austrian acute myeloid leukemia study group.   J Clin Oncol. 2011;29(19):2709-2716. doi:10.1200/JCO.2011.35.0371 PubMedGoogle Scholar
    60.
    Chen  Y, Cortes  J, Estrov  Z,  et al.  Persistence of cytogenetic abnormalities at complete remission after induction in patients with acute myeloid leukemia: prognostic significance and the potential role of allogeneic stem-cell transplantation.   J Clin Oncol. 2011;29(18):2507-2513. doi:10.1200/JCO.2010.34.2873 PubMedGoogle Scholar
    61.
    Gray  JX, McMillen  L, Mollee  P,  et al.  WT1 expression as a marker of minimal residual disease predicts outcome in acute myeloid leukemia when measured post-consolidation.   Leuk Res. 2012;36(4):453-458. doi:10.1016/j.leukres.2011.09.005 PubMedGoogle Scholar
    62.
    Terwijn  M, Kelder  A, Snel  AN,  et al.  Minimal residual disease detection defined as the malignant fraction of the total primitive stem cell compartment offers additional prognostic information in acute myeloid leukaemia.   Int J Lab Hematol. 2012;34(4):432-441. doi:10.1111/j.1751-553X.2012.01416.x PubMedGoogle Scholar
    63.
    Loken  MR, Alonzo  TA, Pardo  L,  et al.  Residual disease detected by multidimensional flow cytometry signifies high relapse risk in patients with de novo acute myeloid leukemia: a report from Children’s Oncology Group.   Blood. 2012;120(8):1581-1588. doi:10.1182/blood-2012-02-408336 PubMedGoogle Scholar
    64.
    Yin  JA, O’Brien  MA, Hills  RK, Daly  SB, Wheatley  K, Burnett  AK.  Minimal residual disease monitoring by quantitative RT-PCR in core binding factor AML allows risk stratification and predicts relapse: results of the United Kingdom MRC AML-15 trial.   Blood. 2012;120(14):2826-2835. doi:10.1182/blood-2012-06-435669 PubMedGoogle Scholar
    65.
    Inaba  H, Coustan-Smith  E, Cao  X,  et al.  Comparative analysis of different approaches to measure treatment response in acute myeloid leukemia.   J Clin Oncol. 2012;30(29):3625-3632. doi:10.1200/JCO.2011.41.5323 PubMedGoogle Scholar
    66.
    Xu  XJ, Feng  JH, Tang  YM,  et al.  Prognostic significance of flow cytometric minimal residual disease assessment after the first induction course in Chinese childhood acute myeloid leukemia.   Leuk Res. 2013;37(2):134-138. doi:10.1016/j.leukres.2012.11.002 PubMedGoogle Scholar
    67.
    Jourdan  E, Boissel  N, Chevret  S,  et al; French AML Intergroup.  Prospective evaluation of gene mutations and minimal residual disease in patients with core binding factor acute myeloid leukemia.   Blood. 2013;121(12):2213-2223. doi:10.1182/blood-2012-10-462879 PubMedGoogle Scholar
    68.
    Zhang  L, Li  Q, Li  W,  et al.  Monitoring of minimal residual disease in acute myeloid leukemia with t(8;21)(q22;q22).   Int J Hematol. 2013;97(6):786-792. doi:10.1007/s12185-013-1344-6 PubMedGoogle Scholar
    69.
    Hoyos  M, Nomdedeu  JF, Esteve  J,  et al.  Core binding factor acute myeloid leukemia: the impact of age, leukocyte count, molecular findings, and minimal residual disease.   Eur J Haematol. 2013;91(3):209-218. doi:10.1111/ejh.12130 PubMedGoogle Scholar
    70.
    Shayegi  N, Kramer  M, Bornhäuser  M,  et al; Study Alliance Leukemia (SAL).  The level of residual disease based on mutant NPM1 is an independent prognostic factor for relapse and survival in AML.   Blood. 2013;122(1):83-92. doi:10.1182/blood-2012-10-461749 PubMedGoogle Scholar
    71.
    Marani  C, Clavio  M, Grasso  R,  et al.  Integrating post induction WT1 quantification and flow-cytometry results improves minimal residual disease stratification in acute myeloid leukemia.   Leuk Res. 2013;37(12):1606-1611. doi:10.1016/j.leukres.2013.07.005 PubMedGoogle Scholar
    72.
    Rossi  G, Minervini  MM, Melillo  L,  et al.  Predictive role of minimal residual disease and log clearance in acute myeloid leukemia: a comparison between multiparameter flow cytometry and Wilm’s tumor 1 levels.   Ann Hematol. 2014;93(7):1149-1157. doi:10.1007/s00277-014-2029-9 PubMedGoogle Scholar
    73.
    Hubmann  M, Köhnke  T, Hoster  E,  et al.  Molecular response assessment by quantitative real-time polymerase chain reaction after induction therapy in NPM1-mutated patients identifies those at high risk of relapse.   Haematologica. 2014;99(8):1317-1325. doi:10.3324/haematol.2014.104133 PubMedGoogle Scholar
    74.
    Wang  L, Gao  L, Xu  S,  et al.  High prognostic value of minimal residual disease detected by flow-cytometry-enhanced fluorescence in situ hybridization in core-binding factor acute myeloid leukemia (CBF-AML).   Ann Hematol. 2014;93(10):1685-1694. doi:10.1007/s00277-014-2107-z PubMedGoogle Scholar
    75.
    Köhnke  T, Sauter  D, Ringel  K,  et al.  Early assessment of minimal residual disease in AML by flow cytometry during aplasia identifies patients at increased risk of relapse.   Leukemia. 2015;29(2):377-386. doi:10.1038/leu.2014.186 PubMedGoogle Scholar
    76.
    Zhang  L, Cao  Z, Ruan  M,  et al.  Monitoring the AML1/ETO fusion transcript to predict outcome in childhood acute myeloid leukemia.   Pediatr Blood Cancer. 2014;61(10):1761-1766. doi:10.1002/pbc.25109 PubMedGoogle Scholar
    77.
    Hirsch  P, Labopin  M, Viguié  F,  et al.  Interest of cytogenetic and FISH evaluation for prognosis evaluation in 198 patients with acute myeloid leukemia in first complete remission in a single institution.   Leuk Res. 2014;38(8):907-912. doi:10.1016/j.leukres.2014.05.021 PubMedGoogle Scholar
    78.
    Lambert  J, Lambert  J, Nibourel  O,  et al.  MRD assessed by WT1 and NPM1 transcript levels identifies distinct outcomes in AML patients and is influenced by gemtuzumab ozogamicin.   Oncotarget. 2014;5(15):6280-6288. doi:10.18632/oncotarget.2196 PubMedGoogle Scholar
    79.
    Yoon  JH, Kim  HJ, Kim  JW,  et al.  Identification of molecular and cytogenetic risk factors for unfavorable core-binding factor-positive adult AML with post-remission treatment outcome analysis including transplantation.   Bone Marrow Transplant. 2014;49(12):1466-1474. doi:10.1038/bmt.2014.180 PubMedGoogle Scholar
    80.
    Terwijn  M, Zeijlemaker  W, Kelder  A,  et al.  Leukemic stem cell frequency: a strong biomarker for clinical outcome in acute myeloid leukemia.   PLoS One. 2014;9(9):e107587. doi:10.1371/journal.pone.0107587 PubMedGoogle Scholar
    81.
    Pigazzi  M, Manara  E, Buldini  B,  et al.  Minimal residual disease monitored after induction therapy by RQ-PCR can contribute to tailor treatment of patients with t(8;21) RUNX1-RUNX1T1 rearrangement.   Haematologica. 2015;100(3):e99-e101. doi:10.3324/haematol.2014.114579 PubMedGoogle Scholar
    82.
    Chen  X, Xie  H, Wood  BL,  et al.  Relation of clinical response and minimal residual disease and their prognostic impact on outcome in acute myeloid leukemia.   J Clin Oncol. 2015;33(11):1258-1264. doi:10.1200/JCO.2014.58.3518 PubMedGoogle Scholar
    83.
    Buccisano  F, Maurillo  L, Piciocchi  A,  et al.  Minimal residual disease negativity in elderly patients with acute myeloid leukemia may indicate different postremission strategies than in younger patients.   Ann Hematol. 2015;94(8):1319-1326. doi:10.1007/s00277-015-2364-5 PubMedGoogle Scholar
    84.
    Shibasaki  Y, Seki  Y, Tanaka  T,  et al.  The association of level of reduction of Wilms’ tumor gene 1 mRNA transcript in bone marrow and outcome in acute myeloid leukemia patients.   Leuk Res. 2015;39(6):667-671. doi:10.1016/j.leukres.2015.03.021 PubMedGoogle Scholar
    85.
    Klco  JM, Miller  CA, Griffith  M,  et al.  Association between mutation clearance after induction therapy and outcomes in acute myeloid leukemia.   JAMA. 2015;314(8):811-822. doi:10.1001/jama.2015.9643 PubMedGoogle Scholar
    86.
    Zeijlemaker  W, Kelder  A, Oussoren-Brockhoff  YJ,  et al.  Peripheral blood minimal residual disease may replace bone marrow minimal residual disease as an immunophenotypic biomarker for impending relapse in acute myeloid leukemia.   Leukemia. 2016;30(3):708-715. doi:10.1038/leu.2015.255 PubMedGoogle Scholar
    87.
    Vidriales  MB, Pérez-López  E, Pegenaute  C,  et al; PETHEMA Programa para el Estudio de la Terapéutica en Hemopatías Malignas Cooperative Study Group.  Minimal residual disease evaluation by flow cytometry is a complementary tool to cytogenetics for treatment decisions in acute myeloid leukaemia.   Leuk Res. 2016;40:1-9. doi:10.1016/j.leukres.2015.10.002 PubMedGoogle Scholar
    88.
    Willekens  C, Blanchet  O, Renneville  A,  et al; French AML Intergroup.  Prospective long-term minimal residual disease monitoring using RQ-PCR in RUNX1-RUNX1T1-positive acute myeloid leukemia: results of the French CBF-2006 trial.   Haematologica. 2016;101(3):328-335. doi:10.3324/haematol.2015.131946 PubMedGoogle Scholar
    89.
    Malagola  M, Skert  C, Borlenghi  E,  et al.  Postremission sequential monitoring of minimal residual disease by WT1 Q-PCR and multiparametric flow cytometry assessment predicts relapse and may help to address risk-adapted therapy in acute myeloid leukemia patients.   Cancer Med. 2016;5(2):265-274. doi:10.1002/cam4.593 PubMedGoogle Scholar
    90.
    Keino  D, Kinoshita  A, Tomizawa  D,  et al.  Residual disease detected by multidimensional flow cytometry shows prognostic significance in childhood acute myeloid leukemia with intermediate cytogenetics and negative FLT3-ITD: a report from the Tokyo Children’s Cancer Study Group.   Int J Hematol. 2016;103(4):416-422. doi:10.1007/s12185-016-1937-y PubMedGoogle Scholar
    91.
    Kim  Y, Lee  GD, Park  J,  et al.  Quantitative fragment analysis of FLT3-ITD efficiently identifying poor prognostic group with high mutant allele burden or long ITD length.   Blood Cancer J. 2015;5:e336. doi:10.1038/bcj.2015.61 PubMedGoogle Scholar
    92.
    Tierens  A, Bjørklund  E, Siitonen  S,  et al.  Residual disease detected by flow cytometry is an independent predictor of survival in childhood acute myeloid leukaemia; results of the NOPHO-AML 2004 study.   Br J Haematol. 2016;174(4):600-609. doi:10.1111/bjh.14093 PubMedGoogle Scholar
    93.
    Othus  M, Wood  BL, Stirewalt  DL,  et al.  Effect of measurable (“minimal”) residual disease (MRD) information on prediction of relapse and survival in adult acute myeloid leukemia.   Leukemia. 2016;30(10):2080-2083. doi:10.1038/leu.2016.120 PubMedGoogle Scholar
    94.
    Manara  E, Basso  G, Zampini  M,  et al.  Characterization of children with FLT3-ITD acute myeloid leukemia: a report from the AIEOP AML-2002 study group.   Leukemia. 2017;31(1):18-25. doi:10.1038/leu.2016.177 PubMedGoogle Scholar
    95.
    Huang  S, Yang  H, Li  Y,  et al.  Prognostic significance of mixed-lineage leukemia (MLL) gene detected by real-time fluorescence quantitative PCR assay in acute myeloid leukemia.   Med Sci Monit. 2016;22:3009-3017. doi:10.12659/MSM.900429 PubMedGoogle Scholar
    96.
    Buldini  B, Rizzati  F, Masetti  R,  et al.  Prognostic significance of flow-cytometry evaluation of minimal residual disease in children with acute myeloid leukaemia treated according to the AIEOP-AML 2002/01 study protocol.   Br J Haematol. 2017;177(1):116-126. doi:10.1111/bjh.14523 PubMedGoogle Scholar
    97.
    Martínez-Laperche  C, Kwon  M, Franco-Villegas  AC,  et al; Cooperative Group for Molecular Biology in Haematology (GBMH).  Wilms tumor 1 gene expression levels improve risk stratification in AML patients: results of a multicentre study within the Spanish Group for Molecular Biology in Haematology.   Br J Haematol. 2018;181(4):542-546. doi:10.1111/bjh.14635 PubMedGoogle Scholar
    98.
    Daver  N, Kantarjian  H, Garcia-Manero  G,  et al.  Vosaroxin in combination with decitabine in newly diagnosed older patients with acute myeloid leukemia or high-risk myelodysplastic syndrome.   Haematologica. 2017;102(10):1709-1717. doi:10.3324/haematol.2017.168732 PubMedGoogle Scholar
    99.
    Frairia  C, Aydin  S, Audisio  E,  et al.  Post-remissional and pre-transplant role of minimal residual disease detected by WT1 in acute myeloid leukemia: a retrospective cohort study.   Leuk Res. 2017;61:10-17. doi:10.1016/j.leukres.2017.08.008 PubMedGoogle Scholar
    100.
    Boddu  P, Jorgensen  J, Kantarjian  H,  et al.  Achievement of a negative minimal residual disease state after hypomethylating agent therapy in older patients with AML reduces the risk of relapse.   Leukemia. 2018;32(1):241-244. doi:10.1038/leu.2017.285 PubMedGoogle Scholar
    101.
    Lacombe  F, Campos  L, Allou  K,  et al; Groupe d’Etude Immunologique des Leucémies (GEIL).  Prognostic value of multicenter flow cytometry harmonized assessment of minimal residual disease in acute myeloblastic leukemia.   Hematol Oncol. 2018;36(2):422-428. doi:10.1002/hon.2488 PubMedGoogle Scholar
    102.
    Ferret  Y, Boissel  N, Helevaut  N,  et al.  Clinical relevance of IDH1/2 mutant allele burden during follow-up in acute myeloid leukemia: a study by the French ALFA group.   Haematologica. 2018;103(5):822-829. doi:10.3324/haematol.2017.183525 PubMedGoogle Scholar
    103.
    Höllein  A, Jeromin  S, Meggendorfer  M,  et al.  Minimal residual disease (MRD) monitoring and mutational landscape in AML with RUNX1-RUNX1T1: a study on 134 patients.   Leukemia. 2018;32(10):2270-2274. doi:10.1038/s41375-018-0086-0 PubMedGoogle Scholar
    104.
    Morita  K, Kantarjian  HM, Wang  F,  et al.  Clearance of somatic mutations at remission and the risk of relapse in acute myeloid leukemia.   J Clin Oncol. 2018;36(18):1788-1797. doi:10.1200/JCO.2017.77.6757 PubMedGoogle Scholar
    105.
    Boddu  P, Gurguis  C, Sanford  D,  et al.  Response kinetics and factors predicting survival in core-binding factor leukemia.   Leukemia. 2018;32(12):2698-2701. doi:10.1038/s41375-018-0158-1 PubMedGoogle Scholar
    106.
    Onecha  E, Linares  M, Rapado  I,  et al.  A novel deep targeted sequencing method for minimal residual disease monitoring in acute myeloid leukemia.   Haematologica. 2019;104(2):288-296. doi:10.3324/haematol.2018.194712 PubMedGoogle Scholar
    107.
    Ok  CY, Loghavi  S, Sui  D,  et al.  Persistent IDH1/2 mutations in remission can predict relapse in patients with acute myeloid leukemia.   Haematologica. 2019;104(2):305-311. doi:10.3324/haematol.2018.191148 PubMedGoogle Scholar
    108.
    Zhu  HH, Zhang  XH, Qin  YZ,  et al.  MRD-directed risk stratification treatment may improve outcomes of t(8;21) AML in the first complete remission: results from the AML05 multicenter trial.   Blood. 2013;121(20):4056-4062. doi:10.1182/blood-2012-11-468348 PubMedGoogle Scholar
    109.
    Hourigan  CS, Dillon  LW, Gui  G,  et al.  Impact of conditioning intensity of allogeneic transplantation for acute myeloid leukemia with genomic evidence of residual disease.   J Clin Oncol. 2020;38(12):1273-1283. doi:10.1200/JCO.19.03011 PubMedGoogle Scholar
    110.
    Sockel  K, Wermke  M, Radke  J,  et al.  Minimal residual disease-directed preemptive treatment with azacitidine in patients with NPM1-mutant acute myeloid leukemia and molecular relapse.   Haematologica. 2011;96(10):1568-1570. doi:10.3324/haematol.2011.044388 PubMedGoogle Scholar
    111.
    Platzbecker  U, Wermke  M, Radke  J,  et al.  Azacitidine for treatment of imminent relapse in MDS or AML patients after allogeneic HSCT: results of the RELAZA trial.   Leukemia. 2012;26(3):381-389. doi:10.1038/leu.2011.234 PubMedGoogle Scholar
    112.
    Platzbecker  U, Middeke  JM, Sockel  K,  et al.  Measurable residual disease-guided treatment with azacitidine to prevent haematological relapse in patients with myelodysplastic syndrome and acute myeloid leukaemia (RELAZA2): an open-label, multicentre, phase 2 trial.   Lancet Oncol. 2018;19(12):1668-1679. doi:10.1016/S1470-2045(18)30580-1 PubMedGoogle Scholar
    113.
    Ragon  BK, Daver  N, Garcia-Manero  G,  et al.  Minimal residual disease eradication with epigenetic therapy in core binding factor acute myeloid leukemia.   Am J Hematol. 2017;92(9):845-850. doi:10.1002/ajh.24782 PubMedGoogle Scholar
    114.
    Gökbuget  N, Dombret  H, Bonifacio  M,  et al.  Blinatumomab for minimal residual disease in adults with B-cell precursor acute lymphoblastic leukemia.   Blood. 2018;131(14):1522-1531. doi:10.1182/blood-2017-08-798322 PubMedGoogle Scholar
    ×