Contradicted and Initially Stronger Effects in Highly Cited Clinical Research | Research, Methods, Statistics | JAMA | JAMA Network
[Skip to Navigation]
Access to paid content on this site is currently suspended due to excessive activity being detected from your IP address Please contact the publisher to request reinstatement.
LeLorier J, Gregoire G, Benhaddad A, Lapierre J, Derderian F. Discrepancies between meta-analyses and subsequent large randomized, controlled trials.  N Engl J Med. 1997;337:536-5429262498Google ScholarCrossref
Cappelleri JC, Ioannidis JP, Schmid CH.  et al.  Large trials vs meta-analysis of smaller trials: how do their results compare?  JAMA. 1996;276:1332-13388861993Google ScholarCrossref
Benson K, Hartz AJ. A comparison of observational studies and randomized, controlled trials.  N Engl J Med. 2000;342:1878-188610861324Google ScholarCrossref
Concato J, Shah N, Horwitz RI. Randomized, controlled trials, observational studies, and the hierarchy of research designs.  N Engl J Med. 2000;342:1887-189210861325Google ScholarCrossref
Ioannidis JP, Haidich AB, Pappa M.  et al.  Comparison of evidence of treatment effects in randomized and nonrandomized studies.  JAMA. 2001;286:821-83011497536Google ScholarCrossref
Institute for Scientific Information.  Web of Knowledge. Available at Accessed August 20, 2004
Dickersin K, Scherer R, Suci ES, Gil-Montero M. Problems with indexing and citation of articles with group authorship.  JAMA. 2002;287:2772-277412038908Google ScholarCrossref
 Cochrane Reviewer’s HandbookAvailable at:
Volberding PA, Lagakos SW, Koch MA.  et al. the AIDS Clinical Trials Group of the National Institute of Allergy and Infectious Diseases.  Zidovudine in asymptomatic human immunodeficiency virus infection: a controlled trial in persons with fewer than 500 CD4-positive cells per cubic millimeter.  N Engl J Med. 1990;322:941-9491969115Google ScholarCrossref
Brown G, Albers JJ, Fisher LD.  et al.  Regression of coronary artery disease as a result of intensive lipid-lowering therapy in men with high levels of apolipoprotein B.  N Engl J Med. 1990;323:1289-12982215615Google ScholarCrossref
Moertel CG, Fleming TR, Macdonald JS.  et al.  Levamisole and fluorouracil for adjuvant therapy of resected colon carcinoma.  N Engl J Med. 1990;322:352-3582300087Google ScholarCrossref
Cohn JN, Johnson G, Ziesche S.  et al.  A comparison of enalapril with hydralazine-isosorbide dinitrate in the treatment of chronic congestive heart failure.  N Engl J Med. 1991;325:303-3102057035Google ScholarCrossref
Stampfer MJ, Colditz GA, Willett WC.  et al.  Postmenopausal estrogen therapy and cardiovascular disease: ten-year follow-up from the Nurses' Health Study.  N Engl J Med. 1991;325:756-7621870648Google ScholarCrossref
North American Symptomatic Carotid Endarterectomy Trial Collaborators.  Beneficial effect of carotid endarterectomy in symptomatic patients with high-grade carotid stenosis.  N Engl J Med. 1991;325:445-4531852179Google ScholarCrossref
Ziegler EJ, Fisher CJ Jr, Sprung CL.  et al. the HA-1A Sepsis Study Group.  Treatment of gram-negative bacteremia and septic shock with HA-1A human monoclonal antibody against endotoxin: a randomized, double-blind, placebo-controlled trial.  N Engl J Med. 1991;324:429-4361988827Google ScholarCrossref
The SOLVD Investigators.  Effect of enalapril on survival in patients with reduced left ventricular ejection fractions and congestive heart failure.  N Engl J Med. 1991;325:293-3022057034Google ScholarCrossref
Pfeffer MA, Braunwald E, Moye LA.  et al. the SAVE Investigators.  Effect of captopril on mortality and morbidity in patients with left ventricular dysfunction after myocardial infarction: results of the survival and ventricular enlargement trial.  N Engl J Med. 1992;327:669-6771386652Google ScholarCrossref
Grines CL, Browne KF, Marco J.  et al. the Primary Angioplasty in Myocardial Infarction Study Group.  A comparison of immediate angioplasty with thrombolytic therapy for acute myocardial infarction.  N Engl J Med. 1993;328:673-6798433725Google ScholarCrossref
Lewis EJ, Hunsicker LG, Bain RP, Rohde RD.the Collaborative Study Group.  The effect of angiotensin-converting-enzyme inhibition on diabetic nephropathy.  N Engl J Med. 1993;329:1456-14628413456Google ScholarCrossref
Rimm EB, Stampfer MJ, Ascherio A, Giovannucci E, Colditz GA, Willett WC. Vitamin E consumption and the risk of coronary heart disease in men.  N Engl J Med. 1993;328:1450-14568479464Google ScholarCrossref
Stampfer MJ, Hennekens CH, Manson JE, Colditz GA, Rosner B, Willett WC. Vitamin E consumption and the risk of coronary disease in women.  N Engl J Med. 1993;328:1444-14498479463Google ScholarCrossref
Rossaint R, Falke KJ, Lopez F, Slama K, Pison U, Zapol WM. Inhaled nitric oxide for the adult respiratory distress syndrome.  N Engl J Med. 1993;328:399-4058357359Google ScholarCrossref
The Diabetes Control and Complications Trial Research Group.  The effect of intensive treatment of diabetes on the development and progression of long-term complications in insulin-dependent diabetes mellitus.  N Engl J Med. 1993;329:977-9868366922Google ScholarCrossref
The EPIC Investigation.  Use of a monoclonal antibody directed against the platelet glycoprotein IIb/IIIa receptor in high-risk coronary angioplasty.  N Engl J Med. 1994;330:956-9618121459Google ScholarCrossref
Connor EM, Sperling RS, Gelber R.  et al. Pediatric AIDS Clinical Trials Group Protocol 076 Study Group.  Reduction of maternal-infant transmission of human immunodeficiency virus type 1 with zidovudine treatment.  N Engl J Med. 1994;331:1173-11807935654Google ScholarCrossref
Fischman DL, Leon MB, Baim DS.  et al. Stent Restenosis Study Investigators.  A randomized comparison of coronary-stent placement and balloon angioplasty in the treatment of coronary artery disease.  N Engl J Med. 1994;331:496-5018041414Google ScholarCrossref
Serruys PW, de Jaegere P, Kiemeneij F.  et al. BENESTENT Study Group.  A comparison of balloon-expandable-stent implantation with balloon angioplasty in patients with coronary artery disease.  N Engl J Med. 1994;331:489-4958041413Google ScholarCrossref
The Alpha-Tocopherol, Beta Carotene Cancer Prevention Study Group.  The effect of vitamin E and beta carotene on the incidence of lung cancer and other cancers in male smokers.  N Engl J Med. 1994;330:1029-10358127329Google ScholarCrossref
The National Institute of Neurological Disorders and Stroke rt-PA Stroke Study Group.  Tissue plasminogen activator for acute ischemic stroke.  N Engl J Med. 1995;333:1581-15877477192Google ScholarCrossref
Shepherd J, Cobbe SM, Ford I.  et al. West of Scotland Coronary Prevention Study Group.  Prevention of coronary heart disease with pravastatin in men with hypercholesterolemia.  N Engl J Med. 1995;333:1301-13077566020Google ScholarCrossref
Sacks FM, Pfeffer MA, Moye LA.  et al. Cholesterol and Recurrent Events Trial Investigators.  The effect of pravastatin on coronary events after myocardial infarction in patients with average cholesterol levels.  N Engl J Med. 1996;335:1001-10098801446Google ScholarCrossref
Packer M, Bristow MR, Cohn JN.  et al. US Carvedilol Heart Failure Study Group.  The effect of carvedilol on morbidity and mortality in patients with chronic heart failure.  N Engl J Med. 1996;334:1349-13558614419Google ScholarCrossref
Omenn GS, Goodman GE, Thornquist MD.  et al.  Effects of a combination of beta carotene and vitamin A on lung cancer and cardiovascular disease.  N Engl J Med. 1996;334:1150-11558602180Google ScholarCrossref
Ridker PM, Cushman M, Stampfer MJ, Tracy RP, Hennekens CH. Inflammation, aspirin, and the risk of cardiovascular disease in apparently healthy men.  N Engl J Med. 1997;336:973-9799077376Google ScholarCrossref
Hammer SM, Squires KE, Hughes MD.  et al. AIDS Clinical Trials Group 320 Study Team.  A controlled trial of two nucleoside analogues plus indinavir in persons with human immunodeficiency virus infection and CD4 cell counts of 200 per cubic millimeter or less.  N Engl J Med. 1997;337:725-7339287227Google ScholarCrossref
The EPILOG Investigators.  Platelet glycoprotein IIb/IIIa receptor blockade and low-dose heparin during percutaneous coronary revascularization.  N Engl J Med. 1997;336:1689-16969182212Google ScholarCrossref
McHutchison JG, Gordon SC, Schiff ER.  et al. Hepatitis Interventional Therapy Group.  Interferon alfa-2b alone or in combination with ribavirin as initial treatment for chronic hepatitis C.  N Engl J Med. 1998;339:1485-14929819446Google ScholarCrossref
The Long-Term Intervention with Pravastatin in Ischaemic Disease (LIPID) Study Group.  Prevention of cardiovascular events and death with pravastatin in patients with coronary heart disease and a broad range of initial cholesterol levels.  N Engl J Med. 1998;339:1349-13579841303Google ScholarCrossref
Pitt B, Zannad F, Remme WJ.  et al. Randomized Aldactone Evaluation Study Investigators.  The effect of spironolactone on morbidity and mortality in patients with severe heart failure.  N Engl J Med. 1999;341:709-71710471456Google ScholarCrossref
Yusuf S, Sleight P, Pogue J, Bosch J, Davies R, Dagenais G.the Heart Outcomes Prevention Evaluation Study Investigators.  Effects of an angiotensin-converting-enzyme inhibitor, ramipril, on cardiovascular events in high-risk patients.  N Engl J Med. 2000;342:145-15310639539Google ScholarCrossref
SHEP Cooperative Research Group.  Prevention of stroke by antihypertensive drug treatment in older persons with isolated systolic hypertension: final results of the Systolic Hypertension in the Elderly Program (SHEP).  JAMA. 1991;265:3255-32642046107Google ScholarCrossref
The Writing Group for the PEPI Trial.  Effects of estrogen or estrogen/progestin regimens on heart disease risk factors in postmenopausal women.  JAMA. 1995;273:199-2087807658Google ScholarCrossref
Executive Committee for the Asymptomatic Carotid Atherosclerosis Study.  Endarterectomy for asymptomatic carotid artery stenosis.  JAMA. 1995;273:1421-14287723155Google ScholarCrossref
Hulley S, Grady D, Bush T.  et al. Heart and Estrogen/progestin Replacement Study (HERS) Research Group.  Randomized trial of estrogen plus progestin for secondary prevention of coronary heart disease in postmenopausal women.  JAMA. 1998;280:605-6139718051Google ScholarCrossref
Downs JR, Clearfield M, Weis S.  et al.  Primary prevention of acute coronary events with lovastatin in men and women with average cholesterol levels: results of AFCAPS/TexCAPS.  JAMA. 1998;279:1615-16229613910Google ScholarCrossref
Rossouw JE, Anderson GL, Prentice RL.  et al.  Risks and benefits of estrogen plus progestin in healthy postmenopausal women: principal results from the Women's Health Initiative randomized controlled trial.  JAMA. 2002;288:321-33312117397Google ScholarCrossref
MRC Vitamin Study Research Group.  Prevention of neural tube defects: results of the Medical Research Council Vitamin Study.  Lancet. 1991;338:131-1371677062Google ScholarCrossref
Hertog MG, Feskens EJ, Hollman PC, Katan MB, Kromhout D. Dietary antioxidant flavonoids and risk of coronary heart disease: the Zutphen Elderly Study.  Lancet. 1993;342:1007-10118105262Google ScholarCrossref
 Randomised trial of cholesterol lowering in 4444 patients with coronary heart disease: the Scandinavian Simvastatin Survival Study (4S).  Lancet. 1994;344:1383-13897968073Google Scholar
CAPRIE Steering Committee.  A randomised, blinded, trial of clopidogrel vs aspirin in patients at risk of ischaemic events (CAPRIE).  Lancet. 1996;348:1329-13398918275Google ScholarCrossref
Stephens NG, Parsons A, Schofield PM, Kelly F, Cheeseman K, Mitchinson MJ. Randomised controlled trial of vitamin E in patients with coronary disease: Cambridge Heart Antioxidant Study (CHAOS).  Lancet. 1996;347:781-7868622332Google ScholarCrossref
Hansson L, Zanchetti A, Carruthers SG.  et al. HOT Study Group.  Effects of intensive blood-pressure lowering and low-dose aspirin in patients with hypertension: principal results of the Hypertension Optimal Treatment (HOT) randomised trial.  Lancet. 1998;351:1755-17629635947Google ScholarCrossref
Poynard T, Marcellin P, Lee SS.  et al. International Hepatitis Interventional Therapy Group (IHIT).  Randomised trial of interferon alpha-2b plus ribavirin for 48 weeks or for 24 weeks vs interferon alpha-2b plus placebo for 48 weeks for treatment of chronic infection with hepatitis C virus.  Lancet. 1998;352:1426-14329807989Google ScholarCrossref
UK Prospective Diabetes Study (UKPDS) Group.  Effect of intensive blood-glucose control with metformin on complications in overweight patients with type 2 diabetes (UKPDS 34).  Lancet. 1998;352:854-8659742977Google ScholarCrossref
 The Cardiac Insufficiency Bisoprolol Study II (CIBIS-II): a randomised trial.  Lancet. 1999;353:9-1310023943Google ScholarCrossref
Castaigne S, Cjomienne C, Daniel MT.  et al.  All-trans-retinoic acid as a differentiation therapy for acute promyelocytic leukemia, 1: clinical results.  Blood. 1990;76:1704-17092224119Google Scholar
Fisher B, Costantino JP, Wickerham DL.  et al.  Tamoxifen for prevention of breast cancer: report of the National Surgical Adjuvant Breast and Bowel Project P-1 Study.  J Natl Cancer Inst. 1998;90:1371-13889747868Google ScholarCrossref
Ioannidis JP, Cappelleri JC, Lau J.  et al.  Early or deferred zidovudine therapy in HIV-infected patients without an AIDS-defining illness.  Ann Intern Med. 1995;122:856-8667741372Google ScholarCrossref
Law MR, Wald NJ, Rudnicka AR. Quantifying effect of statins on low density lipoprotein cholesterol, ischaemic heart disease, and stroke: systematic review and meta-analysis.  BMJ. 2003;326:142312829554Google ScholarCrossref
Gill S, Loprinzi CL, Sargent DJ.  et al.  Pooled analysis of fluorouracil-based adjuvant therapy for stage II and III colon cancer: who benefits and by how much?  J Clin Oncol. 2004;22:1797-180615067028Google ScholarCrossref
Rothwell PM, Eliasziw M, Gutnikov SA.  et al.  Analysis of pooled data from the randomised controlled trials of endarterectomy for symptomatic carotid stenosis.  Lancet. 2003;361:107-11612531577Google ScholarCrossref
McCloskey RV, Straube RC, Sanders C, Smith SM, Smith CR.CHESS Trial Study Group.  Treatment of septic shock with human monoclonal antibody HA-1A: a randomized, double-blind, placebo-controlled trial.  Ann Intern Med. 1994;121:1-58198341Google ScholarCrossref
Garg R, Yusuf S.Collaborative Group on ACE Inhibitor Trials.  Overview of randomized trials of angiotensin-converting enzyme inhibitors on mortality and morbidity in patients with heart failure.  JAMA. 1995;273:1450-14567654275Google ScholarCrossref
Rodrigues EJ, Eisenberg MJ, Pilote L. Effects of early and late administration of angiotensin-converting enzyme inhibitors on mortality after myocardial infarction.  Am J Med. 2003;115:473-47914563504Google ScholarCrossref
Cucherat M, Bonnefoy E, Tremeau G. Primary angioplasty vs intravenous thrombolysis for acute myocardial infarction.  Cochrane Database Syst Rev. 2003;3:CD00156012917910Google Scholar
Yusuf S, Dagenais G, Pogue J, Bosch J, Sleight P.The Heart Outcomes Prevention Evaluation Study Investigators.  Vitamin E supplementation and cardiovascular events in high-risk patients.  N Engl J Med. 2000;342:154-16010639540Google ScholarCrossref
Sokol J, Jacobs SE, Bohn D. Inhaled nitric oxide for acute hypoxemic respiratory failure in children and adults.  Cochrane Database Syst Rev. 2003;1:CD00278712535438Google Scholar
Karvouni E, Katritsis DG, Ioannidis JP. Intravenous glycoprotein IIb/IIIa receptor antagonists reduce mortality after percutaneous coronary interventions.  J Am Coll Cardiol. 2003;41:26-3212570940Google ScholarCrossref
Brophy JM, Belisle P, Joseph L. Evidence for use of coronary stents: a hierarchical bayesian meta-analysis.  Ann Intern Med. 2003;138:777-78612755549Google ScholarCrossref
Hacke W, Donnan G, Fieschi C.  et al.  Association of outcome with early stroke treatment: pooled analysis of ATLANTIS, ECASS, and NINDS rt-PA stroke trials.  Lancet. 2004;363:768-77415016487Google ScholarCrossref
Brophy JM, Joseph L, Rouleau JL. Beta-blockers in congestive heart failure: a Bayesian meta-analysis.  Ann Intern Med. 2001;134:550-56011281737Google ScholarCrossref
Yazdanpanah Y, Sissoko D, Egger M, Mouton Y, Zwahlen M, Chene G. Clinical efficacy of antiretroviral combination therapy based on protease inhibitors or non-nucleoside analogue reverse transcriptase inhibitors: indirect comparison of controlled trials.  BMJ. 2004;328:24914742351Google ScholarCrossref
Kjaergard LL, Krogsgaard K, Gluud C. Interferon alfa with or without ribavirin for chronic hepatitis C: systematic review of randomised trials.  BMJ. 2001;323:1151-115511711405Google ScholarCrossref
Staessen JA, Gasowski J, Wang JG.  et al.  Risks of untreated and treated isolated systolic hypertension in the elderly: meta-analysis of outcome trials.  Lancet. 2000;355:865-87210752701Google ScholarCrossref
Benavente O, Moher D, Pham B. Carotid endarterectomy for asymptomatic carotid stenosis: a meta-analysis.  BMJ. 1998;317:1477-14809831572Google ScholarCrossref
Huxley RR, Neil HA. The relation between dietary flavonol intake and coronary heart disease mortality: a meta-analysis of prospective cohort studies.  Eur J Clin Nutr. 2003;57:904-90812879084Google ScholarCrossref
Tallman MS, Andersen JW, Schiffer CA.  et al.  All-trans-retinoic acid in acute promyelocytic leukemia.  N Engl J Med. 1997;337:1021-10289321529Google ScholarCrossref
Cuzick J, Powles T, Veronesi U.  et al.  Overview of the main outcomes in breast-cancer prevention trials.  Lancet. 2003;361:296-30012559863Google ScholarCrossref
Watts NB, Harris ST, Genant HK.  et al.  Intermittent cyclical etidronate treatment of postmenopausal osteoporosis.  N Engl J Med. 1990;323:73-792113611Google ScholarCrossref
Smith P, Arnesen H, Holme I. The effect of warfarin on mortality and reinfarction after myocardial infarction.  N Engl J Med. 1990;323:147-1522194126Google ScholarCrossref
Hussey GD, Klein M. A randomized, controlled trial of vitamin A in children with severe measles.  N Engl J Med. 1990;323:160-1642194128Google ScholarCrossref
Smith DW, Frankel LR, Mathers LH, Tang AT, Ariagno RL, Prober CG. A controlled trial of aerosolized ribavirin in infants receiving mechanical ventilation for severe respiratory syncytial virus infection.  N Engl J Med. 1991;325:24-291904551Google ScholarCrossref
The National Institute of Child Health and Human Developments Intravenous Immunoglobulin Study Group.  Intravenous immune globulin for the prevention of bacterial infections in children with symptomatic human immunodeficiency virus infection.  N Engl J Med. 1991;325:73-801675763Google ScholarCrossref
Green DM, Zevon MA, Lowrie G, Seigelstein N, Hall B. Congenital anomalies in children of patients who received chemotherapy for cancer in childhood and adolescence.  N Engl J Med. 1991;325:141-1462052058Google ScholarCrossref
Bradley JD, Brandt KD, Katz BP, Kalasinski LA, Ryan SI. Comparison of an antiinflammatory dose of ibuprofen, an analgesic dose of ibuprofen, and acetaminophen in the treatment of patients with osteoarthritis of the knee.  N Engl J Med. 1991;325:87-912052056Google ScholarCrossref
Crawford J, Ozer H, Stoller R.  et al.  Reduction by granulocyte colony-stimulating factor of fever and neutropenia induced by chemotherapy in patients with small-cell lung cancer.  N Engl J Med. 1991;325:164-1701711156Google ScholarCrossref
Maggioni AP, Franzosi MG, Santoro E, White H, Van de Werf F, Tognoni G.Gruppo Italiano per lo Studio della Sopravvivenza nell'Infarto Miocardico II (GISSI-2) and The International Study Group.  The risk of stroke in patients with acute myocardial infarction after thrombolytic and antithrombotic treatment.  N Engl J Med. 1992;327:1-61598096Google ScholarCrossref
Packer M, Gheorghiade M, Young JB.  et al. RADIANCE Study.  Withdrawal of digoxin from patients with chronic heart failure treated with angiotensin-converting enzyme inhibitors.  N Engl J Med. 1993;329:1-78505940Google ScholarCrossref
Jenkins DJ, Wolever TM, Rao AV.  et al.  Effect on blood lipids of very high intakes of fiber in diets low in saturated fat and cholesterol.  N Engl J Med. 1993;329:21-268389421Google ScholarCrossref
Rosman NP, Colton T, Labazzo J.  et al.  A controlled trial of diazepam administered during febrile illnesses to prevent recurrence of febrile seizures.  N Engl J Med. 1993;329:79-848510706Google ScholarCrossref
Hunter DJ, Manson JE, Colditz GA.  et al.  A prospective study of the intake of vitamins C, E, and A and the risk of breast cancer.  N Engl J Med. 1993;329:234-2408292129Google ScholarCrossref
Ponsonby AL, Dwyer T, Gibbons LE, Cochrane JA, Wang YG. Factors potentiating the risk of sudden infant death syndrome associated with the prone position.  N Engl J Med. 1993;329:377-3828326970Google ScholarCrossref
Winterkorn JM, Kupersmith MJ, Wirtschafter JD, Forman S. Treatment of vasospastic amaurosis fugax with calcium-channel blockers.  N Engl J Med. 1993;329:396-3988326973Google ScholarCrossref
Greenberg ER, Baron JA, Tosteson TD.  et al. Polyp Prevention Study Group.  A clinical trial of antioxidant vitamins to prevent colorectal adenoma.  N Engl J Med. 1994;331:141-1478008027Google ScholarCrossref
Klassen TP, Feldman ME, Watters LK, Sutcliffe T, Rowe PC. Nebulized budesonide for children with mild-to-moderate croup.  N Engl J Med. 1994;331:285-2898022437Google ScholarCrossref
Jain JK, Mishell DR Jr. A comparison of intravaginal misoprostol with prostaglandin E2 for termination of second-trimester pregnancy.  N Engl J Med. 1994;331:290-2938022438Google ScholarCrossref
Loprinzi CL, Michalak JC, Quella SK.  et al.  Megestrol acetate for the prevention of hot flashes.  N Engl J Med. 1994;331:347-3528028614Google ScholarCrossref
O'Connell MJ, Martenson JA, Wieand HS.  et al.  Improving adjuvant therapy for rectal cancer by combining protracted-infusion fluorouracil with radiation therapy after curative surgery.  N Engl J Med. 1994;331:502-5078041415Google ScholarCrossref
Suarez FL, Savaiano DA, Levitt MD. A comparison of symptoms after the consumption of milk or lactose-hydrolyzed milk by people with self-reported severe lactose intolerance.  N Engl J Med. 1995;333:1-47776987Google ScholarCrossref
Singh SN, Fletcher RD, Fisher SG.  et al. Survival Trial of Antiarrhythmic Therapy in Congestive Heart Failure.  Amiodarone in patients with congestive heart failure and asymptomatic ventricular arrhythmia.  N Engl J Med. 1995;333:77-827539890Google ScholarCrossref
Bhasin S, Storer TW, Berman N.  et al.  The effects of supraphysiologic doses of testosterone on muscle size and strength in normal men.  N Engl J Med. 1996;335:1-78637535Google ScholarCrossref
Stegeman CA, Tervaert JW, de Jong PE, Kallenberg CG.Dutch Co-Trimoxazole Wegener Study Group.  Trimethoprim-sulfamethoxazole (co-trimoxazole) for the prevention of relapses of Wegener's granulomatosis.  N Engl J Med. 1996;335:16-208637536Google ScholarCrossref
van Hensbroek MB, Onyiorah E, Jaffar S.  et al.  A trial of artemether or quinine in children with cerebral malaria.  N Engl J Med. 1996;335:69-758649492Google ScholarCrossref
van der Horst CM, Saag MS, Cloud GA.  et al. National Institute of Allergy and Infectious Diseases Mycoses Study Group and AIDS Clinical Trials Group.  Treatment of cryptococcal meningitis associated with the acquired immunodeficiency syndrome.  N Engl J Med. 1997;337:15-219203426Google ScholarCrossref
Levine RJ, Hauth JC, Curet LB.  et al.  Trial of calcium to prevent preeclampsia.  N Engl J Med. 1997;337:69-769211675Google ScholarCrossref
Musch DC, Martin DF, Gordon JF, Davis MD, Kuppermann BD.the Ganciclovir Implant Study Group.  Treatment of cytomegalovirus retinitis with a sustained-release ganciclovir implant.  N Engl J Med. 1997;337:83-909211677Google ScholarCrossref
Adams RJ, McKie VC, Hsu L.  et al.  Prevention of a first stroke by transfusions in children with sickle cell anemia and abnormal results on transcranial Doppler ultrasonography.  N Engl J Med. 1998;339:5-119647873Google ScholarCrossref
Stefanick ML, Mackey S, Sheehan M, Ellsworth N, Haskell WL, Wood PD. Effects of diet and exercise in men and postmenopausal women with low levels of HDL cholesterol and high levels of LDL cholesterol.  N Engl J Med. 1998;339:12-209647874Google ScholarCrossref
Brisinda G, Maria G, Bentivoglio AR, Cassetta E, Gui D, Albanese A. A comparison of injections of botulinum toxin and topical nitroglycerin ointment for the treatment of chronic anal fissure.  N Engl J Med. 1999;341:65-6910395629Google ScholarCrossref
Tepel M, van der Giet M, Schwarzfeld C, Laufer U, Liermann D, Zidek W. Prevention of radiographic-contrast-agent-induced reductions in renal function by acetylcysteine.  N Engl J Med. 2000;343:180-18410900277Google ScholarCrossref
Levine AM, Wernz JC, Kaplan L.  et al.  Low-dose chemotherapy with central nervous system prophylaxis and zidovudine maintenance in AIDS-related lymphoma: a prospective multi-institutional trial.  JAMA. 1991;266:84-881710673Google ScholarCrossref
Freeman EW, Rickels K, Sondheimer SJ, Polansky M. A double-blind trial of oral progesterone, alprazolam, and placebo in treatment of severe premenstrual syndrome.  JAMA. 1995;274:51-577791258Google ScholarCrossref
Williams-Russo P, Sharrock NE, Mattis S, Szatrowski TP, Charlson ME. Cognitive effects after epidural vs general anesthesia in older adults: a randomized trial.  JAMA. 1995;274:44-507791257Google ScholarCrossref
Meduri GU, Headley AS, Golden E.  et al.  Effect of prolonged methylprednisolone therapy in unresolving acute respiratory distress syndrome: a randomized controlled trial.  JAMA. 1998;280:159-1659669790Google ScholarCrossref
Gulick RM, Mellors JW, Havlir D.  et al.  Simultaneous vs sequential initiation of therapy with indinavir, zidovudine, and lamivudine for HIV-1 infection: 100-week follow-up.  JAMA. 1998;280:35-419660361Google ScholarCrossref
Grady D, Herrington D, Bittner V.  et al.  Cardiovascular disease outcomes during 6.8 years of hormone therapy: Heart and Estrogen/progestin Replacement Study follow-up (HERS II).  JAMA. 2002;288:49-5712090862Google ScholarCrossref
Grantham-McGregor SM, Powell CA, Walker SP, Himes JH. Nutritional supplementation, psychosocial stimulation, and mental development of stunted children: the Jamaican Study.  Lancet. 1991;338:1-51676083Google ScholarCrossref
Ghana VAST Study Team.  Vitamin A supplementation in northern Ghana: effects on clinic attendances, hospital admissions, and child mortality.  Lancet. 1993;342:7-128100345Google ScholarCrossref
Sipe JC, Romine JS, Koziol JA, McMillan R, Zyroff J, Beutler E. Cladribine in treatment of chronic progressive multiple sclerosis.  Lancet. 1994;344:9-137912347Google ScholarCrossref
Karonga Prevention Trial Group.  Randomised controlled trial of single BCG, repeated BCG, or combined BCG and killed Mycobacteriumleprae vaccine for prevention of leprosy and tuberculosis in Malawi.  Lancet. 1996;348:17-248691924Google ScholarCrossref
Waldo AL, Camm AJ, deRuyter H.  et al. the SWORD Investigators.  Effect of d-sotalol on mortality in patients with left ventricular dysfunction after recent and remote myocardial infarction.  Lancet. 1996;348:7-128691967Google ScholarCrossref
Veronesi U, Maisonneuve P, Costa A.  et al. Italian Tamoxifen Prevention Study.  Prevention of breast cancer with tamoxifen: preliminary findings from the Italian randomised trial among hysterectomised women.  Lancet. 1998;352:93-979672273Google Scholar
The EPISTENT Investigators.  Randomised placebo-controlled and balloon-angioplasty-controlled trial to assess safety of coronary stenting with use of platelet glycoprotein-IIb/IIIa blockade.  Lancet. 1998;352:87-929672272Google Scholar
CLIP Group (Cancer of the Liver Italian Programme).  Tamoxifen in treatment of hepatocellular carcinoma: a randomised controlled trial.  Lancet. 1998;352:17-209800740Google ScholarCrossref
Torp-Pedersen C, Kober L.TRACE Study Group.  Effect of ACE inhibitor trandolapril on life expectancy of patients with reduced left-ventricular function after acute myocardial infarction.  Lancet. 1999;354:9-1210406358Google ScholarCrossref
Negrin RS, Haeuber DH, Nagler A.  et al.  Maintenance treatment of patients with myelodysplastic syndromes using recombinant human granulocyte colony-stimulating factor.  Blood. 1990;76:36-431694702Google Scholar
Budman DR, Berry DA, Cirrincione CT.  et al. the Cancer and Leukemia Group B.  Dose and dose intensity as determinants of outcome in the adjuvant treatment of breast cancer.  J Natl Cancer Inst. 1998;90:1205-12119719081Google ScholarCrossref
Veronesi U, De Palo G, Marubini E.  et al.  Randomized trial of fenretinide to prevent second breast malignancy in women with early breast cancer.  J Natl Cancer Inst. 1999;91:1847-185610547391Google ScholarCrossref
Rice GP, Filippi M, Comi G.Cladribine MRI Study Group.  Cladribine and progressive MS: clinical and MRI outcomes of a multicenter controlled trial.  Neurology. 2000;54:1145-115510720289Google ScholarCrossref
Ventre K, Randolph A. Ribavirin for respiratory syncytial virus infection of the lower respiratory tract in infants and young children.  Cochrane Database Syst Rev. 2004;4:CD00018115494991Google Scholar
Rantala H, Tarkka R, Uhari M. A meta-analytic review of the preventive treatment of recurrences of febrile seizures.  J Pediatr. 1997;131:922-9259427902Google ScholarCrossref
Creery D, Mikrogianakis A. Sudden infant death syndrome.  Clin Evid. 2003;10:457-46715555102Google Scholar
Russell K, Wiebe N, Saenz A.  et al.  Glucocorticoids for croup.  Cochrane Database Syst Rev. 2004;1:CD00195514973975Google Scholar
Makhlouf AM, Al-Hussaini TK, Habib DM, Makarem MH. Second-trimester pregnancy termination: comparison of three different methods.  J Obstet Gynaecol. 2003;23:407-41112881083Google ScholarCrossref
Nelson R. A systematic review of medical therapy for anal fissure.  Dis Colon Rectum. 2004;47:422-43114994109Google ScholarCrossref
Nallamothu BK, Shojania KG, Saint S.  et al.  Is acetylcysteine effective in preventing contrast-related nephropathy? a meta-analysis.  Am J Med. 2004;117:938-94715629733Google ScholarCrossref
Grantham-McGregor SM, Walker SP, Chang SM, Powell CA. Effects of early childhood supplementation with and without stimulation on later development in stunted Jamaican children.  Am J Clin Nutr. 1997;66:247-2539250101Google Scholar
Adachi JD, Roux C, Pitt PI.  et al.  A pooled data analysis on the use of intermittent cyclical etidronate therapy for the prevention and treatment of corticosteroid induced bone loss.  J Rheumatol. 2000;27:2424-243111036840Google Scholar
Anand SS, Yusuf S. Oral anticoagulant therapy in patients with coronary artery disease: a meta-analysis.  JAMA. 1999;282:2058-206710591389Google ScholarCrossref
Fawzi WW, Chalmers TC, Herrera MG, Mosteller F. Vitamin A supplementation and child mortality: a meta-analysis.  JAMA. 1993;269:898-9038426449Google ScholarCrossref
Spector SA, Gelber RD, McGrath N.  et al. Pediatric AIDS Clinical Trials Group.  A controlled trial of intravenous immune globulin for the prevention of serious bacterial infections in children receiving zidovudine for advanced human immunodeficiency virus infection.  N Engl J Med. 1994;331:1181-11877935655Google ScholarCrossref
Lyman GH, Kuderer NM, Djulbegovic B. Prophylactic granulocyte colony-stimulating factor in patients receiving dose-intensive cancer chemotherapy: a meta-analysis.  Am J Med. 2002;112:406-41111904116Google ScholarCrossref
Dundar Y, Hill R, Dickson R, Walley T. Comparative efficacy of thrombolytics in acute myocardial infarction: a systematic review.  QJM. 2003;96:103-11312589008Google ScholarCrossref
Hood WB Jr, Dans AL, Guyatt GH, Jaeschke R, McMurray JJ. Digitalis for treatment of congestive heart failure in patients in sinus rhythm.  Cochrane Database Syst Rev. 2004;2:CD00290115106182Google Scholar
Anderson JW, Randles KM, Kendall CW, Jenkins DJ. Carbohydrate and fiber recommendations for individuals with diabetes: a quantitative assessment and meta-analysis of the evidence.  J Am Coll Nutr. 2004;23:5-1714963049Google ScholarCrossref
Giorgi A, Weatherby RP, Murphy PW. Muscular strength, body composition and health responses to the use of testosterone enanthate: a double blind study.  J Sci Med Sport. 1999;2:341-35510710012Google ScholarCrossref
Artemether-Quinine Meta-analysis Study Group.  A meta-analysis using individual patient data of trials comparing artemether with quinine in the treatment of severe falciparum malaria.  Trans R Soc Trop Med Hyg. 2001;95:637-65011816438Google ScholarCrossref
Saag MS, Cloud GA, Graybill JR.  et al. National Institute of Allergy and Infectious Diseases Mycoses Study Group.  A comparison of itraconazole vs fluconazole as maintenance therapy for AIDS-associated cryptococcal meningitis.  Clin Infect Dis. 1999;28:291-29610064246Google ScholarCrossref
Martin DF, Kuppermann BD, Wolitz RA, Palestine AG, Li H, Robinson CA.Roche Ganciclovir Study Group.  Oral ganciclovir for patients with cytomegalovirus retinitis treated with a ganciclovir implant.  N Engl J Med. 1999;340:1063-107010194235Google ScholarCrossref
Nieman DC, Brock DW, Butterworth D, Utter AC, Nieman CC. Reducing diet and/or exercise training decreases the lipid and lipoprotein risk factors of moderately obese women.  J Am Coll Nutr. 2002;21:344-35012166532Google ScholarCrossref
Kaplan LD, Straus DJ, Testa MA.  et al. National Institute of Allergy and Infectious Diseases AIDS Clinical Trials Group.  Low-dose compared with standard-dose m-BACOD chemotherapy for non-Hodgkin's lymphoma associated with human immunodeficiency virus infection.  N Engl J Med. 1997;336:1641-16489171066Google ScholarCrossref
Diegoli MS, da Fonseca AM, Diegoli CA, Pinotti JA. A double-blind trial of four medications to treat severe premenstrual syndrome.  Int J Gynaecol Obstet. 1998;62:63-679722128Google ScholarCrossref
Flather MD, Yusuf S, Kober L.  et al. ACE-Inhibitor Myocardial Infarction Collaborative Group.  Long-term ACE-inhibitor therapy in patients with heart failure or left-ventricular dysfunction: a systematic overview of data from individual patients.  Lancet. 2000;355:1575-158110821360Google ScholarCrossref
Ossenkoppele GJ, van der Holt B, Verhoef GE.  et al. Dutch-Belgian Hemato-Oncology Cooperative Group.  A randomized study of granulocyte colony-stimulating factor applied during and after chemotherapy in patients with poor risk myelodysplastic syndromes: a report from the HOVON Cooperative Group.  Leukemia. 1999;13:1207-121310450748Google ScholarCrossref
Citron ML, Berry DA, Cirrincione C.  et al.  Randomized trial of dose-dense vs conventionally scheduled and sequential vs concurrent combination chemotherapy as postoperative adjuvant treatment of node-positive primary breast cancer: first report of Intergroup Trial C9741/Cancer and Leukemia Group B Trial 9741.  J Clin Oncol. 2003;21:1431-143912668651Google ScholarCrossref
Lau J, Ioannidis JP, Schmid CH. Summing up evidence: one answer is not enough.  Lancet. 1998;351:123-1279439507Google ScholarCrossref
Ioannidis JP, Cappelleri JC, Lau J. Issues in comparisons between meta-analyses and large trials.  JAMA. 1998;279:1089-10939546568Google ScholarCrossref
Fleming TR, DeMets DL. Surrogate end points in clinical trials: are we being misled?  Ann Intern Med. 1996;125:605-6138815760Google ScholarCrossref
Dickersin K, Min YI. Publication bias: the problem that won't go away.  Ann N Y Acad Sci. 1993;703:135-1468192291Google ScholarCrossref
Easterbrook PJ, Berlin JA, Gopalan R, Matthews DR. Publication bias in clinical research.  Lancet. 1991;337:867-8721672966Google ScholarCrossref
Stern JM, Simes RJ. Publication bias: evidence of delayed publication in a cohort study of clinical research projects.  BMJ. 1997;315:640-6459310565Google ScholarCrossref
Ioannidis JP. Effect of the statistical significance of results on the time to completion and publication of randomized efficacy trials.  JAMA. 1998;279:281-2869450711Google ScholarCrossref
Wheatley K, Clayton D. Be skeptical about unexpected large treatment effects: the case of an MRC AML12 randomization.  Control Clin Trials. 2003;24:66-7012559643Google ScholarCrossref
Poynard T, Munteanu M, Ratziu V.  et al.  Truth survival in clinical research: an evidence-based requiem?  Ann Intern Med. 2002;136:888-89512069563Google ScholarCrossref
DeAngelis C, Drazen JM, Frizelle FA.  et al.  Clinical trial registration: a statement from the International Committee of Medical Journal Editors.  N Engl J Med. 2004;351:1250-125115356289Google ScholarCrossref
Original Contribution
July 13, 2005

Contradicted and Initially Stronger Effects in Highly Cited Clinical Research

Author Affiliations

Author Affiliations: Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece, and the Institute for Clinical Research and Health Policy Studies, Department of Medicine, Tufts-New England Medical Center, Boston, Mass.

JAMA. 2005;294(2):218-228. doi:10.1001/jama.294.2.218

Context Controversy and uncertainty ensue when the results of clinical research on the effectiveness of interventions are subsequently contradicted. Controversies are most prominent when high-impact research is involved.

Objectives To understand how frequently highly cited studies are contradicted or find effects that are stronger than in other similar studies and to discern whether specific characteristics are associated with such refutation over time.

Design All original clinical research studies published in 3 major general clinical journals or high-impact-factor specialty journals in 1990-2003 and cited more than 1000 times in the literature were examined.

Main Outcome Measure The results of highly cited articles were compared against subsequent studies of comparable or larger sample size and similar or better controlled designs. The same analysis was also performed comparatively for matched studies that were not so highly cited.

Results Of 49 highly cited original clinical research studies, 45 claimed that the intervention was effective. Of these, 7 (16%) were contradicted by subsequent studies, 7 others (16%) had found effects that were stronger than those of subsequent studies, 20 (44%) were replicated, and 11 (24%) remained largely unchallenged. Five of 6 highly-cited nonrandomized studies had been contradicted or had found stronger effects vs 9 of 39 randomized controlled trials (P = .008). Among randomized trials, studies with contradicted or stronger effects were smaller (P = .009) than replicated or unchallenged studies although there was no statistically significant difference in their early or overall citation impact. Matched control studies did not have a significantly different share of refuted results than highly cited studies, but they included more studies with “negative” results.

Conclusions Contradiction and initially stronger effects are not unusual in highly cited research of clinical interventions and their outcomes. The extent to which high citations may provoke contradictions and vice versa needs more study. Controversies are most common with highly cited nonrandomized studies, but even the most highly cited randomized trials may be challenged and refuted over time, especially small ones.