Background
In the last decades, studies have estimated the upper gastrointestinal tract bleeding/perforation (UGIB) risk associated with individual nonsteroidal anti-inflammatory drugs (NSAIDs). Later analyses have also included the effect of patterns of NSAID use, risk factors for UGIB, and modifiers of NSAID effect.
Methods
Systematic review of case-control and cohort studies on serious gastrointestinal tract complications and nonaspirin NSAIDs published between 1990 and 1999 using MEDLINE. Eighteen original studies were selected according to predefined criteria. Two researchers extracted the data independently. Pooled relative risk estimates were calculated according to subject and exposure characteristics. Heterogeneity of effects was tested and reasons for heterogeneity were considered.
Results
Advanced age, history of peptic ulcer disease, and being male were risk factors for UGIB. Nonsteroidal anti-inflammatory drug users with advanced age or a history of peptic ulcer had the highest absolute risks. The pooled relative risk of UGIB after exposure to NSAIDs was 3.8 (95% confidence interval, 3.6-4.1). The increased risk was maintained during treatment and returned to baseline once treatment was stopped. A clear dose response was observed. There was some variation in risk between individual NSAIDs, though these differences were markedly attenuated when comparable daily doses were considered.
Conclusions
The elderly and patients with a history of peptic ulcer could benefit the most from a reduction in NSAID gastrotoxicity. Whenever possible, physicians may wish to recommend lower doses to reduce the UGIB risk associated with all individual NSAIDs, especially in the subgroup of patients with the greatest background risk.
NONASPIRIN nonsteriodal anti-inflammatory drugs (NSAIDs) have been shown to increase the risk of upper gastrointestinal tract bleeding/perforation (UGIB).1-24 Because of the widespread use of NSAIDs as analgesic, anti-inflammatory, and antipyretic drugs, their serious upper gastrointestinal tract complications constitute a major public health concern. To reduce the morbidity associated with NSAIDs, it will be necessary to establish specific estimates for individual drugs and individual groups of patients with different risk profiles.
Before 1990, observational epidemiologic studies addressed the association between NSAIDs, studied as a therapeutic class, and UGIB.1-4 However, very few publications presented data on individual NSAIDs or took into account other risk factors. In the last decade, studies have provided not only overall risk estimates but also specific risks for individual NSAIDs and the effects of patterns of use.5-25 We present a summary of the main results from observational epidemiologic studies published from 1990 to 1999.
We conducted a MEDLINE search covering the period from 1990 to 1999, searching for the terms "anti-inflammatory agents" (both overall and for specific drug names); "non-steroidal"; "adverse effects"; "toxicity," "peptic ulcer," "stomach ulcer," and "gastrointestinal diseases." The search was restricted to human studies on adults. To be included in the analysis, articles had to be case-control or cohort studies on nonaspirin NSAID use and UGIB (defined as bleeding, perforation, or other serious upper gastrointestinal tract events resulting in hospitalization or visit to specialist), and the articles had to provide enough data to estimate the relative risk (RR) comparing NSAID users with nonusers (Table 1).
We identified 853 entries and examined their abstracts. When the study abstract had no clear reason for exclusion, the full article was obtained. We also reviewed the references of selected articles and previous reviews related to NSAIDs and upper gastrointestinal tract disorders. A total of 32 original research articles were considered. Predefined inclusion criteria were applied, and data extraction was done independently by the 2 authors using a standardized data extraction form. Information on study methods and objective quality-related characteristics was collected and entered into a database. The list of characteristics was based on literature about the methods of epidemiologic studies in general and on previous meta-analyses on NSAIDs and UGIB (Table 1).2,3,26 Decisions regarding inclusion of studies and data extraction were reached by consensus. Eleven studies were rejected, for the following reasons: absence of control group (n = 3), the outcome was identification of gastrointestinal tract lesions with endoscopy (n = 1) or bleeding of esophageal varices (n = 1), the results for NSAIDs included aspirin (n = 5), or methodological inconsistencies (n = 1).27 When 2 articles reported results from the same study population, the more recent version was chosen. However, if the earlier version provided additional subanalyses, they were considered. Of the 21 published epidemiologic studies specifically addressing UGIB and exposure to nonaspirin NSAIDs, one reported results from the same study population as a more recent study,22 one reported excess rates while a previous version presented relative risks,23 and one was based on a subsample of a previous publication.24 Hence, the final number of articles that underwent meta-analysis was 18. We did not contact study authors for additional data.
To determine whether it was appropriate to pool the individual results into one common summary measure, the heterogeneity of effects between studies was analyzed. Since the number of studies that underwent meta-analysis was relatively small, we used a parametric bootstrap version of the DerSimonian and Laird test statistic for heterogeneity (Q), with 1000 replications.28 We calculated a summary RR and 95% confidence interval (CI),29 weighing study estimates by the inverse of the variance and estimating linear predictors for the log effect measure.30 The odds ratio from case-control studies was assumed to provide a valid estimate of the RR.31 When possible, RRs for subgroups of interest were extracted or computed from raw data provided in the publication. Definitions of categories for these purposes reflected those used in the original studies. When heterogeneity between study effects was significant, we examined the possible relation of disagreement among results to a variety of study characteristics by means of meta-regression.32 We explored potential publication bias qualitatively using a funnel plot.33
The individual RRs of UGIB associated with NSAID use in the 18 studies identified are shown in Table 2. The pooled RR was 3.8 (95% CI, 3.6-4.1). Statistically significant heterogeneity was found between effect measures obtained in different studies. We first explore sources of variability among results and then present risks according to patient characteristics, patterns of NSAID use, and individual drugs.
Among the 18 studies considered, 3 were cohort15,17,19 and 15 case-control studies (10 included hospital controls6-12,14,20,25 and 5 used controls randomly selected from data files5,13,16,18,21). Four case-control studies were nested in a well-defined cohort (none used hospital controls).5,13,18,21 Eleven case-control studies used matched designs.5-12,14,16,25 All nested case-control and cohort studies (n = 7) used computerized records as the source of exposure and outcome information, while 9 nonnested case-control studies were based on patient interview/hospital diagnoses and 2 on registries.9,16 All patients had been hospitalized (n = 14)5-12,14,17,18,20,21,25 or referred to a specialist for a UGIB (n = 4).13,15,16,19 Nine studies specifically excluded esophageal lesions and only considered lesions located in the stomach or duodenum.6,9,11-14,17,18,21 Many studies had the following exclusion criteria: cancer (n = 10),5-8,10,13,15,18,21,25 esophageal varices (n = 10),5-8,10,13,15,18,21,25 Mallory-Weiss disease (n = 10),6,8,10,11,13-15,18,21,25 alcoholism (n = 6),6,8,11,13,18,21 chronic liver disease (n = 6),6,8,11,13,18,21 and/or coagulopathies (n = 6).6,8,11-13,21
Study design was associated with differences in RRs. Cohort studies and nested case-control studies had a higher summary estimate (RR, 4.2; 95% CI, 3.9-4.5) than nonnested case-control studies (RR, 3.2; 95% CI, 2.9-3.6). The summary RR for studies that limited their case definition to stomach or duodenum lesion was higher (RR, 4.3; 95% CI, 4.0-4.7) than that of studies that accepted esophageal lesions (RR, 3.4; 95% CI, 3.1-3.7).
Regarding quality-related characteristics, all the studies had specific definitions of exposure and outcome and similar ascertainment of compared groups. The 7 studies using computerized records and the 2 studies using registries verified the information by chart review,5,9,13,15-19,21 and all but 2 studies attempted to control for potential confounders.9,25 The most frequent confounders considered were age (n = 15),5-8,10-18,20,21 sex (n = 15),5-8,10-18,20,21 history of ulcer, (n = 8),6,11,13,14,17-19,21 and concomitant medication use (n = 8).5,6,11-13,16-18 Among the 11 matched case-control studies, 6 used statistical analysis for matched data,6,7,10-12,16 3 considered the matching factors in the multivariate model,5,8,14 and 2 did not consider the matching factors during the analysis at all.9,25 Restricting the analysis to those that attempted to control for potential confounders did not materially change the RR (RR, 3.9; 95% CI, 3.6-4.1).
Despite heterogeneity in effects of the studies for certain analyses, the qualitative results were consistent among them. Age13,15,17,18,21 and a history of ulcer6,7,13,17,18,21 were strong predictors of UGIB, and men had a 2-fold greater risk of developing UGIB than women (Table 3).13,15,17,18,21
The impact of NSAIDs on the risk of UGIB was greater in women (RR, 5.1; 95% CI, 4.6-5.7) than in men (RR, 3.5; 95% CI, 3.1-4.0) (Table 4).5-7,10,13,17,18 Nonsteroidal anti-inflammatory drug use was associated with an increased risk across all age groups, with a slight elevation in RR with age.5,6,8,10,12,14,17,22 Assuming a baseline rate of 1 case per 1000 person-years, patients older than 75 years taking NSAIDs would have an absolute incidence rate around 20 UGIB cases per 1000 person-years.18 The RR was greater in subjects without (RR, 5.0; 95% CI, 4.5-5.5) vs with (RR, 2.5; 95% CI, 2.1-3.0) a history of ulcer.6,10,13,17,18,21 However, patients with a previous complicated history of ulcer (bleeding or perforation) would have the greatest absolute risk of UGIB when taking NSAIDs, equivalent to an in cidence rategreater than 30 per 1000 person-years (ie, 1 per 1000 × 15.4 × 2.5).
The RR of UGIB dropped quickly once treatment was stopped. On average, 2 months after the end of therapy the risk returned to the baseline incidence among persons not using NSAIDs.5,13,15,16,18,21 Nonsteroidal anti-inflammatory drug use increased the risk of UGIB among new users and among those already on therapy for several months, at least during the first year of treatment. However, the pooled RR was greater for new users (RR, 5.7; 95% CI, 4.9-6.6).5,10,13,14,18,21 The risk of developing UGIB increased proportionally with NSAID daily dose. The pooled RRs were 3.0 (95% CI, 2.6-3.4) for low, 4.1 (95% CI, 3.6-4.5) for medium, and 6.9 (95% CI, 5.8-8.1) for high doses.5,10,13,14,18,21
Ibuprofen was associated with the lowest risk, followed by diclofenac, sulindac, naproxen sodium, indomethacin, and ketoprofen. Piroxicam had a higher RR, although apazone was the only NSAID with an RR greater than 10.5.6,8,10-16,18,21 In the analysis stratified by daily dose, all individual NSAIDs were associated with an RR between 2 and 4 when administrated at low-medium doses, except piroxicam, with an RR of 5.6. All individual NSAIDs presented a greater risk with increasing dose. Only drugs that appeared in 2 or more studies were included in these dose-response analyses (Table 5).5,13,18,21
Relative risk estimates were practically unchanged when we used random-effect models, and the CIs were only slightly wider. Publication bias is unlikely in this meta-analysis; the plot of sample size vs effect size was shaped like a pyramid, with the apex pointing up around an RR of 4 (Figure 1).
Results of studies published in the 1990s consistently showed that nonaspirin NSAID use is associated with around a 4-fold increased risk of serious upper gastrointestinal tract disease. The elevation in risk is dose dependent, is maintained even after many months of treatment, and disappears completely about 2 months after treatment is stopped. The baseline risk of UGIB increases strongly with age and with the severity of a history of peptic ulcer disease and is slightly higher in men than in women. Nonsteroidal anti-inflammatory drug use increases the risk of UGIB across all age groups, though the effect is slightly stronger for the elderly. The risk associated with NSAID use is stronger for patients without vs with a history of ulcer. However, since the background incidence is higher for patients with a history of ulcer, NSAID users with a history of complicated ulcer have a greater absolute increased risk. When examining the gastrotoxicity of individual NSAIDs, ibuprofen has the lowest RR. However, all NSAIDs are associated with similar increase in risk when administrated at low-medium doses. Certain NSAIDs, such as piroxicam, are not used at low doses (eg, 10 mg) comparable with those of most individual NSAIDs; consequently, the risk of piroxicam at low doses cannot be evaluated. The risk is higher for all individual NSAIDs at anti-inflammatory doses than at analgesic doses.
The present study is consistent with results obtained in previous meta-analyses, which reported global RRs of 4.0,1 2.7,2 and 2.6.3 In those reviews, advanced age and history of gastrointestinal tract events were pointed out as risk factors.2 In another meta-analysis, ibuprofen had the lowest risk and apazone the greatest, with all drugs reflecting a clear dose-response effect.4 Earlier summary estimates were somewhat lower than those obtained with later studies. Restriction to severe gastrointestinal tract complications and improvements in study design, definition of variables, and data collection over time may have reduced patient and exposure misclassification and could explain at least part of such differences.
Original reports disagree on whether the risk associated with NSAIDs diminishes or remains stable during the treatment period. A biological explanation has been proposed to explain a lower gastrotoxic NSAID effect for patients receiving long-term treatment: the gastric mucosa tends to adapt to continuous insults.34 However, methodological pitfalls, such as uncontrolled confounding, might also explain the apparent effect modification over time. Gastrointestinal tract NSAID intolerance in regular users is a reason for NSAID withdrawal, dose reduction, or cotreatment with antiulcer drugs. Because of self-selection (ie, only patients who can tolerate NSAIDs will remain on treatment) or precautions taken to reduce gastrotoxicity, patients undergoing long-term treatments will likely have lower risks associated with NSAID use than those patients who have recently started treatment.35 Control for a history of ulcer will only partially solve the problem. First, not all symptoms that cause cessation of treatment are registered (ie, there will be residual confounding). Second, correct adjustment for time-dependent confounders requires advanced analytical methods not yet widely used.36
Although identified studies agree on an increased risk of UGIB associated with NSAIDs, the size of the reported risks varies. Summary estimates computed from observational studies with discernibly different results have been criticized.29,37,38 We cannot avoid the variability of original results but only try to explain it. Heterogeneity among publications may arise from different sources, including differences in the distribution of effect modifiers across study populations, study design, disease definition, and variation in specific drugs or doses used by the population. Bollini et al3 conducted a meta-analysis to investigate sources of variability among published estimates of the risk of UGIB associated with NSAIDs. Cohort studies included in their analysis provided lower RR values than case-control studies. We found an effect of study design in the opposite direction. Cohort studies and nested case-control studies published in the last decade reported, on average, higher RRs than nonnested case-control studies. The nonnested case-control studies also used interviews as the source of exposure information and hospital patients as controls. The effect of study design on the results might be the result of different sources of information (automated databases vs personal interviews) used in these 2 types of studies or of biases introduced by using hospital controls. Selection bias appears in hospital-based case-control studies when the reason a control is at the hospital is related to the use of NSAIDs or other gastrotoxic drugs, such as prophylactic aspirin. That would result in underestimation of RRs. In addition, disease definition affected the results. Studies excluding gastrointestinal tract diseases with known causes (ie, not due to medication) and focusing on stomach and duodenal lesions had, on average, higher RR estimates.
The quality of the studies may also have affected the results; higher quality was associated with lower risk estimates in studies performed in the 1980s.3 In the present review, the RR estimate was practically identical for the highest-quality studies.5-8,10-21 Some meta-analyses assign quality weights to each of a series of individual studies. Instead, to reduce subjective judgments and to explore the relevance of specific characteristics, we estimated RRs by characteristics related to study quality.32,37-39 Nonetheless, since the evidence was qualitatively consistent across studies, quality scoring might have changed the magnitude of the estimate but not the direction of the association or the practical implications of the results. Moreover, very few studies on NSAIDs and UGIB published in the 1990s had obviously poor quality.
Among the 11 matched case-control studies, only 6 studies used matched analysis. Matching itself does not control for confounding in case-control studies. Instead, it can introduce bias if the matching factor is associated with the exposure. The bias can be removed by stratifying the analysis on matching factors. However, when more than one factor is used to define matched pairs, including the variables in the multivariate analysis may not be enough. Specific methods for matched designs or the inclusion of all matching factors and their interactions in the models is indicated.39 In the present review, results from studies that considered the matching factors in standard multivariate models were not appreciably different from those using the "most valid" statistical analyses.
As with any review, our study shares the limitations of the primary studies plus a potential publication bias (ie, preferential publication of significant associations). Such bias would result in an overestimation of the risks and an exaggeration of effect modifications. However, the number of nonpublished negative studies would need to be huge to counterbalance the positive association found. Moreover, the funnel plot testified against publication bias.
Helicobacter pylori has been identified in the recent decades as a factor in the development of gastrointestinal tract ulcer; whether there is an interaction between H pylori infection and NSAIDs is still being discussed.40 Future research may be able to better identify patients most likely to have adverse effects (gastrointestinal tract, cardiovascular, hepatic, or others) after NSAID therapy. We will also see whether the new selective cyclooxygenase-2 inhibitors are clinically safer and more cost-effective than other therapeutic approaches to reduce toxic effects associated with conventional NSAIDs.
In summary, individuals with advanced age or a history of complicated peptic ulcer disease have a much higher baseline risk of UGIB and the greatest absolute risk when taking NSAIDs. The overall 4-fold increased risk associated with current NSAID use is maintained with treatment and decreases once treatment is stopped. The increased risk is common to all studied NSAIDs and is dose dependent, and consequently speaks forcefully in favor of a class effect. Whenever possible, NSAID therapy should be stopped, or lower effective NSAID doses should be administered in clinical practice to reduce the morbidity associated with all traditional NSAIDs. The above-mentioned patient- and drug-specific risk factors have to be considered to minimize the public health burden associated with NSAID treatment.
Accepted for publication January 1, 2000.
Corresponding author: Sonia Hernández-Díaz, MD, MPH, Department of Epidemiology, Harvard School of Public Health, 677 Huntington Ave, Boston, MA 02115 (e-mail: shernan@hsph.harvard.edu).
1.Hawkey
C Non-steroidal anti-inflammatory drugs and peptic ulcers: facts and figures multiply, but do they add up?
BMJ. 1990;300278- 284
Google ScholarCrossref 2.Gabriel
SEJaakkimainen
LBombardier
C Risk for serious gastrointestinal complications related to use of nonsteroidal anti-inflammatory drugs.
Ann Intern Med. 1991;115787- 796
Google ScholarCrossref 3.Bollini
PGarcía Rodríguez
LAPérez Gutthann
SWalker
AM The impact of research quality and study design on epidemiologic estimates of the effect of nonsteroidal anti-inflammatory drugs on upper gastrointestinal tract disease.
Arch Intern Med. 1992;1521289- 1295
Google ScholarCrossref 4.Henry
DLim
LLGarcía Rodríguez
LA
et al. Variability in risk of gastrointestinal complications with individual non-steroidal anti-inflammatory drugs: results of a collaborative meta-analysis.
BMJ. 1996;3121563- 1566
Google ScholarCrossref 5.Griffin
MRPiper
JMDaugherty
JRSnowden
MRay
WA Nonsteroidal anti-inflammatory drug use and increased risk for peptic ulcer disease in elderly persons.
Ann Intern Med. 1991;114257- 263
Google ScholarCrossref 6.Laporte
JRCarné
XVidal
XMoreno
VJuan
J Upper gastrointestinal bleeding in relation to previous use of analgesics and non-steroidal anti-inflammatory drugs.
Lancet. 1991;33785- 89
Google ScholarCrossref 7.Holvoet
JTerriere
LVan Hee
WVerbist
LFierens
EHautekeete
M Relation of upper gastrointestinal bleeding to non-steroidal anti-inflammatory drugs and aspirin: a case-control study.
Gut. 1991;32730- 734
Google ScholarCrossref 8.Nobili
AMosconi
PFranzosi
MGTognoni
G Non-steroidal anti-inflammatory drugs and upper gastrointestinal bleeding, a post-marketing surveillance case-control study.
Pharmacoepidemiol Drug Safety. 1992;165- 72
Google ScholarCrossref 9.Keating
J Antiinflammatory drugs and emergency surgery for peptic ulcers in the Waikato.
N Z Med J. 1992;105127- 129
Google Scholar 10.Henry
DDobson
ATruner
C Variability in the risk of major gastrointestinal complications from nonaspirin nonsteroidal anti-inflammatory drugs.
Gastroenterology. 1993;1051078- 1088
Google Scholar 11.Kaufman
DWKelly
JPSheehan
JE
et al. Nonsteroidal anti-inflammatory drug use in relation to major upper gastrointestinal bleeding.
Clin Pharmacol Ther. 1993;53485- 494
Google ScholarCrossref 12.Savage
RLMoller
PWBallantyne
CLWells
JE Variation in the risk of peptic ulcer complications with nonsteroidal antiinflammatory drug therapy.
Arthritis Rheum. 1993;3684- 90
Google ScholarCrossref 13.García Rodríguez
LAJick
H Risk of upper gastrointestinal bleeding and perforation associated with individual non-steroidal anti-inflammatory drugs.
Lancet. 1994;343769- 772
Google ScholarCrossref 14.Langman
MJWeil
JWainwright
P
et al. Risks of bleeding peptic ulcer associated with individual non-steroidal anti-inflammatory drugs.
Lancet. 1994;3431075- 1078
Google ScholarCrossref 15.Lanza
LLWalker
AMBoetnichack
EADreyer
NA Peptic ulcer and gastrointestinal hemorrhage associated with nonsteroidal anti-inflammatory drug use in patients younger than 65 years: a large health maintenance organization cohort study.
Arch Intern Med. 1995;1551371- 1377
Google ScholarCrossref 16.Traversa
GWalker
AMIppolito
FM
et al. Gastroduodenal toxicity of different nonsteroidal antiinflammatory drugs.
Epidemiology. 1995;649- 54
Google ScholarCrossref 17.Hallas
JLauritsen
JDalsgard Villadsen
HGram
LF Nonsteroidal anti-inflammatory drugs and upper gastrointestinal bleeding, identifying high-risk groups by excess risk estimates.
Scand J Gastroenterol. 1995;30438- 444
Google ScholarCrossref 18.Pérez-Gutthann
SGarcía Rodríguez
LARaiford
DS Individual nonsteroidal antiinflammatory drugs and other risk factors for upper gastrointestinal bleeding and perforation.
Epidemiology. 1997;818- 24
Google ScholarCrossref 19.MacDonald
TMMorant
SVRobinson
GC
et al. Association of upper gastrointestinal toxicity of non-steroidal anti-inflammatory drugs with continued exposure: cohort study.
BMJ. 1997;3151333- 1337
Google ScholarCrossref 20.Wilcox
CMAlexander
LNCotsonis
GAClark
WS Nonsteroidal antiinflammatory drugs are associated with both upper and lower gastrointestinal bleeding.
Dig Dis Sci. 1997;42990- 997
Google ScholarCrossref 21.García Rodríguez
LACattaruzzi
CTroncon
MGAgostinis
L Risk of hospitalization for upper gastrointestinal tract bleeding associated with ketorolac, other nonsteroidal anti-inflammatory drugs, calcium antagonists, and other antihypertensive drugs.
Arch Intern Med. 1998;15833- 39
Google ScholarCrossref 22.García Rodríguez
LAWalker
AMPérez Gutthann
S Nonsteroidal antiinflammatory drugs and gastrointestinal hospitalizations in Saskatchewan: a cohort study.
Epidemiology. 1992;3337- 342
Google ScholarCrossref 23.Smalley
WERay
WADaugherty
JRGriffin
MR Nonsteroidal anti-inflammatory drugs and the incidence of hospitalizations for peptic ulcer disease in elderly persons.
Am J Epidemiol. 1995;141539- 545
Google Scholar 24.Cullen
DJHawkey
GMGreenwood
DC
et al. Peptic ulcer bleeding in the elderly: relative roles of
Helicobacter pylori and non-steroidal anti-inflammatory drugs.
Gut. 1997;41459- 462
Google ScholarCrossref 25.Matikainen
MKangas
E Is there a relationship between the use of analgesics and non-steroidal anti-inflammatory drugs and acute upper gastrointestinal bleeding? a Finnish case-control prospective study.
Scand J Gastroenterol. 1996;31912- 916
Google ScholarCrossref 26.Jick
HVessey
MP Case-control studies in the evaluation of drug-induced illness.
Am J Epidemiol. 1978;1071- 7
Google Scholar 27.Chan
TCritchley
JLau
JSung
JChung
SAnderson
D The relationship between upper gastrointestinal hemorrhage and drug use: a case control study.
Int J Clin Pharmacol Ther. 1996;34304- 308
Google Scholar 28.Takkouche
BCadarso-Suárez
CSpiegelman
D Evaluation of old and new tests of heterogeneity in epidemiologic meta-analysis.
Am J Epidemiol. 1999;150206- 215
Google ScholarCrossref 29.Poole
CGreenland
S Random-effects meta-analyses are not always conservative.
Am J Epidemiol. 1999;150469- 475
Google ScholarCrossref 31.Walker
AM Observation and Inference: An Introduction to the Methods of Epidemiology. Newton Lower Falls, Mass Epidemiology Resources Inc1991;
32.Greenland
S Invited commentary: a critical look at some popular meta-analytic methods.
Am J Epidemiol. 1994;140290- 296
Google Scholar 33.Cooper
HHedges
LV The Handbook of Research Synthesis. New York, NY Russell Sage Foundation1994;
34.Olivero
JGraham
D Gastric adaptation to nonsteroidal anti-inflammatory drugs in man.
Scand J Gastroenterol Suppl. 1992;19353- 58
Google ScholarCrossref 35.Yola
MLucien
A Evidence of the depletion of susceptibles effect in non-experimental pharmacoepidemiologic research.
J Clin Epidemiol. 1994;47731- 737
Google ScholarCrossref 36.Robins
JM Causal inference from complex longitudinal data. Berkane
Med.
Lecture Notes in Statistics New York Springer-Verlag NY Inc1997;69- 117
Google Scholar 37.Shapiro
S Meta-analysis/shmeta-analysis.
Am J Epidemiol. 1994;140771- 778
Google Scholar 38.Greenland
S Can meta-analysis be salvaged?
Am J Epidemiol. 1994;140783- 787
Google Scholar 39.Rothman
KJGreenland
S Modern Epidemiology. Philadelphia, Pa Lippincott-Raven Publishers1998;
40.Wolfe
MMLichtenstein
DRSingh
G Gastrointestinal toxicity of nonsteroidal antiinflammatory drugs.
N Engl J Med. 1999;3401888- 1899
Google ScholarCrossref