Context An estimated correlation between 2 variables is valid only within the
range of observed data. Extrapolation is risky and should be performed with
caution.
Methods To assess the prevalence of problems with data extrapolation in the
medical literature, all articles published from January through June 2000
in BMJ, JAMA, The Lancet,
and The New England Journal of Medicine (NEJM) were reviewed manually. Articles containing at least 1 scatterplot
with raw data and a corresponding fitted regression line were included in
the analysis. Articles were considered to involve extrapolation if they contained
at least 1 fitted line beyond the observed data in any scatter plot.
Results A total of 178 articles presenting at least 1 scatterplot were identified.
Among them, 37 articles (21%) (5 from BMJ, 7 from JAMA,
23 from The Lancet, and 2 from NEJM) were included. Twenty-two articles (59% [95% confidence interval,
42%-75%]) from all 4 journals involved extrapolation. None changed the line
type to indicate extrapolation. Four articles (11%) contained a plot in which
the fitted line reached unreasonable or meaningless values. Three articles
(8%) stated explicit conclusions about values outside the range of the observed
data.
Conclusions A high proportion of the articles analyzed from all 4 weekly general
medical journals involved extrapolation without indication. Researchers, reviewers,
and editors should be aware of this problem and work to eliminate it.
Adding a fitted regression line to a scatterplot is helpful for describing
an estimated relationship between 2 continuous variables. However, this estimation
is valid only within the range of data. Therefore, without knowing information
beyond the observed characteristics, it is very risky to extrapolate the fitted
line.1-3 In spite
of this concern, several articles have been published in which the fitted
line not only exceeded the range of the data but also reached an undesirable
value.4-6
This study was undertaken to assess how prevalent the extrapolation
problem is and how this issue was managed in 4 general medical journals.
All of the articles published from January through June 2000 in 4 weekly
general medical journals (BMJ, JAMA, The Lancet, and The New England Journal of Medicine [NEJM]) were manually reviewed. The main
outcome measure was the proportion of articles that involved data extrapolation
problems. Extrapolation was defined as an instance in which the fitted line
exceeded the observed data range of explanatory variables in the regression
model (as shown in Figure 1). The
first step was to identify articles in which at least 1 scatter plot was presented.
Subsequently, to assess extrapolation, only articles showing a scatterplot
of raw data and a corresponding fitted regression line were included in the
analysis.
A total of 178 articles with at least 1 scatter plot were identified.
Among them, 37 articles (21%) with scatterplots presenting raw data and a
corresponding fitted regression line were included in this study: 5 from BMJ,7-11
7 from JAMA,12-18
23 from The Lancet,19-41
and 2 from NEJM.42,43
All together, 22 articles (59% [95% confidence interval, 42%-75%]) from
these journals had an extrapolation problem. The proportions (from lowest
to highest, 40%, 50%, 57%, and 86%) were not statistically significantly different
(P = .37 by 2-sided Fisher exact test) among journals
because of the small sample size. This problem was found regardless of whether
the correlation was assessed using simple linear regression. None of the illustrations
of fitted lines included graphic distinction of the line to indicate extrapolation.
Four articles (11%) presented a fitted line that reached a meaningless11,40 or unreasonable14,28
value. In addition, 3 articles (8%) stated explicit conclusions about values
outside the range of the observed data.33,40,43
This study reveals that almost 60% of the included articles involved
extrapolation when displaying a correlation between 2 variables with a fitted
line. This common problem was found in all 4 weekly general medical journals.
Mathematically, a fitted regression line can be drawn by plugging in
almost any real numbers to the estimated equation. However, in clinical applications,
this line should not be presented to exceed the range of data. Otherwise,
extrapolation can result in reaching an undesirable value. For example, in
one study, the fitted line reached a negative value for time from stroke symptom
onset to emergency department arrival.14 This
suggests that patients arrived at the emergency department before the onset
of stroke symptoms. Another article presented 2 regression lines that unreasonably
arrived at a negative level of proteinuria.28
Sometimes the error of reaching an undesirable value is not obvious
and is difficult to discern unless readers can carefully identify the reasonable
range of data in the study (which is not always available). For example, in
one article, the fitted line crossed a meaningless area of the Townsend score.11 The range of scores was not described in the legend
of the graph but, rather, in the text. Another study showed a fitted line
reaching a value of undetectable cytomegalovirus viral load.40
A nominal value of negative result was described in the text but not shown
in the graph. As a consequence, using an undetectable amount to make a prediction
is apparently meaningless.
Limitations of or errors generated by computer programs could be a possible
cause of extrapolating or reaching an undesirable value. For example, in the
4 articles that involved extrapolating an undesirable value, all of the lines
reached the edges of the graph.11,14,28,40
However, 9 of 22 articles had extrapolation problems in which the fitted lines
did not reach the margins of the graph. Authors are responsible for ensuring
that the estimation and presentation of their data are clinically meaningful
and should carefully check all data and graphs generated by a computer program.
Problems with extrapolation also involved stating explicit conclusions
about values outside the range of the observed data. In some cases, the reader
is required to perform some calculations to become aware of the extrapolation
problem. For example, in one study, 2 doses used for demonstrating the effect
of inhaled corticosteroids on bone mineral density were both outside the range
of observed data; one was apparently higher; the other one, after computation,
was lower.33 In another study, the expected
time to reach undetectable levels of thymus-dependent T-cell antigen-receptor
episomes not only exceeded the maximum number of years after transplantation
but also did not correspond to the fitted line in the graph as well as the
regression equation.43
With raw data, it is very easy for a reader to be aware of an extrapolation
problem from a graph. However, if only a fitted line is presented without
the original data points, or even an integrated plot, the extrapolation problem
becomes harder to identify. For example, in one article, some of the areas
in the constructed contour plot exceeded the range of data.40
However, this problem could not be identified readily by the reader without
performing calculations based on information from the text.44
Extrapolation is very dangerous for medical decision making and can
result in damaging outcomes.1-3
Describing or presenting the estimated correlation within the range of data
can prevent potential problems. However, if making an extrapolation is necessary
for a researcher, such analysis must be handled with caution. That is, it
needs to be described explicitly in the text or indicated in the plot by use
of differentiating line types, especially when the range of data is not provided.
In addition, during the peer review process, editors and reviewers need to
be aware of and identify such extrapolation issues to prevent and eliminate
potential problems.
1.Armitage P, Berry G. Statistical Methods in Medical Research. 3rd ed. Cambridge, Mass: Blackwell Scientific Publications; 1994.
2.Daniel WW. Biostatistics: A Foundation for Analysis in the Health
Sciences. 7th ed. New York, NY: John Wiley & Sons; 1999.
3.Altman DG, Bland JM. Generalisation and extrapolation.
BMJ.1998;317:409-410.Google Scholar 4.Doty RL, Li C, Mannon LJ, Yousem DM. Olfactory dysfunction in multiple sclerosis.
N Engl J Med.1997;336:1918-1919.Google Scholar 5.Smith JK, Dykes R, Douglas JE.
et al. Long-term exercise and atherogenic activity of blood mononuclear cells
in persons at risk of developing ischemic heart disease.
JAMA.1999;281:1722-1727.Google Scholar 6.Fukumoto M, Ushida T, Zinchuk VS.
et al. Contralateral thalamic perfusion in patients with reflex sympathetic
dystrophy syndrome.
Lancet.1999;354:1790-1791.Google Scholar 7.Copas JB, Shi JQ. Reanalysis of epidemiological evidence on lung cancer and passive smoking.
BMJ.2000;320:417-418.Google Scholar 8.Ross NA, Wolfson MC, Dunn JR.
et al. Relation between income inequality and mortality in Canada and in the
United States: cross sectional assessment using census data and vital statistics.
BMJ.2000;320:898-902.Google Scholar 9.Donnelly R, Emslie-Smith AM, Gardner ID, Morris AD. ABC of arterial and venous disease: vascular complications of diabetes.
BMJ.2000;320:1062-1066.Google Scholar 10.Lynch JW, Smith GD, Kaplan GA, House JS. Income inequality and mortality: importance to health of individual
income, psychosocial environment, or material conditions.
BMJ.2000;320:1200-1204.Google Scholar 11.Packham C, Pearson J, Robinson J, Gray D. Use of statins in general practices, 1996-8: cross sectional study.
BMJ.2000;320:1583-1584.Google Scholar 12.Ebrahim SH, Floyd RL, Merritt II RK.
et al. Trends in pregnancy-related smoking rates in the United States, 1987-1996.
JAMA.2000;283:361-366.Google Scholar 13.Schairer C, Lubin J, Troisi R.
et al. Menopausal estrogen and estrogen-progestin replacement therapy and
breast cancer risk.
JAMA.2000;283:485-491.Google Scholar 14.Albers GW, Bates VE, Clark WM.
et al. Intravenous tissue-type plasminogen activator for treatment of acute
stroke: the Standard Treatment with Alteplase to Reverse Stroke (STARS) study.
JAMA.2000;283:1145-1150.Google Scholar 15.Rubinstein S, Caballero B. Is Miss America an undernourished role model?
JAMA.2000;283:1569.Google Scholar 16.Hutter JC, Kuehnert MJ, Wallis RR.
et al. Acute onset of decreased vision and hearing traced to hemodialysis
treatment with aged dialyzers.
JAMA.2000;283:2128-2134.Google Scholar 17.Yokota F, Thompson KM. Violence in G-rated animated films.
JAMA.2000;283:2716-2720.Google Scholar 18.Gallagher RP, Rivers JK, Lee TK.
et al. Broad-spectrum sunscreen use and the development of new nevi in white
children.
JAMA.2000;283:2955-2960.Google Scholar 19.Powers HJ, Fraser R, Gibson AT. Antioxidants and pre-eclampsia.
Lancet.2000;355:64-65.Google Scholar 20.von Dadelszen P, Ornstein MP, Bull SB.
et al. Fall in mean arterial pressure and fetal growth restriction in pregnancy
hypertension.
Lancet.2000;355:87-92.Google Scholar 21.Port S, Demer L, Jennrich R.
et al. Systolic blood pressure and mortality.
Lancet.2000;355:175-180.Google Scholar 22.Hauben E, Nevo U, Yoles E.
et al. Autoimmune T cells as potential neuroprotective therapy for spinal
cord injury.
Lancet.2000;355:286-287.Google Scholar 23.Checkley W, Epstein LD, Gilman RH.
et al. Effect of El Nino and ambient temperature on hospital admissions for
diarrhoeal diseases in Peruvian children.
Lancet.2000;355:442-450.Google Scholar 24.Karlsson MK, Linden C, Karlsson C.
et al. Exercise during growth and bone mineral density and fractures in old
age.
Lancet.2000;355:469-470.Google Scholar 25.Kuulasmaa K, Tunstall-Pedoe H, Dobson A.
et al. Estimation of contribution of changes in classic risk factors to trends
in coronary-event rates across the WHO MONICA Project populations.
Lancet.2000;355:675-687.Google Scholar 26.Tunstall-Pedoe H, Vanuzzo D, Hobbs M.
et al. Estimation of contribution of changes in coronary care to improving
survival, event rates, and coronary heart disease mortality across the WHO
MONICA Project populations.
Lancet.2000;355:688-700.Google Scholar 27.EURODIAB ACE Study Group. Variation and trends in incidence of childhood diabetes in Europe.
Lancet.2000;355:873-876.Google Scholar 28.Diskin CJ, Stokes TJ, Dansby LM.
et al. Surface tension, proteinuria, and the urine bubbles of Hippocrates.
Lancet.2000;355:901-902.Google Scholar 29.Stark J, Gallivan S, Lovegrove J.
et al. Mortality rates after surgery for congenital heart defects in children
and surgeons' performance.
Lancet.2000;355:1004-1007.Google Scholar 30.Jokeit H, Luerding R, Ebner A. Cognitive impairment in temporal-lobe epilepsy.
Lancet.2000;355:1018-1019.Google Scholar 31.Haber HP, Busch A, Ziebach R, Stern M. Bowel wall thickness measured by ultrasound as a marker of Crohn's
disease activity in children.
Lancet.2000;355:1239-1240.Google Scholar 32.Maccarrone M, Valensise H, Bari M.
et al. Relation between decreased anandamide hydrolase concentrations in human
lymphocytes and miscarriage.
Lancet.2000;355:1326-1329.Google Scholar 33.Wong CA, Walsh LJ, Smith CJ.
et al. Inhaled corticosteroid use and bone-mineral density in patients with
asthma.
Lancet.2000;355:1399-1403.Google Scholar 34.van der Gaag MS, Ubbink JB, Sillanaukee P.
et al. Effect of consumption of red wine, spirits, and beer on serum homocysteine.
Lancet.2000;355:1522.Google Scholar 35.Gereda JE, Leung DY, Thatayatikom A.
et al. Relation between house-dust endotoxin exposure, type 1 T-cell development,
and allergen sensitisation in infants at high risk of asthma.
Lancet.2000;355:1680-1683.Google Scholar 36.Spivak JL. The blood in systemic disorders.
Lancet.2000;355:1707-1712.Google Scholar 37.Laureys S, Faymonville ME, Luxen A.
et al. Restoration of thalamocortical connectivity after recovery from persistent
vegetative state.
Lancet.2000;355:1790-1791.Google Scholar 38.Idle JR. The heart of psychotropic drug therapy.
Lancet.2000;355:1824-1825.Google Scholar 39.Douek DC, Vescio RA, Betts MR.
et al. Assessment of thymic output in adults after haematopoietic stem-cell
transplantation and prediction of T-cell reconstitution.
Lancet.2000;355:1875-1881.Google Scholar 40.Emery VC, Sabin CA, Cope AV.
et al. Application of viral-load kinetics to identify patients who develop
cytomegalovirus disease after transplantation.
Lancet.2000;355:2032-2036.Google Scholar 41.Ylitalo N, Sorensen P, Josefsson AM.
et al. Consistent high viral load of human papillomavirus 16 and risk of cervical
carcinoma in situ: a nested case-control study.
Lancet.2000;355:2194-2198.Google Scholar 42.Hariharan S, Johnson CP, Bresnahan BA.
et al. Improved graft survival after renal transplantation in the United States,
1988 to 1996.
N Engl J Med.2000;342:605-612.Google Scholar 43.Patel DD, Gooding ME, Parrott RE.
et al. Thymic function after hematopoietic stem-cell transplantation for the
treatment of severe combined immunodeficiency.
N Engl J Med.2000;342:1325-1332.Google Scholar 44.Kuo YH, Kuo YL. Viral-load kinetics and CMV disease.
Lancet.2000;356:1352-1353.Google Scholar