The most frequently cited article is represented by 1.
Research denotes any hypothesis-driven research article; other denotes any other article type. The horizontal bar inside the boxes indicates the median; lower and upper ends of the boxes, the first and third quartiles of the data; whiskers, the values within 1.5 times the interquartile range from the upper or lower quartile; open circles, outliers; and notches, 95% CIs.
Customize your JAMA Network experience by selecting one or more topics from the list below.
Identify all potential conflicts of interest that might be relevant to your comment.
Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.
Err on the side of full disclosure.
If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.
Not all submitted comments are published. Please see our commenting policy for details.
Giustini AJ, Axelrod DM, Lucas BP, Schroeder AR. Association Between Citations, Altmetrics, and Article Views in Pediatric Research. JAMA Netw Open. 2020;3(7):e2010784. doi:10.1001/jamanetworkopen.2020.10784
Citations are used to assess the impact of scientific authors, articles, and journals but may not fully reflect how they affect journal readership or the public. Newer metrics include article views and the Altmetric attention score (AAS),1 which uses an algorithm to weight mentions of an article in social platforms such as Twitter and Facebook and in the news media. Prior investigations have shown only a modest association between these newer metrics and article citations, but none have assessed the pediatric literature, which may have a different readership.1-4 The primary objective of this study was to examine the association between traditional metrics (citations), the AAS, and views of pediatric articles from 4 high-impact journals. Strong correlation would imply that citations-based metrics indicate popular engagement in scientific articles.
We examined all pediatric articles published in 2014—to allow sufficient time for citations to accrue—from 2 general pediatric journals (Pediatrics and JAMA Pediatrics) and 2 general medical journals (JAMA and the New England Journal of Medicine). Only articles tagged as pediatrics in JAMA and the New England Journal of Medicine were reviewed. All data were obtained between April and May 2018. We excluded supplementary issues of journals as well as news, videos, pictures, letters, errata, patient handouts, republished articles, calls for papers, and retracted articles.
Citations data were obtained from Web of Science (Clarivate), and the AAS and total cumulative page views were recorded from the journal sites. We communicated with each journal’s editorial office and confirmed that the sum of PDF downloads and page views was provided by each. We classified articles as original research, clinical practice guidelines, and other (predominantly editorials and reviews, not all of which were peer-reviewed).
Spearman rank correlation coefficients were calculated to compare the ranks of the 3 metrics. A Kruskal-Wallis test was used to compare metrics between research articles, practice guidelines, and other types of articles. Statistical analysis was done using R, version 3.3.1 (R Foundation for Statistical Computing) and the RStudio interface, version 1.0.143 (RStudio Inc).
Of 1162 included articles, 686 were classified as original research, 40 as clinical practice guidelines, and 436 as other. The correlation (95% CI) between ranks was 0.69 (0.66-0.72) for citations and page views, 0.53 (0.49-0.57) for citations and the AAS, and 0.53 (0.49-0.57) for page views and the AAS. Similar correlations were found when comparing actual citations rather than ranks (data not shown).
Figure 1 shows the AAS and page view ranks for the 100 most-cited articles in 2014. Original research articles were more frequently cited than articles classified as other, and clinical practice guidelines were the most frequently viewed of any article type (Figure 2).
The results show only modest correlation between article citations and newer measures of journal impact (page views and the AAS) in pediatric articles from 4 journals. Citations may not fully convey an article’s impact. The AAS and page views may provide a different perspective on an article’s immediate impact and should be considered when evaluating the impact of an article.
This investigation had several limitations. Citations, views, and the AAS are all cumulative metrics, but the AAS tends to accumulate most of its value immediately after an article is published and may, uniquely, decrease over time5; therefore, the AAS may be an indicator of an article’s initial impact. Thus, this study compared snapshots in time of 3 dynamic processes. We examined only 4 journals, which were selected on the basis of their impact factor and the immediate online availability of all 3 metrics. Finally, page views do not capture print readership of articles.
In summary, citations may not fully capture the impact of scientific work. Newer metrics such as the AAS and page views should be considered when determining the overall impact of publications.
Accepted for Publication: May 6, 2020.
Published: July 20, 2020. doi:10.1001/jamanetworkopen.2020.10784
Open Access: This is an open access article distributed under the terms of the CC-BY License. © 2020 Giustini AJ et al. JAMA Network Open.
Corresponding Author: Andrew J. Giustini, MD, PhD, Department of Anesthesiology, Stanford University School of Medicine, 300 Pasteur Dr, MC 5640, Room H3580, Stanford, CA 94305 (email@example.com).
Author Contributions: Dr Giustini had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Concept and design: Giustini, Axelrod, Schroeder.
Acquisition, analysis, or interpretation of data: Giustini, Lucas.
Drafting of the manuscript: Giustini.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: Giustini, Lucas.
Administrative, technical, or material support: Axelrod.
Supervision: Axelrod, Lucas, Schroeder.
Conflict of Interest Disclosures: None reported.