Association of Author Gender With Sex Bias in Surgical Research

Key Points

Question  Does author gender affect sex bias in surgical clinical research?

Findings  In this bibliometric analysis of research articles published in 5 surgical journals, male and female authors were equally poor at sex-based reporting, analysis, and discussion. Female authors were more likely than male authors to study female participants but were also more likely to include a greater disparity between the number of male and female participants.

Meaning  Sex bias in surgical clinical research is prevalent regardless of author gender.

Importance  Previous studies demonstrate sex bias in surgical research. Female participants and investigators are underrepresented in surgical scientific research.

Objectives  To describe the distribution of male and female authors in 5 general-interest surgery journals, assess the association of author gender with sex bias, and explore whether investigators benefit from performing sex-inclusion research.

Design, Setting, and Participants  For this bibliometric analysis, data were abstracted from 1921 original, peer-reviewed articles published from January 1, 2011, through December 31, 2012, in Annals of Surgery, American Journal of Surgery, JAMA Surgery, The Journal of Surgical Research, and Surgery. Excluded articles pertained to a sex-specific disease or did not report the number of study participants. An additional 119 articles contained gender-ambiguous author names and were omitted. Data were analyzed from April to June 2017.

Main Outcomes and Measures  Male and female first and senior authors, number of female and male participants in each study, surgical specialty, and number of citations received per article.

Results  Of the 3604 authors of 1802 articles included in this study, 2791 first and senior authors (77.4%) were male and 813 (22.6%) were female. The prevalence of male and female authors was consistent across all 5 journals and among clinical and basic science research. Articles by female authors included a higher median number of female study participants compared with their male counterparts (27.5 vs 16.0; P = .01), but sex matched the inclusion of participants less frequently (36% vs 45%; P = .001). No sex-based differences occurred between male and female authors in reporting, statistical analysis, and discussion of the data or in the number of citations received. Compared with studies that did not report, analyze, or discuss data by sex, studies that performed sex-specific data reporting yielded a mean of 2.8 more citations (95% CI, 1.2-4.4; P = .001); those that performed statistical analysis, a mean of 3.5 more citations (95% CI, 1.8-5.1; P = .001); and those that discussed the data, a mean of 2.6 more citations (95% CI, 0.7-4.5; P = .001). Articles with a higher percentage of sex matching of participants also received more citations, with an increase of 1 citation per 4.8% (95% CI, 2.0%-7.7%; P = .001) increase in percentage of sex matching.

Conclusions and Relevance  Sex bias in surgical research is prevalent among male and female authors; however, female authors included proportionally more female participants in their studies compared with male authors. Notably, studies that addressed sex bias were rewarded by the scientific community with increased citations of their published work.

Equal representation of both sexes in biomedical research is of paramount importance, as illustrated by the number of medications, disease processes, and treatment modalities that affect one sex differently than the other.^1-6 The divergence between the sexes is pervasive in surgical fields as well; men and women have diverse postoperative outcomes, complications, complication rates, and readmission rates.^7,8 However, the studies on which our medical knowledge is founded consist mostly of male participants.^9,10 The disparity between conclusions drawn from male-dominated studies and the subsequent application to the female sex has led to a number of adverse events and patient outcomes.^1,11-15 Actively identifying and neutralizing sex-biased research is essential to effectively understand, diagnose, and cure the diseases that affect men and women without further propagating iatrogenic harm.

Despite the National Institutes of Health (NIH) Revitalization Act of 1993 that mandated inclusion of women in NIH-funded clinical trials,¹⁶ studies continue to demonstrate the lack of equal recruitment of female participants in clinical trials.¹⁴ Previous research has suggested that the exclusion of women in scientific research is a product of historical factors and preconceptions regarding women as suboptimal research participants.^17,18 Investigators have raised the notion of male bias and have questioned whether the overwhelming predominance of male investigators plays a role in the exclusion of female participants.^19,20 A type of observer bias, male bias suggests that male researchers may be more inclined to study and recruit male participants, and the paucity of female researchers may leave women underrepresented as participants.^19,20 This situation may be even more pronounced in surgical fields, with a greater preponderance of male researchers compared with nonsurgical medical specialties.^21,22 To our knowledge, no studies have identified factors that influence the degree of sex bias in scientific research. Furthermore, outside of the altruistic incentive of producing good science, no studies demonstrate any quantifiable benefit, such as increased recognition through citations, to researchers who conduct sex-inclusive research.

Sex bias in research remains prevalent and pervasive despite adverse events, the enactment of numerous laws, and overwhelming data cautioning against its practice.^6,14,23,24 Our objectives were to describe the distribution of male and female authors in top surgical journals; to elucidate whether the gender of the authors influences the inclusion of female participants in research; to determine the association of author gender with article sex bias; and to explore whether investigators achieved quantifiable rewards, such as the number of citations for performing sex-inclusive research. We hypothesized that author gender does not affect the degree of sex bias in scientific research and that researchers who perform sex-inclusive research are cited more often in the literature.

Data were collected as previously described by Yoon el at²⁴ and Mansukhani et al.²³ Briefly, we reviewed all original articles published from January 1, 2011, through December 31, 2012, in Annals of Surgery, American Journal of Surgery, JAMA Surgery, The Journal of Surgical Research, and Surgery. These journals were selected owing to their relevance to the field of surgery. This period was selected so that comparisons of these analyses can be made to the prior published studies and to allow for an analysis of citations received after publication.^23,24 Reviews, letters to the editor, case reports, and other nonoriginal research articles were excluded from data abstraction. Two data abstractors (N.X. and D.F.M.) independently collected data concerning author gender. Intra-abstractor reliability and verification of author genders was performed by comparing both data sets. This study was deemed exempt from institutional review board approval by Northwestern University.

This study includes data previously abstracted as described by Mansukhani et al,²³ including whether the article reported a sex-specific disease by anatomical criteria, the surgical specialty responsible for each study (for clinical studies), the numbers by sex of each participant studied, and the presence of sex bias as reported by 3 sex-bias metrics (reporting of data by sex, analysis of data by sex, and inclusion of sex-based results in the Discussion section). In addition, the gender of the first author, gender of the senior author, and number of citations received were abstracted for this study.

Genders of the first and senior authors were manually collected through a web search by 1 of the 2 data collectors (N.X.). The second collector (D.F.M.) used a validated name disambiguation procedure.²⁵ Author names were extracted from Web of Science and cross-matched with the gender assignment software SexMachine.²⁶ The system uses first names and associated countries to assign a likely gender. If the gender could not be classified, data were cross-referenced to demographic data from the US Census Bureau,²⁷ which has documented first names, their frequency, and their respective gender since 1880. Intra-abstractor agreement was greater than 95%, and data points that were discordant were reviewed manually. For cases in which definitive agreement could not be reached, the data point was omitted from analysis.

The number of citations received for each article was gathered using code written in Python software (version 3.6.1; Python Software Foundation) and the Web of Science database (Clarivate Analytics). Python code was written to count the frequency that each article occurred in a references database containing all references after the specified article’s date of publication. For example, an article published in 2010 would be cross-referenced with a database consisting of all reference sections from articles published from 2010 to the present (April 2017), and the total count of instances was collected as the citation number.

The percentage matching as reported in this study was calculated as previously described.²³ Briefly, the number of participants of the sex (male or female) with the lesser value were divided by the number of participants of the sex with the greater value, then multiplied by 100. For example, a study with the same numbers of male and female participants would have yielded a 100% match. A study with twice the number of one sex compared with the other would yield a 50% match, as shown by the following calculation:

(No. of Participants of the Sex With Lesser Value/No. of Participants of the Sex With Greater Value) × 100 = (1x/2x) × 100% = 50%

The expected frequency of each first-senior author combination was calculated using proportions of male and female authors among first and senior authors from all articles included in the study. Calculations were performed with the assumption of stochastic independence between the gender of the first and senior authors. A deviation from the expected result was reported as the difference between calculated and observed author combinations. A goodness-of-fit (χ²) test was used to determine whether the difference between the frequency of actual and expected combinations of male and female first and senior authors was significant (P < .05).

Data were analyzed from April to June 2017. We used χ² tests to examine differences among male and female author distributions by journal and specialty and to assess differences in observed and expected male and female author distributions in this study. Simple linear regression models were used to assess the association of male and female authors with sex-based reporting, analysis, and discussion of data on the number of citations received. Regression model diagnostics were performed to evaluate for linearity, independence, and normality. Simple logistic regression was used to assess the association of male and female authors with author gender combinations on sex-bias metrics. Linear regressions were expressed as dependent-variable coefficients with 95% CIs. Logistic regressions were expressed as odds ratios with 95% CIs. Significance was assumed at 2-sided P < .05. Analyses and graph design were conducted using STATA/SE software (version 14.2; StataCorp) and SigmaPlot (version 10.0; Systat Software, Inc).

A total of 2347 articles were reviewed among the 5 selected surgery journals. After excluding articles that reported a sex-specific disease or did not report the number of study participants, 1921 articles remained, with 1303 articles including human participants and 618 articles including animal or cell research. The genders of the first and senior authors were successfully obtained for 1802 articles; 119 articles were omitted because the gender of the first or senior author could not be verified. Of the 1802 articles, 1271 (70.5%) were articles with human participants, and the remaining 531 (29.5%) were articles with cells or animal subjects. Of the 3604 first and senior authors, 2791 (77.4%) were male and 813 (22.6%) were female. The predominance of male authors was more pronounced among senior authors (1525 of 1802 [84.6%]) than first authors (1266 of 1802 [70.3%]; P = .001) (Table 1). The proportion of male and female authors was consistent and similar among all 5 journals studied as well as among clinical and basic or translational research articles. When male and female authors were stratified by specialty, more female first and senior authors produced articles classified as breast surgery (24 of 54 [44.4%] and 15 of 54 [33.3%], respectively; P = .01), endocrine surgery (51 of 120 [42.5%] and 34 of 120 [28.3%], respectively; P = .01), or surgical education and training (42 of 98 [42.9%] and 23 of 98 [23.5%], respectively; P = .02) compared with the other specialties. Despite this finding, the most authors in these specialties were still male.

Most articles (1111 [61.7%]) were written by a combination of a male first author and male senior author (Figure 1A), with far fewer articles written by a female first or senior author (691 [38.3%]). When compared with random pairing, male-male and female-female first and senior author combinations were observed 2.2% more often than expected, and female-male and male-female first and senior author combinations were observed 2.2% less often than expected (P = .001) (Figure 1B). These author gender combinations remained consistent throughout all 5 journals studied (Figure 1C) and among most specialties (Figure 1D). A significant difference of author gender combinations (female first and senior authors) compared with the other specialties was demonstrated in the specialties of breast surgery (34.4% and 33.3%, respectively; P = .01), endocrine surgery (42.5% and 28.3%, respectively; P = .01), and surgical education and training (42.9% and 23.5%, respectively; P = .02), with a higher prevalence of female authors.

Female authors were more likely than male authors to study exclusively female participants (5.6% vs 2.3%; P = .001) (Figure 2A). This difference was most pronounced in the female-female author combination, in which 9.7% of clinical articles written by these authors studied only female participants (vs 2.7%; P = .001). Articles written by the female-female author combination were the only articles with more female than male participants (27.5% vs 20.5%; P = .001) (Figure 2B). Alternatively, compared with their male counterparts, female authors included both sexes less frequently (74.3% vs 79.8% for first authors and 71.8% vs 79.4% for senior authors; P = .01) and female senior authors matched participants by sex at lower percentages compared with male senior authors (38.0% vs 44.5%; P = .001) (Figure 2C). When differences in the number of participants studied were compared for each sex among various author combinations, no difference occurred between male-male, female-male, or male-female combinations of first and senior authors; however, female-female author combinations sex matched 36% of participants vs 45% among male-male authors (P = .01).

When studies labeled as breast or endocrine surgical subspecialties were removed from the data set, male senior authors sex matched participants at 48% compared with 44% of female senior authors (P = 6). Furthermore, female-female author combinations no longer included more female than male participants.

Author gender did not have a significant predictive association with whether an article contained sex-based reporting of data, sex-based statistical analysis, or inclusion of sex-specific results in the Discussion section (Table 2). This statistical nondifference between author genders prevailed among all author gender combinations, as well as after stratifying data into human participants vs animal or cell research.

The mean number of citations received by articles included for analysis was 11.9 (95% CI, 11.2-12.6). No significant difference in the number of citations received occurred by author gender or author gender combinations. Articles that conducted sex-specific data reporting received a mean of 2.8 more citations than those that did not (95% CI, 1.2-4.4; P = .001). Articles that performed sex-specific statistical analysis received a mean of 3.5 more citations than those that did not (95% CI, 1.8-5.1; P = .001). Articles that included a sex-specific discussion received a mean of 2.6 more citations than those that did not (95% CI, 0.7-4.5; P = .001). Articles with a higher percentage of sex matching of participants also received more citations, with an increase of 1 citation per 4.8% increase (95% CI, 2.0%-7.7%; P = .001) in percentage of sex matching (Table 3).

We demonstrate in this study that first and senior female authors, and especially first-senior female-female pairings, are more likely than their male counterparts to include female participants in their studies. Our data also reveal that, regardless of author gender, overall rates of sex-inclusive research in these surgical journals remain low and concerning, with neither male nor female authors producing a superior result. However, one encouraging finding is that adequately addressing sex bias by performing sex-specific data reporting and sex-specific statistical analysis, including a sex-specific discussion, and including similar numbers of male and female participants are all rewarded with a significant increase in the number of citations after publication.

Historically, physicians and researchers have been predominantly male. Census data in 2017 showed that nearly 66% of all physicians were male across all specialties.²² A study by Jagsi et al²⁸ in 2008 showed that women were dramatically underrepresented in surgical fields compared with the other medical specialties, which is corroborated in our study of surgical investigators. They also showed an effect of women attracting other women to the field, because a female department chair, the number of female residents, and the exposure to female full-time faculty resulted in the recruitment of more female medical students. This study uncovered a similar “mutual sex attraction phenomenon” in which female authors tended to work with other female authors and female researchers tended to recruit more female patients. Similarly, the observed prevalence of same-gender author combinations (male-male and female-female) was significantly higher than would be expected from statistics and random pairing alone. In accordance with this sex bias, the observed prevalence of the mixed combinations of authors (male-female and female-male) was significantly lower than expected. We conclude that there appears to be an inherent sex bias among investigators and that authors are more inclined to collaborate with an author of the same gender.

Our data revealed that female authors included more female participants. With most participants being male, we estimated that a higher inclusion of female participants would tend toward a higher percentage of matched sex. Surprisingly, the female authors with the most pronounced increase in female participants yielded a lower percentage of matched sex. This unexpected finding suggests 1 of the following 2 scenarios: (1) female authors included more female participants at the neglect of male participants, yielding a lower percentage of matched sex or (2) that female authors studied disease processes that were more prevalent in women, naturally leading to a higher quantity of female participants and a lower percentage of sex matching. When examining all the surgical specialties, we found that breast and endocrine surgical specialties and surgical education or training produced a significant difference in author genders, with appreciably more female authors than other specialties. Although these surgical specialties deal with disease processes in both sexes, most patients are female, especially in breast and thyroid surgery. We therefore reanalyzed the data set after removing breast and endocrine surgical studies and discovered that the difference in sex matching of participants between male and female senior authors narrowed and became statistically insignificant. Furthermore, female-female author combinations no longer included more female participants. As expected, the number of included female participants decreased while the number of male participants increased, confirming that many of the omitted studies were predominantly female. These data are heavily supportive of the second scenario.

Sex-based reporting of data, sex-based statistical analysis of data, and sex-based discussion were used as metrics for sex bias in this study. We demonstrated that author gender, regardless of combination, did not have a statistically significant association with these metrics. All authors were equally poor at performing sex-inclusive research. Mansukhani et al²³ reported that, in these 5 surgical journals, only 38% of clinical papers performed sex-based reporting, only 33% statistically analyzed data by sex, and only 23% included a discussion of sex-based results. More recent studies reviewing institutional review board protocols at major research institutions revealed that, despite the NIH policy establishing sex as a biological variable, only 14.5% of projects provided justification for choice of participant sex despite studying disease processes that exhibited a moderate degree of sex differences.⁶ In this study, we demonstrate that these concerning percentages were pervasive across male and female authors in both clinical and basic science literature (eTable in the Supplement). Although these rates are unacceptably low and should incite researchers to action for the sake of good science and medical advancement, we also show a quantifiable benefit to performing unbiased research reflected in an increase in citations. Investigators are rewarded with a 25% increase in citations for undertaking the task of exploring sex differences. It is evident from this recognition that the scientific community has no issues with identifying and rewarding good, sex-inclusive science. We hope that by validating a tangible, quantifiable reward for performing unbiased research, editors and authors alike will be incentivized to scrutinize their articles for sex bias.

Several limitations exist in this study. We included these 1921 studies published in 5 common general surgery journals, but the data may not be representative of all surgical research. These journals are based in the United States and may not be representative of an international audience. However, more than 700 authors in this study are international investigators. Second, these studies were published from 2011 to 2012. Because these articles are not current, the data may not be representative of the recent NIH policy to include male and female participants in all research and grant applications.^29,30 However, by analyzing articles published 5 years ago, we were able to gather data on citations. Finally, these data did not account or correct for specific disease prevalence among the sexes when analyzing the number of recruited participants.

This study shows that author gender may play a role in the sex of participants included in surgical research. Despite this, author gender was not predictive of the degree of sex bias in this study. Investigators who addressed sex bias were rewarded by the scientific community with increased citations of their published work. Sex bias perseveres in the current age of scientific research and evidence-based medicine. This issue must be adequately addressed and explored for production of meaningful scientific knowledge and for optimizing patient outcomes.

Accepted for Publication: January 9, 2018.

Corresponding Author: Melina R. Kibbe, MD, Department of Surgery, University of North Carolina at Chapel Hill, 4041 Burnett Womack, 101 Manning Dr, Campus Box 7050, Chapel Hill, NC 27599 (melina_kibbe@med.unc.edu).

Published Online: March 28, 2018. doi:10.1001/jamasurg.2018.0040

Author Contributions: Mr Xiao and Dr Mansukhani had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Xiao, Mansukhani, Kibbe.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: All authors.

Critical revision of the manuscript for important intellectual content: Xiao, Mansukhani, Kibbe.

Statistical analysis: Xiao, Mansukhani, Mendes de Oliveira.

Obtained funding: Mansukhani.

Administrative, technical, or material support: Xiao, Mansukhani, Kibbe.

Study supervision: Kibbe.

Conflict of Interest Disclosures: None reported.

Funding/Support: This study was supported in part by grant T32HL094293 from the National Heart, Lung, and Blood Institute of the National Institutes of Health (NIH) (Dr Mansukhani).

Role of the Funder/Sponsor: The funder had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Disclaimer: The content of this article is solely the responsibility of the authors and does not necessarily represent the official views of the NIH. Dr Kibbe is the editor of JAMA Surgery, but she was not involved in any of the decisions regarding review of the manuscript or its acceptance.