Context In fall 2009 in Quebec, Canada, an immunization campaign was launched against the 2009 influenza A(H1N1) pandemic strain, mostly using an AS03 adjuvant vaccine. By the end of the year, 57% of the 7.8 million residents had been vaccinated.
Objective To assess the risk of Guillain-Barré syndrome (GBS) following pandemic influenza vaccine administration.
Design Population-based cohort study with follow-up over the 6-month period October 2009 through March 2010. The investigation was ordered by the chief medical officer of health in accordance with the Quebec Public Health Act.
Setting All acute care hospitals and neurology clinics in Quebec.
Population Suspected and confirmed GBS cases reported by physicians, mostly neurologists, during active surveillance or identified in the provincial hospital summary discharge database. Medical records were reviewed and cases classified according to Brighton Collaboration definitions (categorized as level 1, 2, or 3, corresponding to criteria of decreasing certainty in diagnosis). Immunization status was verified and denominators were estimated from the provincial immunization registry (4.4 million vaccinated) and census data (total target population aged ≥6 months, 7.8 million), with a total of 3 623 046 person-years of observation.
Main Outcome Measures Relative and attributable risks were calculated using a Poisson model and the self-controlled case-series method.
Results Over a 6-month period, 83 confirmed GBS cases were identified, including 71 Brighton level 1 through 3 cases. Twenty-five confirmed cases had been vaccinated against 2009 influenza A(H1N1) 8 or fewer weeks before disease onset, with most (19/25) vaccinated 4 or fewer weeks before onset. In the Poisson model, the age- and sex-adjusted relative risk was 1.80 (95% CI, 1.12-2.87) for all confirmed cases during the 8-week postvaccination period and was 2.75 (95% CI, 1.63-4.62) during the 4-week postvaccination period. Using the self-controlled case-series method, relative risk estimates during the 4-week postvaccination period were 3.02 (95% CI, 1.64-5.56) for all confirmed cases (n = 42) and 2.33 (95% CI, 1.19-4.57) for Brighton level 1 through 3 cases (n = 36). The number of GBS cases attributable to vaccination was approximately 2 per 1 million doses. There was no indication of an excess risk in persons younger than 50 years.
Conclusions In Quebec, the 2009 influenza A(H1N1) vaccine was associated with a small but significant risk of GBS. It is likely that the benefits of immunization outweigh the risks.
Quiz Ref IDGuillain-Barré syndrome (GBS) is a peripheral neuropathy with acute onset and is characterized, in its typical presentation, by rapidly developing motor weakness and areflexia.1,2 The disease is thought to be autoimmune and triggered by a stimulus of external origin.1,2 In 1976-1977, an unusually high rate of GBS was identified in the United States following the administration of inactivated “swine” influenza A(H1N1) vaccines.3 In 2003, the Institute of Medicine (IOM) concluded that the evidence favored acceptance of a causal relationship between the 1976 swine influenza vaccines and GBS in adults.4 Studies of seasonal influenza vaccines administered in subsequent years have found small or no increased risk.5 In mice, different influenza vaccines can induce antiganglioside antibodies that are associated with the development of GBS in humans.6 Extrapolation of results of animal studies to humans, however, is difficult. In a more recent assessment of epidemiologic studies on seasonal influenza vaccines, experimental studies in animals, and case reports in humans, the IOM Committee to Review Adverse Effects of Vaccines concluded that the evidence was inadequate to accept or reject a causal relationship.7
In the province of Quebec, Canada, a mass immunization campaign was launched in the fall of 2009 to control a pandemic caused by a new influenza A(H1N1) virus.8,9 Herein we report results of a population-based epidemiologic investigation ordered by the chief medical officer of health, based on GBS cases notified to public health authorities and others found in the MEDECHO provincial hospitalization database.
In Quebec, the mass immunization campaign started on October 26, 2009. The target population included all residents aged 6 months or older (total = 7.8 million). Pandemic vaccines were administered by the public health service only. All immunizations were recorded in a specific registry linked to the universal provincial health insurance database. By the end of the year, 57% of the target population had been vaccinated (4.4 million). The majority (96%) received the inactivated monovalent ASO3 adjuvant influenza A(H1N1) vaccine (Arepanrix, GlaxoSmithKline).
As a precautionary measure, the chief medical officer of health ordered an epidemiologic investigation of GBS in accordance with the Quebec Public Health Act.10 This meant that approval by an ethics committee was not required and that all records pertaining to GBS patients could be reviewed.
In early October 2009, all physicians in the province were informed that GBS had been included in the list of reportable diseases and should be reported irrespective of a patient's immunization status. In addition, all neurologists were contacted by mail twice a month from early October 2009 to mid-April 2010 and asked to report any confirmed or suspected GBS cases. In addition, a nominative list of admissions to any acute care hospital with a main diagnosis of GBS (International Statistical Classification of Diseases, Tenth Revision code G610) during the period October 2009 through March 2010 was received. The lists of patients reported prospectively by physicians and those retrospectively identified in the hospital admission database were linked. Medical directors of health care facilities in which these patients had been treated were contacted and archivists were asked to provide a copy of medical notes and results of diagnostic investigations, but patients were not contacted at this stage.
Quiz Ref IDAll patients' records were reviewed by a physician (G.D.) with the assistance of an adult neurologist (D.B.) and a pediatric neurologist (R.-M.B.). Reviewers were blinded to the immunization status of cases. Cases of GBS were classified into 1 of 3 categories (level 1, 2, or 3) based on the Brighton criteria of decreasing certainty in diagnosis.11 A fourth category (level 4) was used for cases with a diagnosis of GBS confirmed by a neurologist but with insufficient evidence to meet Brighton levels 1 through 3 definitions. The date of disease onset was determined for each case, based on the history of the earliest signs and symptoms related to the current neurological disease. Finally, the immunization status of each GBS case was verified in the provincial immunization registry, including the date of vaccine administration and the type of vaccine. Cases of GBS with disease onset between October 13, 2009, and March 31, 2010, were selected for statistical analyses.
The number of residents in the province of Quebec and their distribution by age and sex were estimated from census data as of July 1, 2009, provided by the Quebec Statistics Institute. The number of persons who had received a pandemic influenza vaccine by day of vaccination was extracted from the immunization registry. The number of persons not vaccinated was estimated as the number of residents minus the number of persons vaccinated. Based on observations made at the time of the influenza A(H1N1) vaccinations,3 GBS incidence rates in the exposed population were calculated using 3 risk periods: 8 weeks (56 days), 6 weeks (42 days), and 4 weeks (28 days) after vaccine administration. Incidence rates of GBS in the unexposed population were calculated by combining the incidence among persons not vaccinated and the incidence among those vaccinated and observed up to the date of vaccine administration and more than 8 weeks after vaccine administration. Age- and sex-adjusted rate ratios or relative risks (RRs) and their 95% confidence intervals were estimated using a Poisson model.12 Attributable risk per 1 million vaccine doses was computed as the number of GBS cases observed during a specific risk period multiplied by (RR − 1)/RR.12 For comparison of percentages, the χ2 or Fisher exact tests were used. Statistical significance was considered for P <.05 by 2-sided test.
The self-controlled case-series method, as described by Whitaker et al,13 was also used for estimating RRs. In this method, the analysis is restricted to persons who have been vaccinated and developed the condition under study. The method allows for the control of all permanent characteristics of patients in addition to seasonal variation in risk. Because a recent history of GBS may be a reason to decline influenza vaccination, the analysis was restricted to persons who were vaccinated before GBS onset. The incidence density of GBS during the risk period (≤8 weeks after vaccine administration) was compared with the incidence density during a reference period extending from the end of the exposure period to the end of the study period.
Several analyses were performed using different definitions of GBS cases and of the risk and reference periods. Seasonality was controlled for by using a dichotomous indicator variable representing the winter period (December 21, 2009–March 20, 2010) as opposed to another period, or a categorical variable representing each calendar month. A stratified analysis was performed to take into account possible GBS risk associated with the pandemic influenza infection, and 2 periods of vaccine administration were considered: up to November 28, 2009, when 20% or more of viral diagnostic test results were positive for influenza A(H1N1),14 or after this date. Another stratified analysis was performed according to existence vs no existence of a history of respiratory infection or influenzalike illness prior (<1 month) to hospital admission, as reported in the medical chart.
To exclude a possible bias related to selective recording of GBS cases with a history of vaccination, an analysis was restricted to cases identified in the hospital administrative database only. For this analysis, truncation of the observation period on March 17, 2010, was performed to exclude GBS cases possibly admitted late in March and discharged in April, after the date of closure of the hospital file on March 31, 2010. Statistical analyses were performed using SAS software, version 9.2 (SAS Institute Inc).
During the active surveillance period, 61 possible GBS cases were reported to public health authorities. Seventy-seven possible GBS cases were retrospectively identified in the MEDECHO hospital admission database. Thirty-seven cases were found in both sources, for a total of 101 cases. For all 101, medical charts were retrieved and analyzed. Eighteen possible cases were excluded: 12 cases with a final diagnosis other than GBS, 2 recurrent GBS cases, 2 cases with disease onset before October 13, 2009, and 2 other cases with onset after March 31, 2010. Thus, 83 cases were included in the analysis. The overall GBS incidence rate in the study population, representing 3 623 046 person-years of observation, was 2.3 per 100 000.
Of the 83 confirmed GBS cases included in the analysis, 42 had been immunized before disease onset (1-121 days after immunization) and all had received the ASO3 adjuvant H1N1 vaccine. For 25 cases, disease onset was 8 or fewer weeks after the vaccine was administered and they were considered exposed, whereas the 17 other cases were immunized more than 8 weeks before disease onset and were considered unexposed. Thus, for the cohort analysis, 25 GBS cases were considered exposed and 58 cases were considered unexposed.
The characteristics of GBS cases according to exposure status are shown in Table 1. Forty-nine cases were classified in the Brighton level 1 category, 22 cases in level 2, and 12 cases in level 4. The distribution of cases according to diagnostic category was similar in exposed and unexposed cases. The percentage of male patients was 69%. The median age was 49 years (range, 1-89 years). The percentage of elderly patients was higher in the exposed group than the unexposed group. The majority of patients (96%) were hospitalized; 25% developed severe paralysis of the lower limbs and were unable to walk at some point; and 17% developed respiratory distress syndrome and required intubation and/or assisted ventilation. Four patients died, all of whom were older than 60 years. Conditions occurring within 1 month before GBS onset as reported in medical records included a respiratory tract infection or influenzalike illness in 36% of cases, gastroenteritis in 18%, and trauma in 4%. A history of infection during the month prior to hospitalization was less frequent in exposed than in unexposed patients. The median interval between disease onset and hospitalization was 5 days (range, 1-34 days).
Of the 83 confirmed GBS cases identified during the 6-month study period, 56 (67% of total) occurred during a 12-week period from October 18, 2009 (2009 Centers for Disease Control and Prevention [CDC] week 42) to January 9, 2010 (2010 CDC week 1). The cluster was mostly explained by cases occurring in persons who were recently (≤8 weeks) immunized (22/56). Details on the distribution of cases are provided in eFigure 1.
Results of Poisson models are shown in Table 2. Relative risks were larger than 1 for all selected risk periods and case definitions. Because a majority of GBS cases occurred shortly following vaccine administration, estimates were the highest and were statistically significant for the 4-week postvaccination risk period (RR, 2.75; 95% CI, 1.63-4.62 for all confirmed cases and RR, 2.26; 95% CI, 1.24-4.09 for Brighton level 1-3 cases). The number of GBS cases possibly attributable to vaccination ranged from 1.3 (Brighton level 1-3 cases; weeks 1-6) to 2.7 (Brighton level 1-4 cases; weeks 1-4) per 1 million doses (median, 2.0 per 1 million doses).
As shown in Table 3, an excess risk was observed only in persons aged 50 years or older. In this group, the age- and sex-adjusted RRs were 2.69 (95% CI, 1.51-4.80) for all confirmed cases and 2.85 (95% CI, 1.56-5.21) for Brighton level 1 through 3 cases during the 8-week postvaccination period.
Results of analyses using the self-controlled case-series method are shown in Table 4. In the base model with no covariate, all RR estimates were greater than 1 and the highest value was for all confirmed cases during the 4-week postvaccination period (RR, 3.02; 95% CI, 1.64-5.56). Estimates were similar when a different definition of the risk period was used (model 2), when seasonality was considered (models 3A and 3B), when cases were stratified according to the vaccination period (model 4), or when cases were restricted to those identified in the hospital discharge database (model 5). Risk estimates tended to be higher for cases with a negative rather than a positive history of respiratory infection or influenzalike illness (model 6).
In Quebec, a cluster of GBS cases was observed shortly after the start of the mass immunization campaign using an ASO3 adjuvant 2009 influenza A(H1N1) vaccine. Two different methods were used to assess the risk associated with vaccine administration, and different definitions were applied for GBS cases and the postvaccination risk period. All RR estimates were greater than 1 and statistically significant for the 4-week postvaccination period. The excess risk was observed only in persons aged 50 years or older. The number of cases attributable to vaccination was approximately 2 per 1 million doses.
The postvaccination GBS cluster observed in Quebec is unlikely to have been caused by influenza infection. The second wave of the 2009 pandemic peaked during the first week of November, 3 to 4 weeks before the peak of vaccine administration, which occurred between November 16 and December 6, 2009.8,9Quiz Ref IDIn the context of wait times in vaccination clinics, it is reasonable to assume that people who developed an influenzalike illness did not rush to get a vaccine as soon as they recovered. Also, people who received 1 dose of adjuvant H1N1 vaccine were rapidly protected.15,16 It is thus very unlikely that the rate of pandemic influenza infection would have been higher in the vaccinated portion than in the unvaccinated portion of the population. No GBS cluster was observed in November-December 2009 among unvaccinated persons. Also, RRs remained statistically significant when the analysis was restricted to persons vaccinated after the period of intense circulation of the pandemic virus.8,9
The association between H1N1 influenza vaccination and GBS was found among patients with a negative history but not in those with a positive history of respiratory infection or influenzalike illness. However, lack of power prevents any definitive conclusion for patients with a positive history. In our study, patients were not contacted and interviewed about influenzalike illness prior to disease onset. Information on respiratory tract and gastrointestinal tract infections was extracted from medical charts of GBS patients. Although consulted neurologists reported that collecting data on recent infections and immunizations is routinely performed in the evaluation of patients with suspected GBS, an information bias cannot be excluded. We cannot exclude the possibility of better documentation of respiratory tract infections in persons with no history of recent vaccination.
It is reasonable to think that people who are more likely to seek care for minor disease would also be more likely to get immunized. Such bias would underestimate the true RR when adjusting for reported infections. The risk associated with H1N1 vaccination may also be underestimated in our study if influenza infection plays a role in the etiology of GBS, as suggested in a few studies,17 and if pandemic virus infections were more frequent among persons not vaccinated than persons vaccinated, which is a plausible hypothesis.
Quiz Ref IDThe seasonal influenza vaccination campaign in Quebec was postponed and started in late January 2010.8 Seasonal vaccine uptake was very low compared with previous years.18 During the 2009-2010 winter, seasonal influenza viruses did not circulate highly, representing 2% of all isolates.14 Thus, seasonal influenza infection or vaccination is unlikely to explain the GBS cluster among persons vaccinated mostly in November 2010.8
In Quebec, seasonal variation in the incidence of GBS has not been marked in recent years, as measured in the hospital discharge database (eFigure 2). In our study, results were not modified when this factor was taken into account in the self-controlled cases-series analysis or when GBS cases were stratified according to time of vaccine administration.
The completeness of GBS case identification is an important element to discuss, as selective underreporting could bias RR estimates. Our study is based on 2 independent sources, and several measures were implemented to encourage reporting by neurologists during the period of active surveillance. The overall GBS rate was 2.3 per 100 000 person-years, which is higher than the rates between 1.1 and 1.8 that have been reported in other studies.2 Using the traditional capture-recapture method,12 we estimated that 15 Brighton level 1 through 4 (11 level 1-3) GBS cases could have been missed. Including those hypothetical cases in the unvaccinated group in the cohort analysis (the “worst-case” scenario) would not change the trend (RR for weeks 1-4, GBS level 1-4, 2.14; RR for weeks 1-4, GBS level 1-3, 1.70).
Inconsistent results have been reported from other studies on the association between 2009 influenza A(H1N1) vaccines and GBS. A few studies lacked the power to detect an association of the magnitude of a few cases per 1 million doses.19-21 Results of a case-control study in 5 countries in Europe are difficult to interpret because both adjuvant and nonadjuvant vaccines were used, vaccination coverage varied widely between countries, and case ascertainment was not standardized.22
In the United Kingdom, the majority of H1N1 vaccines administered in 2009 contained the ASO3 adjuvant, as in Quebec. Cases of GBS were identified using 3 independent sources, including neurologists, the GBS support group, and the Hospital Episode Statistics database.23 Information on seasonal and H1N1 influenza vaccination and on preceding infections was obtained from patients' family physicians. A large number of statistical analyses were performed using different methods to take into account uncertainties in the data set. Although the majority of risk estimates were higher than 1, none was statistically significant.
In the United States, results of the CDC's Emerging Infections Program showed a statistically significant association between nonadjuvant 2009 influenza A(H1N1) influenza vaccines and GBS using the self-controlled series method.24 The RR during the 6-week postvaccination period varied between 2.1 (95% CI, 1.2-3.5) and 3.0 (95% CI, 1.4-6.4) according to the definition of the reference period. The corresponding attributable risks were 1.5 (95% CI, 0.3-3.4) and 2.8 (95% CI, 0.6-7.4) per 1 million doses administered.
Quiz Ref IDResults of our study are consistent with the existence of a risk excess of about 2 GBS cases per 1 million doses in the 4 weeks following administration of 2009 influenza A(H1N1) influenza vaccine. In Quebec, the individual risk of hospitalization following a documented influenza A(H1N1) infection was 1 per 2500 and the risk of death was 1/73 000.8 The H1N1 vaccine was very effective in preventing infections and complications.15,16 It is likely that the benefits of immunization outweigh the risks.
Corresponding Author: Philippe De Wals, MD, PhD, Département de Médecine Sociale et Préventive, Université Laval, CRIUCPQ, 2725 Chemin Ste-Foy, Québec, QC G1V 4G5, Canada (Philippe.De.Wals@ssss.gouv.qc.ca).
Author Contributions: Dr De Wals had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: De Wals, Deceuninck, Toth, Boulianne, Landry, De Serres.
Acquisition of data: Deceuninck, Toth, Boulianne.
Analysis and interpretation of data: De Wals, Deceuninck, Brunet, Boucher, De Serres.
Drafting of the manuscript: De Wals, Deceuninck.
Critical revision of the manuscript for important intellectual content: De Wals, Deceuninck, Toth, Boulianne, Brunet, Boucher, Landry, De Serres.
Statistical analysis: Deceuninck.
Obtained funding: De Wals, Boulianne, De Serres.
Administrative, technical, or material support: De Wals, Deceuninck, Toth, Boulianne, Landry.
Study supervision: De Wals, De Serres.
Conflict of Interest Disclosures: All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Dr De Wals reports receiving research grants, honoraria, and reimbursements of travel expenses from vaccine manufacturers including GlaxoSmithKline, Novartis, Sanofi Pasteur, Merck, and Pfizer. Ms Boulianne reports receiving research grants from Sanofi Pasteur and GlaxoSmithKline. Dr De Serres reports receiving research grants from Sanofi Pasteur and GlaxoSmithKline. No other disclosures were reported.
Funding/Support: This study was funded by the Ministère de la Santé et des Services sociaux du Québec and by the Public Health Agency of Canada-Canadian Institutes for Health Research Influenza Research Network.
Role of the Sponsors: The sponsors had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; or preparation, review, and approval of the manuscript.
Additional Contributions: We thank all the health care professionals in Quebec who participated in the data collection.
2. McGrogan A, Madle GC, Seaman HE, de Vries CS. The epidemiology of Guillain-Barré syndrome worldwide: a systematic literature review.
Neuroepidemiology. 2009;32(2):150-16319088488
PubMedGoogle ScholarCrossref 3.Langmuir AD, Bregman DJ, Kurland LT, Nathanson N, Victor M. An epidemiologic and clinical evaluation of Guillain-Barré syndrome reported in association with the administration of swine influenza vaccines.
Am J Epidemiol. 1984;119(6):841-8796328974
PubMedGoogle Scholar 4.Institute of Medicine. Immunization Safety Review: Influenza Vaccines and Neurological Complications. Washington, DC: National Academies Press; 2003
6.Nachamkin I, Shadomy SV, Moran AP,
et al. Anti-ganglioside antibody induction by swine (A/NJ/1976/H1N1) and other influenza vaccines: insights into vaccine-associated Guillain-Barré syndrome.
J Infect Dis. 2008;198(2):226-23318522505
PubMedGoogle ScholarCrossref 7.Stratton K, Ford A, Rusch E, Wright Clayton E, eds; Institute of Medicine Committee to Review Adverse Effects of Vaccines.
Adverse Effects of Vaccines: Evidence and Causality. Washington, DC: National Academies Press; 2012. http://www.nap.edu/catalog.php?record_id=13164. Accessed June 14, 2012 11.Sejvar JJ, Kohl KS, Gidudu J,
et al; Brighton Collaboration GBS Working Group. Guillain-Barré syndrome and Fisher syndrome: case definitions and guidelines for collection, analysis, and presentation of immunization safety data.
Vaccine. 2011;29(3):599-61220600491
PubMedGoogle ScholarCrossref 12.Rothman KJ, Greenland S. Modern Epidemiology. 2nd ed. Boston, MA: Lippincott Williams & Wilkins; 1998
13.Whitaker HJ, Farrington CP, Spiessens B, Musonda P. Tutorial in biostatistics: the self-controlled case series method.
Stat Med. 2006;25(10):1768-179716220518
PubMedGoogle ScholarCrossref 15.Skowronski DM, Janjua NZ, De Serres G,
et al. Effectiveness of AS03 adjuvanted pandemic H1N1 vaccine: case-control evaluation based on sentinel surveillance system in Canada, autumn 2009.
BMJ. 2011;342:c729721292718
PubMedGoogle ScholarCrossref 16.Van Buynder PG, Dhaliwal JK, Van Buynder JL,
et al. Protective effect of single-dose adjuvanted pandemic influenza vaccine in children.
Influenza Other Respi Viruses. 2010;4(4):171-17820629771
PubMedGoogle ScholarCrossref 17.Lehmann HC, Hartung HP, Kieseier BC, Hughes RAC. Guillain-Barré syndrome after exposure to influenza virus.
Lancet Infect Dis. 2010;10(9):643-65120797646
PubMedGoogle ScholarCrossref 19.Wu J, Xu F, Lu L,
et al. Safety and effectiveness of a 2009 H1N1 vaccine in Beijing.
N Engl J Med. 2010;363(25):2416-242321158658
PubMedGoogle ScholarCrossref 20.Bardage C, Persson I, Örtqvist Å, Bergman U, Ludvigsson JF, Granath F. Neurological and autoimmune disorders after vaccination against pandemic influenza A(H1N1) with a monovalent adjuvanted vaccine: population based cohort study in Stockholm, Sweden.
BMJ. 2011;343:d595621994316
PubMedGoogle ScholarCrossref 21.Grimaldi-Bensouda L, Alpérovitch A, Besson G,
et al; GBS-PGRx Study Group. Guillain-Barre syndrome, influenzalike illnesses, and influenza vaccination during seasons with and without circulating A/H1N1 viruses.
Am J Epidemiol. 2011;174(3):326-33521652600
PubMedGoogle ScholarCrossref 22.Dieleman J, Romio S, Johansen K, Weibel D, Bonhoeffer J, Sturkenboom M.VAESCO-GBS Case-Control Study Group. Guillain-Barre syndrome and adjuvanted pandemic influenza A(H1N1) 2009 vaccine: multinational case-control study in Europe.
BMJ. 2011;343:d390821750072
PubMedGoogle ScholarCrossref 23.Andrews N, Stowe J, Al-Shahi Salman R, Miller E. Guillain-Barré syndrome and H1N1 (2009) pandemic influenza vaccination using an AS03 adjuvanted vaccine in the United Kingdom: self-controlled case series.
Vaccine. 2011;29(45):7878-788221875631
PubMedGoogle ScholarCrossref 24.Tokars JI, Lewis P, DeStefano F,
et al. The risk of Guillain-Barré syndrome associated with influenza A(H1N1) 2009 monovalent vaccine and 2009-2010 seasonal influenza vaccines: results from self-controlled analyses.
Pharmacoepidemiol Drug Saf. 2012;21(5):546-55222407672
PubMedGoogle ScholarCrossref