[Skip to Content]
[Skip to Content Landing]
Figure 1.
Conceptual Framework of Patient-Reported Nasal Problems Related to Rhinoplasty
Conceptual Framework of Patient-Reported Nasal Problems Related to Rhinoplasty
Figure 2.
Final Standardized Cosmesis and Health Nasal Outcomes Survey
Final Standardized Cosmesis and Health Nasal Outcomes Survey
Table 1.  
Exploratory Factor Analysis of the SCHNOS—Factors, Eigenvalues, Factor Loadings (Pattern Matrix), and Unique Variances
Exploratory Factor Analysis of the SCHNOS—Factors, Eigenvalues, Factor Loadings (Pattern Matrix), and Unique Variances
Table 2.  
Correlation Matrix of the 10-Item SCHNOS
Correlation Matrix of the 10-Item SCHNOS
1.
Ishii  LE, Tollefson  TT, Basura  GJ,  et al.  Clinical practice guideline: improving nasal form and function after rhinoplasty.  Otolaryngol Head Neck Surg. 2017;156(2 suppl):S1-S30.PubMedGoogle ScholarCrossref
2.
Rhee  JS, Arganbright  JM, McMullin  BT, Hannley  M.  Evidence supporting functional rhinoplasty or nasal valve repair: a 25-year systematic review.  Otolaryngol Head Neck Surg. 2008;139(1):10-20.PubMedGoogle ScholarCrossref
3.
Most  SP, Moubayed  SP.  Patient-reported outcome measures for facial plastic surgery: a specialty finally gets to go to the PROM.  JAMA Facial Plast Surg. 2017;19(2):101.PubMedGoogle ScholarCrossref
4.
Stewart  MG, Witsell  DL, Smith  TL, Weaver  EM, Yueh  B, Hannley  MT.  Development and validation of the Nasal Obstruction Symptom Evaluation (NOSE) scale.  Otolaryngol Head Neck Surg. 2004;130(2):157-163.PubMedGoogle ScholarCrossref
5.
Lipan  MJ, Most  SP.  Development of a severity classification system for subjective nasal obstruction.  JAMA Facial Plast Surg. 2013;15(5):358-361.PubMedGoogle ScholarCrossref
6.
Barone  M, Cogliandro  A, Di Stefano  N, Tambone  V, Persichetti  P.  A systematic review of patient-reported outcome measures after rhinoplasty.  Eur Arch Otorhinolaryngol. 2017;274(4):1807-1811.PubMedGoogle ScholarCrossref
7.
Acaster  S, Cimms  T, Lloyd  A.  The Design and Selection of Patient-Reported Outcomes Measures (PROMs) for Use in Patient Centered Outcomes Research. Washington, DC: Patient Centered Outcomes Research Institute; 2012:1-82.
8.
US Department of Health and Human Services, US Food and Drug Administration, Center for Drug Evaluation and Research, Center for Biologics Evaluation and Research, Center for Devices and Radiological Health.  Guidance for industry: patient-reported outcome measures: use in medical product development to support labeling claims: draft guidance.  Health Qual Life Outcomes. 2006;4:79.PubMedGoogle ScholarCrossref
9.
Aaronson  N, Alonso  J, Burnam  A,  et al.  Assessing health status and quality-of-life instruments: attributes and review criteria.  Qual Life Res. 2002;11(3):193-205.PubMedGoogle ScholarCrossref
10.
Mokkink  LB, Terwee  CB, Patrick  DL,  et al.  The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study.  Qual Life Res. 2010;19(4):539-549.PubMedGoogle ScholarCrossref
11.
Alsarraf  R.  Outcomes research in facial plastic surgery: a review and new directions.  Aesthetic Plast Surg. 2000;24(3):192-197.PubMedGoogle ScholarCrossref
12.
Bulut  C, Wallner  F, Plinkert  PK, Baumann  I.  Development and validation of the Functional Rhinoplasty Outcome Inventory 17 (FROI-17).  Rhinology. 2014;52(4):315-319.PubMedGoogle Scholar
13.
Lee  MK, Most  SP.  A comprehensive quality-of-life instrument for aesthetic and functional rhinoplasty: the RHINO scale.  Plast Reconstr Surg Glob Open. 2016;4(2):e611.PubMedGoogle ScholarCrossref
14.
Naraghi  M, Atari  M.  Development and validation of the expectations of Aesthetic Rhinoplasty Scale.  Arch Plast Surg. 2016;43(4):365-370.PubMedGoogle ScholarCrossref
15.
Klassen  AF, Cano  SJ, East  CA,  et al.  Development and psychometric evaluation of the FACE-Q scales for patients undergoing rhinoplasty.  JAMA Facial Plast Surg. 2016;18(1):27-35.PubMedGoogle ScholarCrossref
Original Investigation
AAFPRS Annual Fall Meeting Featured Article
Jan/Feb 2018

The 10-Item Standardized Cosmesis and Health Nasal Outcomes Survey (SCHNOS) for Functional and Cosmetic Rhinoplasty

Author Affiliations
  • 1Division of Facial Plastic and Reconstructive Surgery, Department of Otolaryngology–Head and Neck Surgery, Stanford Hospital and Clinics, Stanford, California
  • 2Division of Otolaryngology–Head and Neck Surgery, Université de Montréal, Montreal, Quebec, Canada
  • 3Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, California
  • 4Department of Physical and Rehabilitation Medicine, Turku University Hospital, University of Turku, Turku, Finland
JAMA Facial Plast Surg. 2018;20(1):37-42. doi:10.1001/jamafacial.2017.1083
Key Points

Question  Can we develop and validate a novel patient-reported outcome measure (PROM) questionnaire to evaluate both functional and cosmetic outcomes of rhinoplasty?

Findings  In this survey study, the Standardized Cosmesis and Health Nasal Outcomes Survey (SCHNOS) was developed and validated using internationally accepted PROM development methodology. It was also field tested, and significant differences were found between the functional and cosmetic rhinoplasty populations, both preoperative and postoperative.

Meaning  The 10-item SCHNOS provides a short, validated questionnaire that we recommend for use in all functional or cosmetic rhinoplasty patients.

Abstract

Importance  Rhinoplasty is a common operation in which shape and function are intimately related, whether the procedure is cosmetic, functional, or combined in nature. There is currently no properly developed and validated patient-reported outcome measure (PROM) to evaluate both functional and cosmetic components of rhinoplasty.

Objective  To develop, validate, and field test the Standardized Cosmesis and Health Nasal Outcomes Survey (SCHNOS) to evaluate both functional and cosmetic outcomes of rhinoplasty.

Design, Setting, and Participants  Survey development study between October 2016 and April 2017 in a tertiary referral facial plastic and reconstructive surgery clinic. Preoperative and postoperative adult patients undergoing rhinoplasty, whether cosmetic or reconstructive, were included. A fifth group of adult nonrhinoplasty patients (facial cosmetic or reconstructive) were also included for the field test.

Main Outcomes and Measures  Generated and reduced items, psychometric validation measures of the SCHNOS, and differences on scales between groups.

Results  For survey development, a total of 18 patients and 5 experts were interviewed. Of these patients, 5 were male, and 13 were female. Their mean (SD) age was 38 (14.8) years (range, 20-64 years). Field testing included 191 patients (67% were women and the mean [SD] age was 41.5 [15.8] years). A total of 10 items were included after generation, cognitive interviews, and item reduction. The 10 items represent 2 domains: nasal obstruction (first 4 items) and nasal cosmesis (last 6 items). For both domains, Cronbach α was excellent: 0.94 (95% CI, 0.92-0.95) for obstruction and 0.94 (95% CI, 0.93-0.95) for cosmesis. Exploratory factor analysis using scree plots for each domain showed that the domains are unidimensional in nature with each domain evaluating what it is intended to assess (nasal obstruction and cosmesis). The factor loading estimates were high for all the items, varying from 0.74 to 0.92. Kruskal-Wallis testing showed a significance level of P < .001 when evaluating the difference between groups (preoperative cosmetic, postoperative cosmetic, preoperative functional, postoperative functional, and nonrhinoplasty) for all individual questions, composite scores, and Nasal Obstruction Symptom Evaluation (NOSE) score. Correlations between the obstruction composite score and the NOSE scores were r = 0.943 (P < .001), which is very strong. The obstruction and cosmesis composite scores were only weakly correlated (r = 0.388; P < .001).

Conclusions and Relevance  We have developed and validated a new PROM to evaluate both functional and cosmetic rhinoplasty patients. The domains of obstruction and cosmesis were found to be internally consistent and unidimensional. The SCHNOS provides a short, validated questionnaire that we recommend for use in all functional or cosmetic rhinoplasty patients.

Level of Evidence  N/A.

Introduction

Over 200 000 rhinoplasty procedures are performed annually in the United States, making rhinoplasty one of the most common surgical procedures performed in facial plastic surgery and otolaryngology.1 Cosmetic rhinoplasty is aimed at altering the shape of the nose, whereas functional rhinoplasty aims to correct nasal obstruction at the level of the nasal valve.2 Shape and function of the nose are intimately related: aesthetic operations may alter nasal breathing and, vice versa, functional operations may alter nasal appearance.2 The precise and standardized description of both components is paramount in patients undergoing rhinoplasty.

Relevant patient-reported outcome measures (PROMs) are needed to evaluate outcomes in everyday clinical practice.3 For example, to assess the severity of nasal obstruction, the Nasal Obstruction Symptom Evaluation (NOSE) scale was introduced in 2004.4 It is a 5-item questionnaire that produces a score ranging from 0 to 100, and is currently the most widely used PROM to evaluate nasal obstruction after nasal surgery. We have previously established that a score of 30 on the NOSE survey best differentiated patients with and without nasal obstruction.5 We have also developed a classification system categorizing patients as having mild (range, 5-25), moderate (range, 30-50), severe (range, 55-75), or extreme (range, 80-100) nasal obstruction.5

A systematic review of PROMs designed to assess rhinoplasty outcomes was recently conducted by Barone et al.6 A total of 10 surgery-specific questionnaires were identified and divided into 3 categories: aesthetic, functional, and aesthetic/functional. However, none of the aesthetic/functional scales met appropriate development and validation criteria, as none of these questionnaires were developed using patient input through the form of interviews.

The objective of this study was to develop and validate a novel PROM to evaluate both functional and cosmetic outcomes of rhinoplasty both in the preoperative and postoperative periods. The current questionnaire is termed the Standardized Cosmesis and Health Nasal Outcomes Survey (SCHNOS).

Methods

We have developed the SCHNOS in 4 phases as described herein. The protocol was approved by the Stanford University institutional review board. Internationally accepted and recognized guidelines for developing PROMs have been followed.7-10 Patients provided written informed consent and were not compensated.

Phase 1: Defining a Conceptual Framework and Relevant Items

Each of the 10 scales evaluated by Barone et al6 was examined item by item to outline a conceptual framework for a new questionnaire. For the task, 4 main concepts were considered relevant: (1) nasal obstruction, (2) overall nasal cosmesis, (3) specific nasal cosmesis, and (4) social perception of nasal appearance (Figure 1). At a university-based facial plastic and reconstructive surgery clinic, 18 patients undergoing rhinoplasty were interviewed in a face-to-face fashion between October 2016 and February 2017. The patients were approached by one of the principal investigators (S.P.M.). Each interview lasted for 20 minutes and consisted of open-ended questions about the most important outcomes of rhinoplasty expected or perceived by the respondents. Using the conceptual framework shown in Figure 1, the respondents were also asked to rank the items in the order of importance. They were also asked to estimate the importance of nasal obstruction vs nasal cosmesis with the combination of both totaling 100%. Next, using the conceptual framework shown in Figure 1 as a guideline, 5 senior surgeons with primary expertise in rhinoplasty were interviewed. Based on the results of the patients’ and the experts’ interviews, a preliminary set of questionnaire items was suggested. Each item was put into a Likert-type scale from 0 to 5 with 0 indicating “no problem” and 5 indicating “extreme problem.” To eliminate potential ambiguity, the preliminary questionnaire (eFigure 1 in the Supplement) containing 13 items was administered to 5 patients in the form of cognitive debriefing interviews. Patients were asked to think aloud as they were reading the questionnaire instructions and items. They were also asked to suggest improvements to clarify and simplify the questions.

Phase 2: Item Reduction

The preliminary 13-item questionnaire was administered to a sample of the 191 English-speaking patients between February and April 2017. No instructions other than “please fill this questionnaire out” were given. The NOSE scale4 was concurrently administered to all the respondents. To minimize potential redundancy, all the items with Spearman correlation coefficients greater than 0.85 were removed resulting in a final 10-item scale (Figure 2). Correlations were calculated using SPSS statistical software (version 22; IBM Corp).

Phase 3: Psychometric Evaluation

The internal consistency of SCHNOS was examined using Cronbach α; α ≥ .90 was considered excellent,α ≥ .80 good, α ≥ .70 acceptable, α ≥ .60 questionable, α ≥ .50 poor, and α < .50 was considered unacceptable. In addition, the Cronbach α 1-sided 95% CI was calculated. Exploratory factor analysis was used to approximate the construct structure of the SCHNOS (Table 1). According to the objective and the design of the study, the SCHNOS consisted of 2 separate domains. The first 4 items describe the severity of nasal obstruction experienced by a respondent. The final 6 items describe the severity of a subjective cosmetic disadvantage and overall state of mood and self-esteem due to nasal problems. The goal was to determine if each of these 2 domains of the SCHNOS measure only 1 latent trait (eg, disability) or if there are other possible significant latent variables affecting the results within each of the domains. The results were analyzed both numerically and graphically. Exploratory factor analysis (principal factors) was applied with a minimum eigenvalue for retention set at greater than 1.0 (Kaiser rule). Retained and excluded factors were also explored visually on a scree plot (visual approximation along with parallel analysis). Composite scores for nasal obstruction and nasal cosmesis were calculated separately as sums of the responses to each item from 0 to 5. This way the maximum total score for the nasal obstruction domain was 20, and, for the cosmesis domain, 30 points. Spearman rank correlation was used to assess the strength of correlation between different items. All the analyses in phase 3 were conducted using Stata/IC statistical software (release 14, StataCorp LP).

Phase 4: Field Testing

Finally, preliminary field testing of the questions and scores was performed in the same population of 191 patients. Patients were divided into 4 groups depending on their preoperative or postoperative status and whether their surgery was purely functional or had a cosmetic component. A fifth group of nonrhinoplasty patients, who presented to the clinic for other facial cosmetic or reconstructive reasons, was also incorporated into analysis. For each group, the results of the Likert scales of each question were compared using Kruskal-Wallis H testing using P <.05 as statistically significant. Composite scores for nasal obstruction and nasal cosmesis were also calculated by adding up the scores of questions domain-specific questions, and compared between groups using the same method (as described herein). Results of NOSE scores were also contrasted between groups with the same method. Pearson correlation coefficients between the composite scores and NOSE score were also calculated. For phase 4, SPSS statistical software (version 22; IBM Corp) was used.

Results

Of the 191 patients invited to answer the questionnaire, 191 of 191 answered (100% response rate): 67% (128) were women and 33% (63) were men. Their mean (SD) age was 41.5 (15.8) years. A total of 18 patients were invited to participate for the interviews and 18 of 18 accepted to participate (a 100% response rate): 9 cosmetic, 4 functional, and 4 combined cosmetic and functional rhinoplasty patients. Of these patients, 5 were male and 13 were female. Their mean (SD) age was 38 (14.8) years (range, 20-64 years). Nine patients were preoperative patients, and 9 patients were postoperative. A total of 5 experts were interviewed. Regardless of the indications for rhinoplasty, most of the patients, and all of the experts, perceived cosmesis and obstruction to be equally important during the initial interview phase. The most commonly elicited and prioritized concepts were included in the preliminary version of the questionnaire (eFigure 1 in the Supplement).

Of the 191 patients, 107 (56%) underwent rhinoplasty for cosmetic reasons, with or without a functional indication. Functional rhinoplasty was performed in 52 patients (27%). Thirty-two patients (17%) were nonrhinoplasty patients presenting for a variety of other conditions (eg, facial trauma, skin cancer reconstruction). Of the operative patients, the SCHNOS was applied preoperatively in 90 patients and after the surgery in 69 patients.

The correlation matrix for the 10 items is presented in Table 2. The correlations between pairs of items varied widely, from 0.13 to 0.85. As expected, the correlations were high within each particular domain but low between the items belonging to different domains. For all questions, less than 3.6% of data were missing. The missing data were included into analysis.

Internal consistency of each of 2 domains was examined using Cronbach α. For both domains, the Cronbach α was excellent: 0.94 (95% CI, 0.92-0.95) for the nasal obstruction domain and 0.94 (95% CI, 0.93-0.95) for the cosmesis domain. There was no need for increasing α by eliminating items.

The factor structures of the domains were evaluated using exploratory factor analysis (Table 2). For the nasal obstruction domain, the parallel analysis of the scree plot suggested a single retained factor with eigenvalue of 3.13 (eFigure 2A in the Supplement). The parallel analysis of the scree plot for the cosmesis domain suggested 2 retained factors with eigenvalues 4.35 and 0.28 (eFigure 2B in the Supplement). However, the second factor was substantially below the Kaiser cutoff point of retaining and, therefore, the domains were considered unidimensional. Factor loadings (pattern matrix) and unique variances of items are also presented in Table 2. The loading estimates were high for all the items varying from 0.74 to 0.92 (Table 2).

We then evaluated patient responses (n = 191 patients) for all individual questions, composite obstruction and cosmesis scores, as well as the NOSE score (eFigure 3 in the Supplement). Kruskal-Wallis testing showed a significance level of P < .001 when evaluating the difference between groups (preoperative cosmetic, postoperative cosmetic, preoperative functional, postoperative functional, and nonrhinoplasty) for all individual questions, composite scores, and NOSE score. Correlations between the obstruction composite score and the NOSE scores were r = 0.943 (P < .001), which is very strong. The obstruction and cosmesis composite scores were only weakly correlated (r = 0.388; P < .001).

Discussion

Four questionnaires have been identified in a previous systematic review6 as intended to assess both functional and cosmetic aspects of rhinoplasty. These are the Rhinoplasty Outcomes Evaluation,11 the Functional Rhinoplasty Outcome Inventory 17,12 the Rhinoplasty Health Inventory and Nasal Outcomes scale,13 and the Expectations of Aesthetic Rhinoplasty Scale.14 Unfortunately, none of these questionnaires were developed using patient input through the form of interviews, which limits their usefulness to assessing patient-centered outcomes. The advantage of the SCHNOS questionnaire is that the questions retained in the initial questionnaire were identified by patients as most relevant to their experience.

The most widely used and accepted PROM to assess nasal obstruction in rhinoplasty is currently the NOSE questionnaire, developed by Stewart et al.4 The nasal obstruction composite score of the SCHNOS showed excellent correlation with the NOSE questionnaire, with a Pearson correlation coefficient of 0.943. The SCHNOS may therefore be administered without the NOSE scale to assess the construct of nasal obstruction.

Currently, the PROM with the most robust methodology to assess nasal cosmesis is the FACE-Q, developed by Klassen et al.15 The nasal module of the FACE-Q involves 3 different questionnaires (satisfaction with nose [10 questions], adverse effects [4 questions], and satisfaction with nostrils [5 questions]), and the total of 19 questions can be a burden on a single patient. Based on our interviews, patients did not identify current adverse effects or nostrils as a priority during their cosmetic or functional rhinoplasty experience. The FACE-Q was developed using input from only 9 rhinoplasty patients compared with 18 in our study. Other disadvantages are that the rhinoplasty population was poorly defined as to whether it was functional or cosmetic, and the recall period for the FACE-Q is the past week. We believe that a recall period of a month is more appropriate to assess nasal function and cosmesis, mainly to avoid tainting the questionnaire by a recent upper respiratory tract infection. For these reasons, we believe that the SCHNOS cosmetic domain is comprehensive enough yet can avoid excessive patient burden.

Evaluation of functional and aesthetic changes in all types of rhinoplasty is paramount. The SCHNOS provides a short, validated questionnaire that we recommend for use in all functional or cosmetic rhinoplasty patients. There are no specific instructions other than handing the questionnaire to the patient and asking him or her to complete it. By tracking the individual patient’s progress, the clinician can identify problems that may not be evident during routine clinical interaction.

Limitations

The generalization of our results should be made with caution before the results are confirmed in bigger samples and in different settings. The Likert-type ordinal data used for the factor analysis were considered interval data. While common, such an approach is always an approximation, and this should be taken into account. The ordinal data obtained from the SCHNOS may further be analyzed using a Rasch analysis or an item response theory (both methods require big samples). However, the SCHNOS was created based on a sound conceptual framework, and it was tested using the commonly recommended psychometric methods for the initial evaluation of a new test.

Conclusions

We have developed and validated a new PROM to evaluate both functional and cosmetic rhinoplasty patients. The SCHNOS contains 2 independent domains describing nasal obstruction and nasal cosmesis. The domains were found to be internally consistent and unidimensional. The use of the SCHNOS was tested on practice in 191 patients presenting to a facial plastic surgery clinic. Future directions include establishing levels of meaningful change and minimal clinically important difference, evaluating the ability to detect change in a single patient, and establishing whether the questionnaire can be administered electronically rather than in paper format.

Back to top
Article Information

Corresponding Author: Sam P. Most, MD, Division of Facial Plastic and Reconstructive Surgery, Department of Otolaryngology–Head and Neck Surgery, Stanford Hospital and Clinics, 801 Welch Rd, Stanford, CA 94305 (smost@stanford.edu).

Accepted for Publication: May 30, 2017.

Published Online: September 7, 2017. doi:10.1001/jamafacial.2017.1083

Author Contributions: Dr Most had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Study concept and design: Moubayed, Ioannidis, Most.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Moubayed, Saltychev.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Moubayed, Ioannidis, Saltychev.

Administrative, technical, or material support: Most.

Study supervision: Most.

Conflict of Interest Disclosures: None reported.

Additional Contributions: We thank Rachel Schwartz, PhD, for help with the development of the Likert scale and the assistance of Monica Delorio, MA, for administrative assistance during questionnaire administration. Both are at Stanford University and were not compensated beyond their regular salaries.

References
1.
Ishii  LE, Tollefson  TT, Basura  GJ,  et al.  Clinical practice guideline: improving nasal form and function after rhinoplasty.  Otolaryngol Head Neck Surg. 2017;156(2 suppl):S1-S30.PubMedGoogle ScholarCrossref
2.
Rhee  JS, Arganbright  JM, McMullin  BT, Hannley  M.  Evidence supporting functional rhinoplasty or nasal valve repair: a 25-year systematic review.  Otolaryngol Head Neck Surg. 2008;139(1):10-20.PubMedGoogle ScholarCrossref
3.
Most  SP, Moubayed  SP.  Patient-reported outcome measures for facial plastic surgery: a specialty finally gets to go to the PROM.  JAMA Facial Plast Surg. 2017;19(2):101.PubMedGoogle ScholarCrossref
4.
Stewart  MG, Witsell  DL, Smith  TL, Weaver  EM, Yueh  B, Hannley  MT.  Development and validation of the Nasal Obstruction Symptom Evaluation (NOSE) scale.  Otolaryngol Head Neck Surg. 2004;130(2):157-163.PubMedGoogle ScholarCrossref
5.
Lipan  MJ, Most  SP.  Development of a severity classification system for subjective nasal obstruction.  JAMA Facial Plast Surg. 2013;15(5):358-361.PubMedGoogle ScholarCrossref
6.
Barone  M, Cogliandro  A, Di Stefano  N, Tambone  V, Persichetti  P.  A systematic review of patient-reported outcome measures after rhinoplasty.  Eur Arch Otorhinolaryngol. 2017;274(4):1807-1811.PubMedGoogle ScholarCrossref
7.
Acaster  S, Cimms  T, Lloyd  A.  The Design and Selection of Patient-Reported Outcomes Measures (PROMs) for Use in Patient Centered Outcomes Research. Washington, DC: Patient Centered Outcomes Research Institute; 2012:1-82.
8.
US Department of Health and Human Services, US Food and Drug Administration, Center for Drug Evaluation and Research, Center for Biologics Evaluation and Research, Center for Devices and Radiological Health.  Guidance for industry: patient-reported outcome measures: use in medical product development to support labeling claims: draft guidance.  Health Qual Life Outcomes. 2006;4:79.PubMedGoogle ScholarCrossref
9.
Aaronson  N, Alonso  J, Burnam  A,  et al.  Assessing health status and quality-of-life instruments: attributes and review criteria.  Qual Life Res. 2002;11(3):193-205.PubMedGoogle ScholarCrossref
10.
Mokkink  LB, Terwee  CB, Patrick  DL,  et al.  The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study.  Qual Life Res. 2010;19(4):539-549.PubMedGoogle ScholarCrossref
11.
Alsarraf  R.  Outcomes research in facial plastic surgery: a review and new directions.  Aesthetic Plast Surg. 2000;24(3):192-197.PubMedGoogle ScholarCrossref
12.
Bulut  C, Wallner  F, Plinkert  PK, Baumann  I.  Development and validation of the Functional Rhinoplasty Outcome Inventory 17 (FROI-17).  Rhinology. 2014;52(4):315-319.PubMedGoogle Scholar
13.
Lee  MK, Most  SP.  A comprehensive quality-of-life instrument for aesthetic and functional rhinoplasty: the RHINO scale.  Plast Reconstr Surg Glob Open. 2016;4(2):e611.PubMedGoogle ScholarCrossref
14.
Naraghi  M, Atari  M.  Development and validation of the expectations of Aesthetic Rhinoplasty Scale.  Arch Plast Surg. 2016;43(4):365-370.PubMedGoogle ScholarCrossref
15.
Klassen  AF, Cano  SJ, East  CA,  et al.  Development and psychometric evaluation of the FACE-Q scales for patients undergoing rhinoplasty.  JAMA Facial Plast Surg. 2016;18(1):27-35.PubMedGoogle ScholarCrossref
×