Importance
Many hospitals do not have inpatient dermatologic consultative services, and most have reduced availability of services during off-hours. Dermatologists based in outpatient settings can find it challenging to determine the urgency with which they need to evaluate inpatients when consultations are requested. Teledermatology may provide a valuable mechanism for dermatologists to triage inpatient consultations and increase efficiency, thereby expanding access to specialized care for hospitalized patients.
Objective
To evaluate whether a store-and-forward teledermatology system is reliable for the initial triage of inpatient dermatology consultations.
Design, Setting, and Participants
Prospective study of 50 consenting adult patients, hospitalized for any indication, for whom an inpatient dermatology consultation was requested between September 1, 2012, and April 31, 2013, at the Hospital of the University of Pennsylvania, an academic medical center. The participants were evaluated separately by both an in-person dermatologist and 2 independent teledermatologists.
Main Outcomes and Measures
The primary study outcomes were the initial triage and decision to biopsy concordance between in-person and teledermatology evaluations.
Results
Triage decisions were as follows: if the in-person dermatologist recommended the patient be seen the same day, the teledermatologist agreed in 90% of the consultations. If the in-person dermatologist recommended a biopsy, the teledermatologist agreed in 95% of cases on average. When the teledermatologist did not choose the same course of action, there was substantial diagnostic agreement between the teledermatologist and the in-person dermatologist. The Kendall τ rank correlation coefficients for initial triage concordance between the in-person dermatologist and teledermatologists were 0.41 and 0.48. The Cohen κ coefficients for decision to biopsy concordance were 0.35 and 0.61. The teledermatologists were able to triage 60% of consultations to be seen the next day or later. The teledermatologists were able to triage, on average, 10% of patients to be seen as outpatients after discharge.
Conclusions and Relevance
Teledermatology is reliable for the triage of inpatient dermatology consultations and has the potential to improve efficiency.
Teledermatology, which has expanded access to care for patients in a variety of settings, has been the subject of much research during the past decade.1,2 In the outpatient setting, teledermatology has been shown to be reliable and diagnostically accurate for a variety of dermatologic complaints and has been shown to reduce wait times, increase access, and improve patient satisfaction and quality of life.2-8 It also has been demonstrated to be more efficient and cost-effective in certain settings, especially when travel times are significant.9-12 In addition, as mobile technology has improved, there is increasing evidence that smartphones represent a simple, feasible, and reliable method for performing store-and-forward teledermatology consultations.4,13-17 However, some studies have questioned the reliability and accuracy of teledermatology.18,19
There is growing interest in exploring whether teledermatology can be applied to the inpatient setting. Many hospitals do not have inpatient dermatology consultative services, and most have reduced availability of dermatology consultation services during off-hours. Teledermatology could expand access to timely dermatologic services in these settings, in addition to reducing travel time and increasing efficiency for a community dermatologist providing inpatient consultative services to local hospitals. A part-time, office-based dermatologist could triage cases via teledermatology and determine the urgency of the cases, as well as potentially avoid unnecessary trips to the hospital or group nonurgent inpatient consultations to be seen together at a convenient time. Muir and colleages20 recently demonstrated that a store-and-forward teledermatology system can be applied effectively for the rapid initial triage and management of dermatologic complaints in the emergency department.
The purpose of this study was to evaluate whether a store-and-forward teledermatology system using a smartphone platform is reliable for the initial triage of inpatient dermatology consultations. Specifically, we sought to examine the initial triage and decision to biopsy concordance between in-person and teledermatology evaluations of patients for whom an inpatient dermatologic consultation is requested. A secondary aim was specific diagnostic agreement between the inpatient dermatologist directly assessing the patients and the teledermatologists performing remote evaluation.
Study Design and Patients
We conducted a prospective study of 50 inpatient dermatology consultations designed to evaluate whether teledermatology is reliable for the initial triage of inpatient dermatology consultations. Reliability, which differs from validity, can be defined as the degree to which different observers agree. The study protocol was approved by the institutional review board of the University of Pennsylvania, and all patients provided written informed consent. Participants were recruited from the population of inpatient dermatologic consultations requested at the Hospital of the University of Pennsylvania between September 1, 2012, and April 31, 2013. Participants were eligible for the study if they were older than 18 years and capable of providing written informed consent.
Study Schedule and Interventions
At the Hospital of the University of Pennsylvania, when an inpatient dermatologic consultation is requested, the patient is seen in person by a dermatology resident and attending physician. For study participants, in addition to providing a complete traditional inpatient consultation on the same day as the request in all cases, the in-person dermatologist (M.R.) recorded a triage decision based on when he believed the patient could have been seen and whether the patient needed a biopsy, using a standardized template (eFigure 1 in the Supplement). Triage decisions were grouped into 4 categories: the patient needs to be seen on the same day, by the next day, within the hospitalization (but not necessarily on the same day or next day), or as an outpatient. In addition, the in-person dermatologist listed up to 3 diagnoses under consideration for the patient using a standardized template (eFigure 2 in the Supplement). For patients enrolled in the study, in addition to the in-person consultation, a teledermatology consultation was submitted by a fourth-year medical student (J.S.B. or C.A.N.) who had completed a medicine subinternship and a 1-month dermatology elective and was not part of the inpatient team. The medical student was masked to the in-person dermatologic consultation note and collected information from the medical record and patient using the prompts in the AccessDerm smartphone platform (Vignet),21 with images captured by a smartphone camera. This consultation was evaluated by 2 independent attending dermatologists (C.L.K. and W.D.J.) experienced in teledermatology who have been involved in a number of teledermatologic studies.5,13,17 The teledermatologists completed the same triage and diagnosis forms as the in-person dermatologist and recorded a subjective self-assessment on whether they would “feel comfortable managing this patient with teledermatology alone.” Neither the in-person dermatologist nor the teledermatologists were aware of the others’ triage or biopsy decisions, diagnosis, or management plan.
The primary study outcomes were concordance of the triage plans and the decision to biopsy. Secondary outcomes included diagnostic agreement between the in-person dermatologist and the teledermatologists and a subjective self-assessment completed by the teledermatologists on whether they would be comfortable managing the consultation using teledermatology alone.
Concordance was assessed using the Cohen κ for bivariate decisions, such as whether to biopsy, and the Kendall τ rank correlation for naturally ordered multivariate decisions, such as the triage decision. In the setting of ordinal categories, the Kendall τ rank correlation is better able to take into account the degree of disagreement between observers than the Cohen κ.22 In addition, we examined the percentage of consultations for which the teledermatologist failed to triage a consultation to be seen the same day when the in-person dermatologist believed it was necessary or did not request a biopsy when the in-person dermatologist requested one. To assess diagnostic agreement, we coded the differential diagnoses as “complete,” “partial,” or “no” agreement based on text reading. Complete agreement was defined as the first diagnoses matching between the in-person dermatologist and teledermatologist. Partial agreement was defined as an overlap between the diagnoses listed by the in-person dermatologist and teledermatologist. Semantic differences in the dermatologic terminology were taken into consideration. Calculations were performed with JMP software, version 10 (SAS Institute, Inc).
Fifty patients were enrolled in the study. The Table summarizes the characteristics of the study population. Among the participants, the mean (SD) age was 55.2 (16.2) years, and 64% were women. The mean (SD) length of stay before the request for a dermatology consultation was 9.5 (13.5) days. Common serious comorbidities included oncologic conditions (48%), immunosuppression (34%), and heart failure (12%). Participants took a mean (SD) of 7.9 (3.9) medications, with 52% receiving intravenous antibiotics.
Frequencies of triage and biopsy decisions made by the in-person dermatologist and the teledermatologists are summarized in Figure 1 and Figure 2. Quiz Ref IDAfter completing the consultation, the in-person dermatologist determined that 66% of consultations could have been safely triaged to the next day or later, with 18% of consultations triaged to outpatient care. Teledermatologist 1 triaged 60% of consultations to the next day or later, with 12% deferred to outpatient care. Teledermatologist 2 triaged 60% of consultations to the next day or later, with 8% of consultations triaged to outpatient care. Notably, if the in-person dermatologist recommended that the patient be seen the same day, teledermatologists 1 and 2 each recommended the patient be seen the same day 90% of the time. When the teledermatologist did not recommend the patient be seen the same day, comparison of the differential diagnoses between the teledermatologist and in-person dermatologist revealed complete diagnostic agreement for 60% of cases and partial diagnostic agreement for the remaining 40%.
Biopsy requests were made by the in-person dermatologist, teledermatologist 1, and teledermatologist 2 for 42%, 64%, and 54% of the consultations, respectively. Quiz Ref IDIn addition, if the in-person dermatologist recommended a biopsy, teledermatologists 1 and 2 advocated a biopsy for 94% and 96% of the consultations, respectively. In all cases where the in-person dermatologist requested a biopsy and the teledermatologist did not, there was complete diagnostic agreement between the differential diagnoses of the teledermatologist and in-person dermatologist.
Following the categorization schema described by Landis and Koch,23 triage concordance between the in-person dermatologist and the teledermatologists was moderate. The Kendall τ rank correlation coefficient was 0.41 (95% CI, 0.18-0.60) between the in-person dermatologist and teledermatologist 1 and 0.48 (95% CI, 0.31-0.65) between the in-person dermatologist and teledermatologist 2. Interrater reliability between the 2 teledermatologists was also moderate, with a Kendall τ rank correlation coefficient of 0.41 (95% CI, 0.19-0.62).
Decision to biopsy concordance was fair to moderate. The Cohen κ coefficient was 0.35 (95% CI, 0.12-0.58) between the in-person dermatologist and teledermatologist 1 and 0.61 (95% CI, 0.39-0.82) between the in-person dermatologist and teledermatologist 2. Interrater reliability between the 2 teledermatologists was substantial, with a Cohen κ coefficient of 0.63 (95% CI, 0.42-0.84).
Quiz Ref IDFor the secondary outcome of diagnostic agreement, there was complete, partial, and no agreement for 64%, 20%, and 16% of the consultations, respectively, between the in-person dermatologist and teledermatologist 1 and for 56%, 26%, and 18% of the consultations, respectively, between the in-person dermatologist and teledermatologist 2. Comparing the 2 teledermatologists, there was complete, partial, and no agreement between teledermatologists 1 and 2 for 58%, 30%, and 12% of consultations, respectively. Thus, there was 82% to 88% complete to partial diagnostic agreement.
Quiz Ref IDFinally, in reference to the comfort level of managing the cases via this modality of care, teledermatologists 1 and 2 reported that they would “feel comfortable managing this patient with teledermatology alone” in 58% and 90% of cases, respectively.
Our findings suggest that teledermatology is reliable for the initial triage of inpatient dermatologic consultations at an academic medical center. Regarding triage decisions, the teledermatologists rarely failed to triage a consultation to be seen the same day when the in-person dermatologist believed it was necessary (<10% of cases). In addition, when the teledermatologist did not find it necessary to see the patient the same day but the in-person dermatologist believed it was imperative, the teledermatologist and in-person dermatologist had diagnostic agreement but had different management decisions. On determining whether to biopsy, the teledermatologists rarely failed to request a biopsy when the in-person dermatologist requested one (<5% of cases). In addition, when the teledermatologist did not find it necessary to biopsy but the in-person dermatologist determined it was required, the teledermatologist and in-person dermatologist agreed on the diagnosis, with the biopsy decision disagreement amounting to a difference in practice styles. In a study of clinic-based dermatologists, agreement on medical and clinic-based therapy was 85% and 77%, respectively,24 suggesting that some level of variation is to be expected when comparing triage and management decisions between clinicians.
In addition to being reliable, teledermatology may also increase efficiency when used to triage inpatient consultations. The teledermatologists on average were able to safely triage 60% of consultations to be seen the next day or later. For an outpatient dermatologist in practice serving as a part-time consultant for a hospital, deferring these consultations to a later time creates an opportunity to batch consultations together, thereby decreasing trips to the hospital. This may reduce disruptions to the teledermatologist’s outpatient clinic, both making the outpatient practice more efficient and optimizing the time spent while in the hospital. In addition, the teledermatologists on average were able to safely triage 10% of patients to outpatient care, eliminating the need to see these patients in the hospital. By reducing or eliminating trips to the hospital, teledermatology has the potential to increase the efficiency with which a dermatologist could provide inpatient consultative services to a community hospital. In addition, the in-person dermatologist believed that 66% of consultations could be seen the next day or later and 18% could be deferred to outpatient care, suggesting there is the possibility for even greater gains in clinical practice efficiency.
It may also be possible to manage some patients with teledermatology alone. Although our study design did not allow dermatologists to attempt to manage patients using teledermatology, teledermatologists 1 and 2 reported that they would “feel comfortable managing this patient with teledermatology alone” for 58% and 90% of consultations, respectively. However, teledermatologists requested biopsies for 54% to 64% of patients, which would require that someone at the hospital be capable of performing a biopsy to effectively implement this strategy. In addition, in our study, the teledermatologists requested more biopsies than did the in-person dermatologist. Other reports in the outpatient setting have not shown a consistent effect of teledermatology on the frequency of biopsies.5,25,26 However, if future studies in the inpatient setting also show an increase in recommended biopsies by remote teledermatology evaluation, that cost would have to be included in analyses regarding the benefits of teledermatology for inpatient dermatology consultations.
On review of cases in which the in-person dermatologist and teledermatologist disagreed about triage decisions or whether to biopsy, there were Quiz Ref ID3 major themes associated with discordance. First, there was a general trend toward more conservative management by the teledermatologists. The teledermatologists saw the patient earlier and requested more biopsies than did the in-person dermatologist. Second, there were inconsistent approaches to patients who had a known dermatologic diagnosis from their medical history, along with variation in the triage and biopsy decisions for patients with drug reactions, which is a common diagnosis associated with inpatient dermatologic consultations.27 These variations in care may again be representative of differences in the practice styles of the dermatologists involved in this study. Finally, there was often triage or decision to biopsy discordance for patients who had multiple, independent lesions. For instance, 1 patient had resolving cellulitis, known chronic graft vs host disease, and new erosions on the back. The in-person dermatologist focused on the cellulitis and graft vs host disease, while the teledermatologists concentrated on the graft vs host disease and new erosions.
For the secondary outcome of diagnostic agreement, there was partial or complete agreement between the in-person dermatologist and teledermatologist for more than 80% of consultations, which is consistent with other reports in the literature.2
The findings of this study should be interpreted in the context of the study design. One limitation is that the study was conducted at an academic medical center. Of the participants in this study, 48% had an oncologic comorbidity, 34% were immunosuppressed, and 52% were receiving intravenous antibiotics. These patients likely had more complex and acute illnesses than patients at a community hospital. Another limitation is that the consultations were submitted by medical students instead of by the primary team. While medical students may submit more comprehensive consultations, they have less experience and a less holistic understanding of the patient than the primary team and cannot provide specific details about the reason for the consultation as effectively as the primary team. In addition, there was no opportunity for follow-up communication or discussion with the primary team, which may have contributed to more conservative management by the teledermatologists. This design also prevented the teledermatologist from being able to institute a management plan and then follow up with the primary team using teledermatology, which could have increased the number of consultations that could be seen the next day or deferred to outpatient care. A further limitation of the study is that the standardized templates did not allow for multiple lesions to be evaluated separately, which created documentation challenges for a small number of consultations for patients with multiple dermatology complaints. Since many of these limitations could be expected to increase the difficulty of the teledermatology consultation, our findings may represent a lower bound to the reliability of teledermatology for the triage of inpatient dermatologic consultations. In addition, with less acutely ill patients, it may be possible to allocate even more patients to be seen the next day or later or refer them to outpatient care, further increasing efficiency.
Future studies will need to explore what improvements can be made to this teledermatology system to increase accuracy and reliability. In other areas of dermatology, increased training and experience have been shown to improve diagnostic accuracy.28-31 We suspect that with increased familiarity, concordance and reliability will improve similarly when inpatient consultative teledermatology is studied further. It will also be important to examine the impact of increased contact with the primary team, including potential for follow-up. Making diagnoses with images alone is challenging, and additional multimodal communication is likely important.28 Increased follow-up with the primary team may also increase the number of cases that could be managed with teledermatology alone, which could heighten efficiency further. In addition, standardized care algorithms and checklists have been shown to improve quality and consistency of care in a variety of health care settings.32-35 Creating standardized triage and management approaches for common diagnoses associated with consultations, such as drug reactions, and for patients with known dermatologic diagnoses from medical history will likely decrease practice variation and improve the accuracy and reliability of teledermatology for the initial triage of inpatient consultations. Finally, future studies will need to assess whether these results can be generalized to a community hospital setting.
Our study suggests that teledermatology is reliable for the initial triage of inpatient dermatologic consultations at an academic medical center and that it can potentially increase efficiency. We anticipate that future studies that refine the model presented here may find stronger concordance and efficiency gains.
Accepted for Publication: October 28, 2013.
Corresponding Author: Misha Rosenbach, MD, Department of Dermatology, Perelman School of Medicine at the University of Pennsylvania, 3600 Spruce St, Maloney Bldg, Second Floor, Philadelphia, PA 19104 (misha.rosenbach@uphs.upenn.edu).
Published Online: February 12, 2014. doi:10.1001/jamadermatol.2013.9517.
Author Contributions: Dr Rosenbach and Mr Barbieri had full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Study concept and design: Barbieri, James, Margolis, Kovarik, Rosenbach.
Acquisition of data: Barbieri, Nelson, James, Littman-Quinn, Kovarik, Rosenbach.
Analysis and interpretation of data: Barbieri, James, Margolis, Kovarik, Rosenbach.
Drafting of the manuscript: Barbieri, Kovarik, Rosenbach.
Critical revision of the manuscript for important intellectual content: All authors.
Statistical analysis: Barbieri, Margolis.
Administrative, technical, or material support: Barbieri, Nelson, Littman-Quinn, Kovarik, Rosenbach.
Study supervision: James, Kovarik, Rosenbach.
Conflict of Interest Disclosures: Mr Littman-Quinn reported receiving 25% of his salary from Vignet for consultation on unrelated projects.
1.Armstrong
AW, Wu
J, Kovarik
CL,
et al. State of teledermatology programs in the United States.
J Am Acad Dermatol. 2012;67(5):939-944.
PubMedGoogle ScholarCrossref 2.Warshaw
EM, Hillman
YJ, Greer
NL,
et al. Teledermatology for diagnosis and management of skin conditions: a systematic review.
J Am Acad Dermatol. 2011;64(4):759-772.
PubMedGoogle ScholarCrossref 3.Pak
H, Triplett
CA, Lindquist
JH, Grambow
SC, Whited
JD. Store-and-forward teledermatology results in similar clinical outcomes to conventional clinic-based care.
J Telemed Telecare. 2007;13(1):26-30.
PubMedGoogle ScholarCrossref 4.Kroemer
S, Frühauf
J, Campbell
TM,
et al. Mobile teledermatology for skin tumour screening: diagnostic accuracy of clinical and dermoscopic image tele-evaluation using cellular phones.
Br J Dermatol. 2011;164(5):973-979.
PubMedGoogle ScholarCrossref 5.Shapiro
M, James
WD, Kessler
R,
et al. Comparison of skin biopsy triage decisions in 49 patients with pigmented lesions and skin neoplasms: store-and-forward teledermatology vs face-to-face dermatology.
Arch Dermatol. 2004;140(5):525-528.
PubMedGoogle ScholarCrossref 6.Kvedar
JC, Edwards
RA, Menn
ER,
et al. The substitution of digital images for dermatologic physical examination.
Arch Dermatol. 1997;133(2):161-167.
PubMedGoogle ScholarCrossref 7.Lewis
K, Gilmour
E, Harrison
PV,
et al. Digital teledermatology for skin tumours: a preliminary assessment using a receiver operating characteristics (ROC) analysis.
J Telemed Telecare. 1999;5(suppl 1):S57-S58.
PubMedGoogle ScholarCrossref 8.Whited
JD, Warshaw
EM, Edison
KE,
et al. Effect of store and forward teledermatology on quality of life: a randomized controlled trial.
JAMA Dermatol. 2013;149(5):584-591.
PubMedGoogle ScholarCrossref 9.Armstrong
AW, Dorer
DJ, Lugn
NE, Kvedar
JC. Economic evaluation of interactive teledermatology compared with conventional care.
Telemed J E Health. 2007;13(2):91-99.
PubMedGoogle ScholarCrossref 10.Eminović
N, Dijkgraaf
MG, Berghout
RM, Prins
AH, Bindels
PJ, de Keizer
NF. A cost minimisation analysis in teledermatology: model-based approach.
BMC Health Serv Res. 2010;10:251. doi:10.1186/1472-6963-10-251.
PubMedGoogle ScholarCrossref 11.Parsi
K, Chambers
CJ, Armstrong
AW. Cost-effectiveness analysis of a patient-centered care model for management of psoriasis.
J Am Acad Dermatol. 2012;66(4):563-570.
PubMedGoogle ScholarCrossref 12.Moreno-Ramirez
D, Ferrandiz
L, Ruiz-de-Casas
A,
et al. Economic evaluation of a store-and-forward teledermatology system for skin cancer patients.
J Telemed Telecare. 2009;15(1):40-45.
PubMedGoogle ScholarCrossref 13.Lamel
SA, Haldeman
KM, Ely
H, Kovarik
CL, Pak
H, Armstrong
AW. Application of mobile teledermatology for skin cancer screening.
J Am Acad Dermatol. 2012;67(4):576-581.
PubMedGoogle ScholarCrossref 14.Massone
C, Hofmann-Wellenhof
R, Ahlgrimm-Siess
V, Gabler
G, Ebner
C, Soyer
HP. Melanoma screening with cellular phones.
PLoS One. 2007;2(5):e483.
PubMedGoogle ScholarCrossref 15.Frühauf
J, Schwantzer
G, Ambros-Rudolph
CM,
et al. Pilot study on the acceptance of mobile teledermatology for the home monitoring of high-need patients with psoriasis.
Australas J Dermatol. 2012;53(1):41-46.
PubMedGoogle ScholarCrossref 16.Börve
A, Holst
A, Gente-Lidholm
A, Molina-Martinez
R, Paoli
J. Use of the mobile phone multimedia messaging service for teledermatology.
J Telemed Telecare. 2012;18(5):292-296.
PubMedGoogle ScholarCrossref 17.Tran
K, Ayad
M, Weinberg
J,
et al. Mobile teledermatology in the developing world: implications of a feasibility study on 30 Egyptian patients with common skin diseases.
J Am Acad Dermatol. 2011;64(2):302-309.
PubMedGoogle ScholarCrossref 18.Viola
KV, Tolpinrud
WL, Gross
CP, Kirsner
RS, Imaeda
S, Federman
DG. Outcomes of referral to dermatology for suspicious lesions: implications for teledermatology.
Arch Dermatol. 2011;147(5):556-560.
PubMedGoogle ScholarCrossref 19.Warshaw
EM, Lederle
FA, Grill
JP,
et al. Accuracy of teledermatology for pigmented neoplasms.
J Am Acad Dermatol. 2009;61(5):753-765.
PubMedGoogle ScholarCrossref 20.Muir
J, Xu
C, Paul
S,
et al. Incorporating teledermatology into emergency medicine.
Emerg Med Australas. 2011;23(5):562-568.
PubMedGoogle ScholarCrossref 22.Healey
JF, Prus
SG. Statistics: A Tool for Social Research.7th ed. Stamford, CT: Cengage Learning; 2004.
24.Whited
JD, Hall
RP, Simel
DL,
et al. Reliability and accuracy of dermatologists’ clinic-based and digital image consultations.
J Am Acad Dermatol. 1999;41(5, pt 1):693-702.
PubMedGoogle ScholarCrossref 25.Pak
HS, Harden
D, Cruess
D, Welch
ML, Poropatich
R; National Capital Area Teledermatology Consortium. Teledermatology: an intraobserver diagnostic correlation study, part II.
Cutis. 2003;71(6):476-480.
PubMedGoogle Scholar 26.Börve
A, Terstappen
K, Sandberg
C, Paoli
J. Mobile teledermoscopy—there’s an app for that!
Dermatol Pract Concept. 2013;3(2):41-48.
PubMedGoogle ScholarCrossref 27.Mancusi
S, Festa Neto
C. Inpatient dermatological consultations in a university hospital.
Clinics (Sao Paulo). 2010;65(9):851-855.
PubMedGoogle ScholarCrossref 28.Tan
E, Levell
NJ. Regular clinical dermatoscope use with training improves melanoma diagnosis by dermatologists.
Clin Exp Dermatol. 2009;34(8):e876-e878.
PubMedGoogle ScholarCrossref 30.Goulart
JM, Quigley
EA, Dusza
S,
et al; INFORMED (INternet curriculum FOR Melanoma Early Detection) Group. Skin cancer education for primary care physicians: a systematic review of published evaluated interventions.
J Gen Intern Med. 2011;26(9):1027-1035.
PubMedGoogle ScholarCrossref 31.Pearl
RA, Townley
W, Stott
D, Grobbelaar
AO. Diagnosis of skin lesions by trainee surgeons: experience improves accuracy.
Ann R Coll Surg Engl. 2009;91(6):494-499.
PubMedGoogle ScholarCrossref 32.Pronovost
P, Needham
D, Berenholtz
S,
et al. An intervention to decrease catheter-related bloodstream infections in the ICU.
N Engl J Med. 2006;355(26):2725-2732.
PubMedGoogle ScholarCrossref 33.Gawande
A. The Checklist Manifesto: How to Get Things Right. New York, NY: Metropolitan Books; 2009.
34.Arriaga
AF, Bader
AM, Wong
JM,
et al. Simulation-based trial of surgical-crisis checklists.
N Engl J Med. 2013;368(3):246-253.
PubMedGoogle ScholarCrossref