Robotic Inguinal vs Transabdominal Laparoscopic Inguinal Hernia Repair: The RIVAL Randomized Clinical Trial | Minimally Invasive Surgery | JAMA Surgery | JAMA Network
[Skip to Navigation]
Figure 1.  Patient Flow Diagram
Patient Flow Diagram
Figure 2.  Postoperative Pain Visual Analog Scale (VAS)
Postoperative Pain Visual Analog Scale (VAS)

On the VAS, a score of 0 indicates no pain and 100 indicates the worst possible pain. The horizontal line in the middle of each box indicates the median, while the top and bottom borders of the box mark the 75th and 25th percentiles, respectively. The horizontal lines at the top and bottom of each vertical line represent the 95% CIs.

Table 1.  Mobility Measured by the Physical Activity Assessment Toola,b
Mobility Measured by the Physical Activity Assessment Toola,b
Table 2.  Operating Surgeon NASA Task Load Index Scalea
Operating Surgeon NASA Task Load Index Scalea
Table 3.  Operating Surgeon Rapid Upper Limb Assessmenta
Operating Surgeon Rapid Upper Limb Assessmenta
1.
Ger  R. Laparoscopic hernia operation. Article in German.  Chirurg. 1991;62(4):266-270.PubMed
2.
Neumayer  L, Giobbie-Hurder  A, Jonasson  O,  et al; Veterans Affairs Cooperative Studies Program 456 Investigators.  Open mesh versus laparoscopic mesh repair of inguinal hernia.   N Engl J Med. 2004;350(18):1819-1827. doi:10.1056/NEJMoa040093 PubMedGoogle ScholarCrossref
3.
HerniaSurge Group.  International guidelines for groin hernia management.   Hernia. 2018;22(1):1-165. doi:10.1007/s10029-017-1668-x PubMedGoogle ScholarCrossref
4.
Finley  DS, Rodriguez  E  Jr, Ahlering  TE.  Combined inguinal hernia repair with prosthetic mesh during transperitoneal robot assisted laparoscopic radical prostatectomy: a 4-year experience.   J Urol. 2007;178(4 Pt 1):1296-1299. doi:10.1016/j.juro.2007.05.154PubMedGoogle ScholarCrossref
5.
Escobar Dominguez  JE, Gonzalez  A, Donkor  C.  Robotic inguinal hernia repair.   J Surg Oncol. 2015;112(3):310-314. doi:10.1002/jso.23905 PubMedGoogle ScholarCrossref
6.
Miserez  M, Alexandre  JH, Campanelli  G,  et al.  The European hernia society groin hernia classification: simple and easy to remember.   Hernia. 2007;11(2):113-116. doi:10.1007/s10029-007-0198-3 PubMedGoogle ScholarCrossref
7.
Huskisson  EC.  Measurement of pain.   Lancet. 1974;2(7889):1127-1131. doi:10.1016/S0140-6736(74)90884-8 PubMedGoogle ScholarCrossref
8.
Ware  JE  Jr, Sherbourne  CD.  The MOS 36-item short-form health survey (SF-36). I. Conceptual framework and item selection.   Med Care. 1992;30(6):473-483. doi:10.1097/00005650-199206000-00002 PubMedGoogle ScholarCrossref
9.
Meriwether  RA, McMahon  PM, Islam  N, Steinmann  WC.  Physical activity assessment: validation of a clinical assessment tool.   Am J Prev Med. 2006;31(6):484-491. doi:10.1016/j.amepre.2006.08.021 PubMedGoogle ScholarCrossref
10.
Ware  JE, Snow  KK, Kosinski  M, Gandek  B.  SF-36 Health Survey: Manual and Interpretation Guide. New England Medical Center; 1993.
11.
Hart  SG, Staveland  LE.  Development of NASA-TLX: results of empirical and theoretical research.   Adv Psychol. 1988;52:139-183. doi:10.1016/S0166-4115(08)62386-9 Google ScholarCrossref
12.
Yurko  YY, Scerbo  MW, Prabhu  AS, Acker  CE, Stefanidis  D.  Higher mental workload is associated with poorer laparoscopic performance as measured by the NASA-TLX tool.   Simul Healthc. 2010;5(5):267-271. doi:10.1097/SIH.0b013e3181e3f329 PubMedGoogle ScholarCrossref
13.
Prabhu  A, Smith  W, Yurko  Y, Acker  C, Stefanidis  D.  Increased stress levels may explain the incomplete transfer of simulator-acquired skill to the operating room.   Surgery. 2010;147(5):640-645. doi:10.1016/j.surg.2010.01.007 PubMedGoogle ScholarCrossref
14.
McAtamney  L, Nigel Corlett  E.  RULA: a survey method for the investigation of work-related upper limb disorders.   Appl Ergon. 1993;24(2):91-99. doi:10.1016/0003-6870(93)90080-S PubMedGoogle ScholarCrossref
15.
Craven  R, Franasiak  J, Mosaly  P, Gehrig  PA.  Ergonomic deficits in robotic gynecologic oncology surgery: a need for intervention.   J Minim Invasive Gynecol. 2013;20(5):648-655. doi:10.1016/j.jmig.2013.04.008 PubMedGoogle ScholarCrossref
16.
Lawson  EH, Curet  MJ, Sanchez  BR, Schuster  R, Berguer  R.  Postural ergonomics during robotic and laparoscopic gastric bypass surgery: a pilot project.   J Robot Surg. 2007;1(1):61-67. doi:10.1007/s11701-007-0016-z PubMedGoogle ScholarCrossref
17.
Wilson  CB.  Adoption of new surgical technology.   BMJ. 2006;332(7533):112-114. doi:10.1136/bmj.332.7533.112 PubMedGoogle ScholarCrossref
18.
Vossler  JD, Pavlosky  KK, Murayama  SM, Moucharite  MA, Murayama  KM, Mikami  DJ.  Predictors of Robotic Versus Laparoscopic Inguinal Hernia Repair.   J Surg Res. 2019;241(808):247-253. doi:10.1016/j.jss.2019.03.056 PubMedGoogle ScholarCrossref
19.
Iraniha  A, Peloquin  J.  Long-term quality of life and outcomes following robotic assisted TAPP inguinal hernia repair.   J Robot Surg. 2018;12(2):261-269. doi:10.1007/s11701-017-0727-8 PubMedGoogle ScholarCrossref
20.
Waite  KE, Herman  MA, Doyle  PJ.  Comparison of robotic versus laparoscopic transabdominal preperitoneal (TAPP) inguinal hernia repair.   J Robot Surg. 2016;10(3):239-244. doi:10.1007/s11701-016-0580-1 PubMedGoogle ScholarCrossref
21.
Gamagami  R, Dickens  E, Gonzalez  A,  et al.  Open versus robotic-assisted transabdominal preperitoneal (R-TAPP) inguinal hernia repair: a multicenter matched analysis of clinical outcomes.   Hernia. 2018;22(5):827-836. doi:10.1007/s10029-018-1769-1 PubMedGoogle ScholarCrossref
22.
Pokala  B, Armijo  PR, Flores  L, Hennings  D, Oleynikov  D.  Minimally invasive inguinal hernia repair is superior to open: a national database review.   Hernia. 2019;23(3):593-599. doi:10.1007/s10029-019-01934-8 PubMedGoogle ScholarCrossref
23.
Zayan  NE, Meara  MP, Schwartz  JS, Narula  VK.  A direct comparison of robotic and laparoscopic hernia repair: patient-reported outcomes and cost analysis.   Hernia. 2019;23(6):1115-1121. doi:10.1007/s10029-019-01943-7 PubMedGoogle ScholarCrossref
24.
Abdelmoaty  WF, Dunst  CM, Neighorn  C, Swanstrom  LL, Hammill  CW.  Robotic-assisted versus laparoscopic unilateral inguinal hernia repair: a comprehensive cost analysis.   Surg Endosc. 2019;33(10):3436-3443. doi:10.1007/s00464-018-06606-9 PubMedGoogle ScholarCrossref
25.
Chen  YJ, Huynh  D, Nguyen  S, Chin  E, Divino  C, Zhang  L.  Outcomes of robot-assisted versus laparoscopic repair of small-sized ventral hernias.   Surg Endosc. 2017;31(3):1275-1279. doi:10.1007/s00464-016-5106-4 PubMedGoogle ScholarCrossref
26.
Allison  N, Tieu  K, Snyder  B, Pigazzi  A, Wilson  E.  Technical feasibility of robot-assisted ventral hernia repair.   World J Surg. 2012;36(2):447-452. doi:10.1007/s00268-011-1389-8 PubMedGoogle ScholarCrossref
27.
Lee  GI, Lee  MR, Clanton  T, Sutton  E, Park  AE, Marohn  MR.  Comparative assessment of physical and cognitive ergonomics associated with robotic and traditional laparoscopic surgeries.   Surg Endosc. 2014;28(2):456-465. doi:10.1007/s00464-013-3213-z PubMedGoogle ScholarCrossref
28.
Lux  MM, Marshall  M, Erturk  E, Joseph  JV.  Ergonomic evaluation and guidelines for use of the daVinci Robot system.   J Endourol. 2010;24(3):371-375. doi:10.1089/end.2009.0197 PubMedGoogle ScholarCrossref
29.
Lee  GI, Lee  MR, Green  I, Allaf  M, Marohn  MR.  Surgeons’ physical discomfort and symptoms during robotic surgery: a comprehensive ergonomic survey study.   Surg Endosc. 2017;31(4):1697-1706. doi:10.1007/s00464-016-5160-y PubMedGoogle ScholarCrossref
30.
Dwyer  A, Huckleby  J, Kabbani  M, Delano  A, De Sutter  M, Crawford  D.  Ergonomic assessment of robotic general surgeons: a pilot study.  [published online July 13, 2019].  J Robot Surg. 2019. doi:10.1007/s11701-019-00996-1 PubMedGoogle Scholar
31.
Donkor  C, Gonzalez  A, Gallas  MR, Helbig  M, Weinstein  C, Rodriguez  J.  Current perspectives in robotic hernia repair.   Robot Surg. 2017;4:57-67. doi:10.2147/RSRR.S101809 PubMedGoogle ScholarCrossref
Limit 200 characters
Limit 25 characters
Conflicts of Interest Disclosure

Identify all potential conflicts of interest that might be relevant to your comment.

Conflicts of interest comprise financial interests, activities, and relationships within the past 3 years including but not limited to employment, affiliation, grants or funding, consultancies, honoraria or payment, speaker's bureaus, stock ownership or options, expert testimony, royalties, donation of medical equipment, or patents planned, pending, or issued.

Err on the side of full disclosure.

If you have no conflicts of interest, check "No potential conflicts of interest" in the box below. The information will be posted with your response.

Not all submitted comments are published. Please see our commenting policy for details.

Limit 140 characters
Limit 3600 characters or approximately 600 words
    1 Comment for this article
    Call it what it is.
    Steven Reid, M.D., F.A.A.N.S. | Doctor Lifeline, Incorporated

    Too many institutions use the robot as a marketing tool. Rational indications for robotic surgery include stereotaxis and extending the working life of aging surgeons who cannot stand for prolonged periods, whose hands might shake and whose eyes may dim. Robotic surgery undoubtably will continue to evolve, eventually automating repetitive tasks such as wound closure, tumor debulking, and anastomoses. At this point in time, however, the robots provide little added value to most procedures, and as this study shows, add to expense, complexity, and frustration.
    CONFLICT OF INTEREST: None Reported
    Original Investigation
    March 18, 2020

    Robotic Inguinal vs Transabdominal Laparoscopic Inguinal Hernia Repair: The RIVAL Randomized Clinical Trial

    Author Affiliations
    • 1Cleveland Clinic Center for Abdominal Core Health, Digestive Diseases and Surgery Institute, Cleveland Clinic, The Cleveland Clinic Foundation, Cleveland, Ohio
    • 2Comprehensive Hernia Center, Department of Surgery, University of South Carolina School of Medicine Greenville, Greenville
    • 3Department of Surgery, New Hanover Regional Medical Center, University of North Carolina Wilmington, Wilmington
    • 4Department of Surgery, Medical College of Wisconsin, Milwaukee
    • 5Department of Surgery, Mount Sinai Hospital, New York, New York
    • 6Department of Surgery, Washington University in St Louis, St Louis, Missouri
    • 7Department of Surgery, University of North Carolina at Chapel Hill, Chapel Hill
    • 8Department of Surgery, University of Texas Medical Branch, Galveston
    JAMA Surg. 2020;155(5):380-387. doi:10.1001/jamasurg.2020.0034
    Key Points

    Question  Does the robotic approach to inguinal hernia repair result in improved postoperative outcomes compared with traditional laparoscopic inguinal hernia repairs?

    Findings  In this randomized clinical trial of 102 patients with inguinal hernia, no significant differences in operative outcomes at 30 days were found between patients who received robotic inguinal hernia repair and those who received laparoscopic inguinal hernia repair in terms of postoperative pain, health-related quality of life, mobility, wound morbidity, or cosmesis. The robotic approach resulted in increased operative time, cost, and surgeon frustration, without discernible ergonomic benefit for surgeons.

    Meaning  In this study, no apparent clinical benefit occurred with the robotic approach compared with the laparoscopic approach to straightforward unilateral inguinal hernia.

    Abstract

    Importance  Despite rapid adoption of the robotic platform for inguinal hernia repair in the US, to date, no level I trials have ever compared robotic inguinal hernia repair to laparoscopic repair. This multicenter randomized clinical trial is the first to compare the robotic platform to laparoscopic approach for minimally invasive inguinal hernia repair.

    Objective  To determine whether the robotic approach to inguinal hernia repair results in improved postoperative outcomes compared with traditional laparoscopic inguinal hernia repairs.

    Design, Setting, and Participants  This multicenter, single-blinded, prospective randomized clinical pilot study was conducted from April 2016 to April 2019, with a follow-up duration of 30 days in 6 academic and academic-affiliated sites. Enrolled in this study were 113 patients with a unilateral primary or recurrent inguinal hernia. After exclusions 102 remained for analysis.

    Interventions  Standard laparoscopic transabdominal preperitoneal repair or robotic transabdominal preperitoneal repair.

    Main Outcomes and Measures  Main outcomes included postoperative pain, health-related quality of life, mobility, wound morbidity, and cosmesis. Secondary outcomes included cost, surgeon ergonomics, and surgeon mental workload. A primary outcome was not selected because this study was designed as a pilot study. The hypothesis was formulated prior to data collection.

    Results  A total of 102 patients were included in the study (54 in the laparoscopic group, mean [SD] age, 57.2 [13.3] years and 48 [88.9%] male; 48 in the robotic group, mean [SD] age, 56.1 [14.1] years and 44 [91.6%] male). There were no differences at the preoperative, 1-week, or 30-day points between the groups in terms of wound events, readmissions, pain as measured by the Visual Analog Scale, or quality of life as measured by the 36-Item Short Form Health Survey. Compared with traditional laparoscopic inguinal hernia repair, robotic transabdominal preperitoneal repair was associated with longer median (interquartile range) operative times (75.5 [59.0-93.8] minutes vs 40.5 [29.2-63.8] minutes, respectively; P < .001), higher median (interquartile range) cost ($3258 [$2568-$4118] vs $1421 [$1196-$1930], respectively; P < .001), and higher mean (SD) frustration levels on the NASA Task Load Index Scale (range, 1-100, with lower scores indicating lower cognitive workload) (32.7 [23.5] vs 20.1 [19.2], respectively; P = .004). There were no differences in ergonomics of the surgeons between the groups as measured by the Rapid Upper Limb Assessment instrument.

    Conclusions and Relevance  Results of this study showed no clinical benefit to the robotic approach to straightforward inguinal hernia repair compared with the laparoscopic approach. The robotic approach incurred higher costs and more operative time compared with the laparoscopic approach, with added surgeon frustration and no ergonomic benefit to surgeons.

    Trial Registration  ClinicalTrials.gov Identifier: NCT02816658

    Introduction

    Minimally invasive inguinal hernia repair was first described by Ger1 in 1991 using a laparoscopic technique. A seminal article by Neumayer et al concluded that open inguinal hernia repair is superior to the laparoscopic approach,2 yet laparoscopic inguinal hernia repair has developed a reputation for shorter recovery time, equivalent recurrence rates, and lower incidence of short-term and long-term pain than open Lichtenstein repair in other modern series and current international guidelines.3 Additional costs associated with the laparoscopic technique are also thought to be offset by less-easily quantified benefits such as earlier return to work.3

    Robotic inguinal hernia repair, previously outlined in urologic literature4 but first described in general surgery literature by Dominguez et al in 2015,5 has emerged as an alternative approach to minimally invasive inguinal hernia repair. Purported benefits of robotic inguinal hernia surgery include decreased postoperative pain related to suturing mesh for fixation (as opposed to tack fixation in traditional laparoscopic repair), and improved surgeon ergonomics (body positioning and use of limbs during surgical procedures) during one of the most common procedures performed by general surgeons.

    Even with a seemingly rapid adoption of robotic technology for this technique in the United States, to date, no prospective head-to-head trials have been performed comparing laparoscopic and robotic transabdominal preperitoneal (TAPP) inguinal hernia repair to investigate relevant outcomes such as postoperative pain, recurrence, cost, and surgeon workload and economics. We hypothesized that the robotic approach to inguinal hernia repair would result in improved postoperative outcomes compared with traditional laparoscopic inguinal hernia repairs. Here, we present findings from a multicenter prospective randomized pilot study designed to address these questions.

    Methods

    Because data on postoperative pain, quality of life, surgeon ergonomics, and costs associated with robotic inguinal hernia were unavailable at the time that this trial was designed, a reasonable power calculation was not possible. Therefore, this study was designed as a patient-blinded prospective randomized clinical pilot study. The trial protocol is available in Supplement 1. After obtaining approval from all appropriate institutional review boards, between April 2016 and April 2019, a pilot study design enrolling 102 patients with an additional 10% enrollment for anticipated dropouts were randomized to either laparoscopic or robotic inguinal hernia repair (Figure 1). Patient consent was informed and written and obtained in person. Patients received no compensation. Six sites participated in enrollment, including Cleveland Clinic, Cleveland, Ohio; Mount Sinai Hospital, New York, New York; Washington University in St Louis, St Louis, Missouri; New Hanover Regional Medical Center, Wilmington, North Carolina; Greenville Health System, Greenville, South Carolina; and Medical College of Wisconsin, Milwaukee. All participating surgeons were required to have previously performed a minimum of 25 robotic and 25 laparoscopic procedures to be eligible to participate, and all operations were performed in their entirety by attending surgeons. All operations were performed using the transabdominal preperitoneal approach (TAPP). To allow for appropriate scheduling of robotic cases, patients were randomized to a treatment group by our study research coordinators (H.A., L.T., and A.F.) at the time of preoperative evaluation based on preoperative Visual Analog Scale (VAS) pain scores (score range, 0-100, with 0 indicating no pain and 100 indicating the worst possible pain). The randomization was performed using a random number of blocks with 1:1 ratio of assigning patients to each group. Patients were blinded to their interventions. Patients in each cohort were matched for age, sex, body mass index (BMI), and medical comorbidities to minimize confounding variables. A primary outcome was not selected because this study was designed as a pilot study. The hypothesis was formulated prior to data collection. Secondary outcomes include cost, surgeon ergonomics, and multidimensional workload.

    Baseline patient demographic characteristics were obtained at initial patient recruitment. All questionnaires were filled out following patient recruitment for baseline comparison. Operative details were collected from the medical record. Patient follow-up occurred at 7 days ±3 days and 1 month ±1 week. Future analysis will be reported at 1 year ±1 month, and 2 years ±2 months.

    Inclusion and Exclusion Criteria, Surgical Procedure, and Patient-Reported Outcomes

    Patients were included if they were aged 21 years or older with no prior open abdominal surgery, presenting for primary or recurrent unilateral inguinal hernia repair, no previous preperitoneal mesh placement, and BMI less than or equal to 40 (calculated as weight in kilograms divided by height in meters squared). Exclusion criteria were the need for an open inguinal hernia repair, patients presenting for evaluation of bilateral inguinal hernias, patients with previous open abdominal surgery at or below the umbilicus, previous preperitoneal mesh placement on the side of the planned inguinal hernia repair, strangulated inguinal hernia, patients with liver disease defined by the presence of ascites, patients with end-stage kidney disease requiring dialysis, and patients unable to give informed written consent.

    All enrolled patients underwent the TAPP approach to inguinal hernia repair. The operative technique was standardized in terms of the 3 ports used and dissection which has been described elsewhere.1 A flat mesh ranging from 10 to 15 cm in diameter of heavy-weight polypropylene (>75 g/m2) was used for all repairs. For the laparoscopic approach, mesh was fixated using a permanent tacking device, and peritoneum was closed using the same device. For the robotic approach, the mesh was fixated using permanent suture and the peritoneum was closed using running suture of the surgeon’s choice. The umbilical port was closed using standard practice for the operating surgeon. All surgeons were required, in their operative report, to detail the size of the mesh used, the location of placement and number of tacks used (if a laparoscopic repair), the type of suture used (if a robotic repair), and the type of inguinal hernia repair according to the European Hernia Society classification of inguinal hernias.6 Perioperative care was performed per standard institutional practice and was left to the discretion of the anesthesia and surgery teams at the participating location.

    Patients filled out pain and health-related quality of life measures at each clinic visit. Pain was evaluated using the VAS.7 Quality of life was measured using the validated 36-Item Short Form Health Survey (SF-36),8 and mobility was assessed using the Physical Activity Assessment Tool (minutes of moderate physical activity and vigorous physical activity were calculated by multiplying the number of minutes per day by the number of days per week; these were summed for moderate-to-vigorous physical activity.9 The SF-36 is an indicator of overall health status which uses 8 scaled scores relating to vitality, physical functioning, bodily pain, general health perceptions, physical role functioning, emotional role functioning, social role functioning, and mental health. Scores range from 0-100, where lower scores relate to more disability, and higher scores relate to less disability.10 The Physical Activity Assessment Tool measures type, frequency, and duration of moderate and vigorous physical activity from all 4 domains of physical activity (leisure, occupational, household, and transportation) in the last 7 days, and compares this with the usual level of activity.9 The Stony Brook Scar Evaluation Form for evaluation of cosmesis was filled out at 1 month postoperatively. This evaluation scale assigns individual scores to scars of 0 to 1 point each for the absence or presence, respectively, of 5 variables: width greater than 2 mm, elevation or depression, discoloration, suture or staple marks, and overall worse appearance. A total cosmetic score is then calculated by adding the individual scores, where 0 is worst and 5 is best.

    Surgeon Mental Workload and Ergonomic Measures

    Surgeon-reported outcomes were collected immediately following the conclusion of each case using the NASA Task Load Index Scale (NASA-TLX) (range, 1-100, with lower scores indicating lower cognitive workload) and Rapid Upper Limb Assessment (RULA) mental workload and ergonomic tools, respectively.

    The NASA-TLX validated instrument was used to determine the mental workload of surgeons, and has precedent as a tool to evaluate the mental workload of surgeons in minimally invasive surgery.11-13 The NASA-TLX provides an overall index of mental workload as well as the relative contribution of 6 subscales: mental, physical, and temporal task demands, effort, frustration, and perceived performance. Mental demand determines the level of intellectual and perceptual work required for completion of a task and physical demand determines the level of physical work required for completion of a task. Temporal demand is the measure for time pressure during the completion of the task. The effort component assesses mental and physical work required to perform at a certain proficiency level. The frustration component evaluates the level of stress associated with completion of the task. The start and end points for the scales used to quantify each of these 5 components are low and high, respectively. The sixth component, performance, is used to assess the degree of the surgeon’s satisfaction on completion of the task. The end points for the performance component are good and poor.11

    The RULA was used to assess the ergonomic status of surgeons involved in this trial. RULA is a survey method originally developed for use in workplaces where risk exists for work-related upper limb injuries, often related to poor ergonomics combined with repetitive motion. It provides an assessment of the postures of the neck, trunk, and upper limb along with muscle function and the external loads experienced by the body.14 RULA has also been used in the health care environment, as well as specifically with regard to laparoscopy and robotics.15,16 To use the instrument, the evaluators (independent research fellows who did not operate) subjectively score posture, muscle use, and force for one side of the body at a time. The scores are then added to obtain a grand score. A RULA grand score of 5 to 6 indicates increased risk for musculoskeletal injury, and a grand score of 7 indicates imminent risk of injury.

    Cost

    Costs per case at each institution were collected and reported as total cost, operating room cost (as calculated by cost per minute of operating room time required for case), and disposable/reusable cost, which was calculated to include disposable materials as well as reusable materials including the robotic instruments. Robotic and laparoscopic capital equipment costs were not amortized for the purpose of this analysis.

    Statistical Analysis

    Simple χ2 tests or Fisher exact tests were used for unadjusted analyses and a logistic regression model was considered in adjusting all the baseline covariates. All tests were 2-tailed and performed at a significance level of .05. As this was a randomized trial, differences in baseline demographic and clinical characteristics between the laparoscopic and robotic cohorts were expected to occur at random. Any significant differences found among the demographic or preoperative clinical characteristics between the 2 treatment groups were controlled for in the final analysis to limit potential confounding of results. All analyses were performed with R statistical software, version 3.5.0 (R Foundation for Statistical Computing).

    Results

    A total of 102 patients were enrolled in this trial; 54 were randomized to the laparoscopic group (mean [SD] age, 57.2 [13.3] years and 48 [88.9%] male), and 48 were randomized to the robotic group (mean [SD] age, 56.1 [14.1] years and 44 [91.6%] male). One patient in the robotic group was converted to a laparoscopic procedure due to bleeding and was analyzed based on intent to treat principles in the robotic group. While the robotic group had a lower BMI than the laparoscopic group (24.9 vs 26.9; P = .01), the groups were otherwise similar. Patient demographic characteristics are outlined in eTable 1 in Supplement 2. The types and location of hernias was similar between the groups; however, there was significant variation in the size of ports used between the groups: for the umbilical port, most ports in the laparoscopic group (74%) were 12 mm, whereas in the robotic group, 58% were 12 mm and 29.2% were 8 mm (P < .001). For the right lateral port, most patients in the laparoscopic group (92.5%) had a 5-mm port, whereas most patients (95.8%) had 8-mm ports (P < .001). The same was seen for the left lateral port (eTable 2 in Supplement 2). There was a significantly greater median (interquartile range [IQR]) time from skin incision to closure in the robotic group (75.5 [59.0-93.8] vs 40.5 [29.2-63.8] minutes; P < .001). Significantly greater median (IQR) time was required in the robotic group for dissection of the hernia (18.0 [12.0-27.0] vs 13.0 [7.0-23.0] minutes; P = .01), mesh fixation (6.88 [5.00-9.00] vs 1.00 minutes [1.00-3.00]; P < .001), and peritoneal closure (7.00 [5.00-9.00] vs 2.00 [1.00-3.00] minutes; P < .001). Time required for mesh placement was similar between the groups (3.00 [2.00-5.00] vs 3.00 [2.00-4.00] minutes; P = .28) (eTable 2 in Supplement 2). While minimally invasive inguinal hernia repair is expected to be an outpatient procedure, there was a similar incidence of unanticipated admission between both groups (2 of 54 [3.8%] vs 4 of 48 [8.3%]; P = .42) (eTable 3 in Supplement 2).

    In terms of patient-reported outcomes, pain reported using the VAS score (preoperatively, 18.8 vs 15.2; P = .42) and change from baseline was similar between the groups preoperatively 1 week (4.60 vs 5.53; P = .86), and 30 at 11 days postoperatively (−7.92 vs −7.00; P = .85) (Figure 2). Health-related quality of life, measured with the SF-36, was also similar between groups at all 3 points for all 10 categories. Preoperatively, there was no difference between groups in physical component summary (48.1 vs 50.4; P = .20), mental component summary (54.3 vs 54.8; P = .73), or general health (79.3 vs 80.7; P = .61). The same was true at 1 week for the physical component summary (−6.52 vs −6.95; P = .80), mental component summary (0.80 vs 0.00; P = .60), and general health (−1.98 vs −1.72; P = .91). No differences were appreciated at 30 days in either group for physical component summary (−0.59 vs −1.98; P = .43), mental component summary (0.65 vs 0.71; P = .97), or general health (−2.31 vs 1.55; P = .07) (eTable 4 in the Supplement). Mobility measured by the Physical Activity Assessment Tool was the same between both groups at all 3 time points. At baseline, both groups were the same for vigorous physical activity (VPA) + moderate physical activity (MPA) (498 vs 601; P = .54). The change in VPA + MPA from baseline was also similar at 1 week (−221.92 vs −450.07; P = .25) and 30 days (−129.62 vs −264.93; P = .46). (Table 1). Mean (SD) patient-reported cosmesis, measured using the Stony Brook Scar Evaluation Form, was also similar between the groups at 30 days (4.02 [1.03] vs 3.84 [1.19]; P = .43).

    Regarding surgeon mental workload, as measured by the NASA-TLX instrument, there was a significantly higher mean (SD) level of surgeon frustration for the robotic group vs the laparoscopic group (32.7 [23.5] vs 20.1 [19.2]; P = .004), and mean [SD] increased effort of surgeons in the robotic group vs the laparoscopic group (36.7 [23.3] vs 27.4 [24.1]; P = .05) (Table 2).

    Surgeon ergonomics were measured using the RULA instrument. Although a significantly worse mean (SD) score was noted in the robotic group compared with the laparoscopic group for left-side upper limbs, arms, and wrists (5.62 [1.10] vs 4.93 [1.59], respectively; P = .01), the mean (SD) Grand Composite Scores were similar between the groups (for left-side laparoscopic, 9.80 [2.75] vs left-side robotic, 10.3 [1.93]; P = .31; and for right-side laparoscopic, 10.1 [3.16] vs right-side robotic, 10.2 [2.27]; P = .94) (Table 3).

    Data for 30-day wound morbidity and overall adverse events were similar between the groups. For laparoscopic vs robotic inguinal hernia repair, no differences were seen in adverse events (9.3% vs 16.7%; P = .37), superficial surgical site infections (1.85% vs 0%; P > .99), purulent drainage from wound (1.85% vs 0%; P > .99), or urinary retention (1.85% vs 2.08%; P > .99) (eTable 5 in the Supplement).

    A significant increase was noted for the robotic group vs the laparoscopic group in all cost categories including median (IQR) total cost ($3258 [$2568-$4118] vs $1421 [$1196-$1930]; P < .001), operating room cost ($1401 [$1234-$1933] vs $879 [$714-$1052]; P < .001), and mean (SD) disposable/reusable cost ($1784 [$1149] vs $623 [$434]; P < .001) (eTable 6 in the Supplement).

    Discussion

    To our knowledge, this trial is the first to prospectively and comprehensively compare outcomes of robotic and laparoscopic TAPP approach to unilateral inguinal hernia repair. Between well-matched groups with similar inguinal hernia disease, there was no significant difference in operative outcomes between robotic and laparoscopic TAPP in terms of postoperative pain, health-related quality of life, mobility, wound morbidity, or cosmesis. Notably, robotic operations were found to have a significantly higher cost than that of the laparoscopic approach. The robotic group was also found to have a significantly longer operating time and demonstrated an increased level of surgeon frustration as well as a trend toward increased surgeon effort despite equal performance of the operation. There was no difference in surgeon ergonomics between the 2 groups, as measured and recorded by independent observers (H.A. and L.T.).

    New technologies may be rapidly adopted in the absence of supporting evidence to establish their superiority compared with existing procedures.17 Despite seemingly exponential increase in the robotic approach to inguinal hernia repair with concurrent static numbers of laparoscopic approach,18 current comparative literature to support this practice has remained sparse. Indeed, studies supportive of the robotic over laparoscopic approach to date are hindered by single-group cohorts limited to quality of life measures,19 single-surgeon experiences with small patient groups and clinical differences of limited practical or meaningful value,20 multicenter retrospective studies with heterogeneous comparison groups,21 and large administrative database reports lacking in operative granularity and limited in their scope to address salient issues related to hernia recurrence and quality of life.22

    Given that our study was performed by all fellowship-trained minimally invasive high-volume hernia surgeons with documented robotic experience and a high level of familiarity with the operative anatomy and steps of TAPP inguinal hernia repair, it could be expected that clinical outcomes were similar between the robotic and laparoscopic groups. Notably, the differences in mesh fixation between robotic suturing and permanent tack fixation were not associated with any difference in patient-reported outcomes such as pain or mobility. Additionally, our findings of increased operative times and cost for the robotic group have been well-described previously and are consistent with multiple prior publications.18,20,22-24 Having annotated the steps of the operation to evaluate the time for each step in a granular fashion, it is clear that the increased operative time for the robotic group was ascribed to each step of the operation, and docking (median 5 minutes) was therefore not the culprit for the 35-minute difference in time that we observed.

    While there is a common perception that the high-resolution 3-dimensional visualization, elimination of tremor, and stability of instruments offered by the robotic platform may ultimately be associated with improved surgeon comfort and performance,22,25,26 our findings do not support this idea for the application of robotics in minimally invasive inguinal hernia repair. While those benefits may exist, we found no association between their existence and any meaningful clinical improvement in the performance or outcomes of robotic TAPP inguinal hernia repair over laparoscopic TAPP. Furthermore, while some have suggested that there may be an ergonomic benefit to surgeons using the robotic platform,16,27 our findings also did not support this idea for the application of robotics in minimally invasive inguinal hernia repair. Others have suggested that the robotic console only allows acceptable positioning for surgeons falling between the heights of 64 in and 73 in28; however, all surgeons except 1 fit this description. Despite fairly wide dissemination and acceptance of the concept of ergonomic superiority associated with the robotic platform, this concept has not been thoroughly investigated to date. To that end, there are studies that do not support the purported ergonomic benefit added by the robot,29,30 and there is a lack of high-level evidence at this point to support a definitive conclusion regarding ergonomics as they relate to the robot. Regardless, RULA Grand Composite Scores for both groups in our study indicated imminent risk for work-related injury, suggesting that there is no objectively measured ergonomic benefit to the robotic platform over the laparoscopic approach. More likely, all surgical procedures may be associated with some form of poor ergonomics and potential for repetitive-motion type occupational injury.

    Surgeons in our study had a significantly higher level of frustration when performing robotic operations over that perceived with laparoscopic operations. As surgeons in this study had performed at least 25 robotic operations prior to the study, we do not feel that this finding can be attributed to the surgeon’s robotic learning curve. We also do not think it is entirely unexpected that surgeons experienced frustration with an operation that requires almost twice as much time to perform as its comparator. While some surgeons have noted an advantage to the robotic approach for more complex operations such as recurrent inguinal hernia,31 our study was designed to investigate the use of robotics in straightforward unilateral inguinal hernia. Further study is required to elucidate the potential advantages of robotics in more complicated clinical scenarios.

    Limitations

    This study has limitations. First, at the time that this study was initially designed, there was essentially no precedent on which to perform a power calculation as robotic adoption was in its infancy for repair of inguinal hernia. Thus, this study was designed as a pilot study. Even with the relatively small cohorts examined herein, we were able to identify statistically significant differences in cost, operating time, and surgeon frustration. While increased surgeon effort in the robotic group approached statistical significance, it is possible that a type II error prevented surgeon effort from reaching statistical significance. Also, as the learning curve for minimally invasive hernia is not well defined in current surgical literature, we are unable to state definitively if the 25-case requirement for each platform was sufficient to assume proficiency of participating surgeons on both platforms. Additionally, proponents of robotic inguinal surgery have inferred that the robotic platform may facilitate acquisition of minimally invasive skills for surgeons who otherwise are only able to perform open inguinal hernia repair.31 That question was not addressed here as it was outside of the scope of this study, and further prospective study should be undertaken to investigate this concept.

    Finally, based on our findings combined with a traditional value equation, in which value equals clinical and patient-reported outcomes divided by cost, we performed a power analysis for the next phase of our trial. We surmised that to offset the cost difference identified in this trial, the outcomes of the operation (numerator of the equation) would need to improve by an order of 2 for the robotic TAPP to provide true value. As clinical outcomes were unlikely to appreciably improve, we selected the VAS score on which to power our next phase. To detect a 50% improvement in VAS scores for the robotic group over the laparoscopic group, our power analysis determined that 23 patients per group would be required to achieve 80% power. Although our initial trial was not designed with this power analysis in mind, it appears that our study was in fact powered to detect a VAS difference of sufficient clinical importance to offset the cost differential associated with robotic TAPP.

    Conclusions

    Our study is to date the only prospective randomized head-to-head comparison of robotic vs laparoscopic minimally invasive inguinal hernia repair. In this study, early clinical outcomes were similar between the groups with regard to postoperative pain, quality of life, mobility, cosmesis, wound morbidity, and complications. The robotic approach was more costly, required greater operative time, resulted in increased surgeon frustration, and failed to provide any objective ergonomic benefit to surgeons compared with the laparoscopic platform. Despite the pilot nature of our study, our outcomes are notable, particularly given the frequency with which inguinal hernia repair is performed in the United States, and suggest that the use of the robotic platform for unilateral uncomplicated inguinal hernia repair is not justified for surgeons able to perform this operation laparoscopically. Our study was not designed to address potential benefits of the robotic platform for open surgeons attempting to adopt minimally invasive techniques, or in more complex clinical scenarios of inguinal hernia repair and further study should be devoted to these specific questions.

    Back to top
    Article Information

    Accepted for Publication: December 10, 2019.

    Corresponding Author: Ajita S. Prabhu, MD, Cleveland Clinic Center for Abdominal Core Health, Digestive Diseases and Surgery Institute, The Cleveland Clinic Foundation, 9500 Euclid Ave, A-100, Cleveland, OH 44195 (prabhua@ccf.org).

    Published Online: March 18, 2020. doi:10.1001/jamasurg.2020.0034

    Author Contributions: Dr Prabhu had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

    Concept and design: Hope, Blatnik, Haskins, Alkhatib, Tastaldi, Tu, Rosen.

    Acquisition, analysis, or interpretation of data: Prabhu, Carbonell, Warren, Higgins, Jacob, Haskins, Alkhatib, Tastaldi, Fafaj, Tu, Rosen.

    Drafting of the manuscript: Prabhu, Haskins, Tastaldi, Fafaj, Rosen.

    Critical revision of the manuscript for important intellectual content: Prabhu, Carbonell, Hope, Warren, Higgins, Jacob, Blatnik, Alkhatib, Tastaldi, Tu, Rosen.

    Statistical analysis: Haskins, Alkhatib, Fafaj, Tu.

    Obtained funding: Haskins, Tastaldi, Rosen.

    Administrative, technical, or material support: Carbonell, Warren, Blatnik, Alkhatib, Tastaldi, Rosen.

    Supervision: Prabhu, Hope, Higgins, Alkhatib, Rosen.

    Conflict of Interest Disclosures: Dr Prabhu reported receiving honoraria for speaking and a research grant paid to her institution from Intuitive Surgical, serving on an advisory board and receiving honoraria from Medtronic and from BD. Dr Carbonell reported receiving honoraria from Intuitive Surgical, Ethicon, and WL Gore & Associates, as well as a research grant from WL Gore & Associates paid to his institution. Dr Hope reported receiving honoraria from speaking from BD, WL Gore & Associates, Intuitive Surgical, honoraria for consulting from BD, Intuitive Surgical, Medtronic, research grant support paid to his institution from BD, and being on the surgeon advisory board for Mesh Suture and Deep Blue. Dr Warren reported receiving honoraria for teaching and speaking from Intuitive Surgical, honoraria for consulting from CMR surgical, and honoraria for speaking from Ethicon. Dr Higgins reported receiving honoraria for teaching from Intuitive Surgical, and honoraria for teaching and speaking from WL Gore & Associates. Dr Jacob reported receiving honoraria from Medtronic, grant support for conferences from Medtronic, having equity in IHC Inc, and receiving honoraria from BD. Dr Blatnik reported receiving honoraria from Intuitive Surgical and BD for consulting and teaching, and receiving research support paid to his institution from Ethicon and Cook. Dr Rosen reported receiving salary support for his role as Medical Director of the Americas Hernia Society Quality Collaborative and grant support paid to his institution from Pacira. No other disclosures were reported.

    Funding/Support: This study was funded by a grant from Intuitive Surgical (IUSI1602MR).

    Role of the Funder/Sponsor: The funding source had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

    Data Sharing Statement: See Supplement 3.

    References
    1.
    Ger  R. Laparoscopic hernia operation. Article in German.  Chirurg. 1991;62(4):266-270.PubMed
    2.
    Neumayer  L, Giobbie-Hurder  A, Jonasson  O,  et al; Veterans Affairs Cooperative Studies Program 456 Investigators.  Open mesh versus laparoscopic mesh repair of inguinal hernia.   N Engl J Med. 2004;350(18):1819-1827. doi:10.1056/NEJMoa040093 PubMedGoogle ScholarCrossref
    3.
    HerniaSurge Group.  International guidelines for groin hernia management.   Hernia. 2018;22(1):1-165. doi:10.1007/s10029-017-1668-x PubMedGoogle ScholarCrossref
    4.
    Finley  DS, Rodriguez  E  Jr, Ahlering  TE.  Combined inguinal hernia repair with prosthetic mesh during transperitoneal robot assisted laparoscopic radical prostatectomy: a 4-year experience.   J Urol. 2007;178(4 Pt 1):1296-1299. doi:10.1016/j.juro.2007.05.154PubMedGoogle ScholarCrossref
    5.
    Escobar Dominguez  JE, Gonzalez  A, Donkor  C.  Robotic inguinal hernia repair.   J Surg Oncol. 2015;112(3):310-314. doi:10.1002/jso.23905 PubMedGoogle ScholarCrossref
    6.
    Miserez  M, Alexandre  JH, Campanelli  G,  et al.  The European hernia society groin hernia classification: simple and easy to remember.   Hernia. 2007;11(2):113-116. doi:10.1007/s10029-007-0198-3 PubMedGoogle ScholarCrossref
    7.
    Huskisson  EC.  Measurement of pain.   Lancet. 1974;2(7889):1127-1131. doi:10.1016/S0140-6736(74)90884-8 PubMedGoogle ScholarCrossref
    8.
    Ware  JE  Jr, Sherbourne  CD.  The MOS 36-item short-form health survey (SF-36). I. Conceptual framework and item selection.   Med Care. 1992;30(6):473-483. doi:10.1097/00005650-199206000-00002 PubMedGoogle ScholarCrossref
    9.
    Meriwether  RA, McMahon  PM, Islam  N, Steinmann  WC.  Physical activity assessment: validation of a clinical assessment tool.   Am J Prev Med. 2006;31(6):484-491. doi:10.1016/j.amepre.2006.08.021 PubMedGoogle ScholarCrossref
    10.
    Ware  JE, Snow  KK, Kosinski  M, Gandek  B.  SF-36 Health Survey: Manual and Interpretation Guide. New England Medical Center; 1993.
    11.
    Hart  SG, Staveland  LE.  Development of NASA-TLX: results of empirical and theoretical research.   Adv Psychol. 1988;52:139-183. doi:10.1016/S0166-4115(08)62386-9 Google ScholarCrossref
    12.
    Yurko  YY, Scerbo  MW, Prabhu  AS, Acker  CE, Stefanidis  D.  Higher mental workload is associated with poorer laparoscopic performance as measured by the NASA-TLX tool.   Simul Healthc. 2010;5(5):267-271. doi:10.1097/SIH.0b013e3181e3f329 PubMedGoogle ScholarCrossref
    13.
    Prabhu  A, Smith  W, Yurko  Y, Acker  C, Stefanidis  D.  Increased stress levels may explain the incomplete transfer of simulator-acquired skill to the operating room.   Surgery. 2010;147(5):640-645. doi:10.1016/j.surg.2010.01.007 PubMedGoogle ScholarCrossref
    14.
    McAtamney  L, Nigel Corlett  E.  RULA: a survey method for the investigation of work-related upper limb disorders.   Appl Ergon. 1993;24(2):91-99. doi:10.1016/0003-6870(93)90080-S PubMedGoogle ScholarCrossref
    15.
    Craven  R, Franasiak  J, Mosaly  P, Gehrig  PA.  Ergonomic deficits in robotic gynecologic oncology surgery: a need for intervention.   J Minim Invasive Gynecol. 2013;20(5):648-655. doi:10.1016/j.jmig.2013.04.008 PubMedGoogle ScholarCrossref
    16.
    Lawson  EH, Curet  MJ, Sanchez  BR, Schuster  R, Berguer  R.  Postural ergonomics during robotic and laparoscopic gastric bypass surgery: a pilot project.   J Robot Surg. 2007;1(1):61-67. doi:10.1007/s11701-007-0016-z PubMedGoogle ScholarCrossref
    17.
    Wilson  CB.  Adoption of new surgical technology.   BMJ. 2006;332(7533):112-114. doi:10.1136/bmj.332.7533.112 PubMedGoogle ScholarCrossref
    18.
    Vossler  JD, Pavlosky  KK, Murayama  SM, Moucharite  MA, Murayama  KM, Mikami  DJ.  Predictors of Robotic Versus Laparoscopic Inguinal Hernia Repair.   J Surg Res. 2019;241(808):247-253. doi:10.1016/j.jss.2019.03.056 PubMedGoogle ScholarCrossref
    19.
    Iraniha  A, Peloquin  J.  Long-term quality of life and outcomes following robotic assisted TAPP inguinal hernia repair.   J Robot Surg. 2018;12(2):261-269. doi:10.1007/s11701-017-0727-8 PubMedGoogle ScholarCrossref
    20.
    Waite  KE, Herman  MA, Doyle  PJ.  Comparison of robotic versus laparoscopic transabdominal preperitoneal (TAPP) inguinal hernia repair.   J Robot Surg. 2016;10(3):239-244. doi:10.1007/s11701-016-0580-1 PubMedGoogle ScholarCrossref
    21.
    Gamagami  R, Dickens  E, Gonzalez  A,  et al.  Open versus robotic-assisted transabdominal preperitoneal (R-TAPP) inguinal hernia repair: a multicenter matched analysis of clinical outcomes.   Hernia. 2018;22(5):827-836. doi:10.1007/s10029-018-1769-1 PubMedGoogle ScholarCrossref
    22.
    Pokala  B, Armijo  PR, Flores  L, Hennings  D, Oleynikov  D.  Minimally invasive inguinal hernia repair is superior to open: a national database review.   Hernia. 2019;23(3):593-599. doi:10.1007/s10029-019-01934-8 PubMedGoogle ScholarCrossref
    23.
    Zayan  NE, Meara  MP, Schwartz  JS, Narula  VK.  A direct comparison of robotic and laparoscopic hernia repair: patient-reported outcomes and cost analysis.   Hernia. 2019;23(6):1115-1121. doi:10.1007/s10029-019-01943-7 PubMedGoogle ScholarCrossref
    24.
    Abdelmoaty  WF, Dunst  CM, Neighorn  C, Swanstrom  LL, Hammill  CW.  Robotic-assisted versus laparoscopic unilateral inguinal hernia repair: a comprehensive cost analysis.   Surg Endosc. 2019;33(10):3436-3443. doi:10.1007/s00464-018-06606-9 PubMedGoogle ScholarCrossref
    25.
    Chen  YJ, Huynh  D, Nguyen  S, Chin  E, Divino  C, Zhang  L.  Outcomes of robot-assisted versus laparoscopic repair of small-sized ventral hernias.   Surg Endosc. 2017;31(3):1275-1279. doi:10.1007/s00464-016-5106-4 PubMedGoogle ScholarCrossref
    26.
    Allison  N, Tieu  K, Snyder  B, Pigazzi  A, Wilson  E.  Technical feasibility of robot-assisted ventral hernia repair.   World J Surg. 2012;36(2):447-452. doi:10.1007/s00268-011-1389-8 PubMedGoogle ScholarCrossref
    27.
    Lee  GI, Lee  MR, Clanton  T, Sutton  E, Park  AE, Marohn  MR.  Comparative assessment of physical and cognitive ergonomics associated with robotic and traditional laparoscopic surgeries.   Surg Endosc. 2014;28(2):456-465. doi:10.1007/s00464-013-3213-z PubMedGoogle ScholarCrossref
    28.
    Lux  MM, Marshall  M, Erturk  E, Joseph  JV.  Ergonomic evaluation and guidelines for use of the daVinci Robot system.   J Endourol. 2010;24(3):371-375. doi:10.1089/end.2009.0197 PubMedGoogle ScholarCrossref
    29.
    Lee  GI, Lee  MR, Green  I, Allaf  M, Marohn  MR.  Surgeons’ physical discomfort and symptoms during robotic surgery: a comprehensive ergonomic survey study.   Surg Endosc. 2017;31(4):1697-1706. doi:10.1007/s00464-016-5160-y PubMedGoogle ScholarCrossref
    30.
    Dwyer  A, Huckleby  J, Kabbani  M, Delano  A, De Sutter  M, Crawford  D.  Ergonomic assessment of robotic general surgeons: a pilot study.  [published online July 13, 2019].  J Robot Surg. 2019. doi:10.1007/s11701-019-00996-1 PubMedGoogle Scholar
    31.
    Donkor  C, Gonzalez  A, Gallas  MR, Helbig  M, Weinstein  C, Rodriguez  J.  Current perspectives in robotic hernia repair.   Robot Surg. 2017;4:57-67. doi:10.2147/RSRR.S101809 PubMedGoogle ScholarCrossref
    ×