In the science fiction film “Prometheus,” set in the year 2093, surgery is depicted as a purely automated process. A woman climbs into a capsule-shaped machine; with the touch of a few buttons, the machine sets to work preparing a sterile field, making an incision, extracting a specimen, and stapling her back up. No surgeon is required.
In today’s world, we still very much rely on surgeons. We expect them to be competent and to deliver the best outcomes possible. But surgeons aren’t machines; no two are exactly alike in experience or skill level. Does this variation matter?
Birkmeyer et al. sought to answer this question by investigating the relationship between surgical skills and operative outcomes in bariatric surgery. They used a double-blinded peer review process to assess the surgical skill level of 20 bariatric surgeons; they then correlated skill level with risk-adjusted complication rates. The surgeons, all part of the Michigan Bariatric Surgery Collaborative, submitted videotapes of themselves performing laparoscopic gastric bypass procedures. Their peers then reviewed the anonymous videos and rated them on various aspects of technical skill, including tissue exposure, instrument handling, and flow of operation. Each skill was rated on a scale of 1 to 5 (1 being the skill level of a general surgery chief resident, 5 being the skill level of a master bariatric surgeon). The primary outcome was the occurrence of any post-operative complication.
The study found that technical skill varied substantially across the cohort of surgeons, with mean ratings ranging from 2.6 to 4.8. These differences in proficiency translated to differences in surgical outcome. As compared to surgeons in the top quartile of skills, those in the bottom quartile carried significantly higher complication rates (14.5% vs. 5.2%; P<0.001). Patients of bottom-ranked surgeons were more likely to suffer from surgical site infections (4.6% vs. 1.0%; P=0.001) and pulmonary complications (3.9% vs. 0.7%; P=0.004). They were more likely to require reoperation (3.4% vs. 1.6%; P=0.01) and readmission within 30 days (6.3% vs. 2.7%; P<0.001). They were also more likely to die from surgery (0.26% vs. 0.05%; P=0.01).
These results, while not entirely surprising, are still unsettling. What explains the large difference in skill level across surgeons? One potential explanation is the difference in procedure volume. As compared to surgeons in the top quartile of skills, those in the bottom quartile had much lower annual volumes of laparoscopic gastric bypass procedures (53 vs. 157; P=0.005) and bariatric procedures overall (106 vs. 241; P=0.02). In contrast, skill level did not correlate significantly with years in practice or completion of a bariatric surgery fellowship.
NEJM Deputy Editor Mary Beth Hamel, M.D., M.P.H., states: “This preliminary but provocative study suggests that videos can be used to assess aspects of surgeons’ technical skills that are associated with patient outcomes.”
In an accompanying editorial, Danny Jacobs, M.D., M.P.H., a surgeon and the Dean of the School of Medicine at the University of Texas Medical Branch at Galveston, underscores the preliminary nature of these findings, stating: “Like many ‘index’ studies, this one raises more questions than it provides answers. There may be concerns about risk adjustment and the authors’ contention that there was no consistent relationship between rankings of surgical skill and patients’ coexisting conditions. If technical skill is such an important determinant, why were no significant differences in the rate of leak, perforation, or hemorrhage observed among the different quartiles of skill?”
To be sure, the study invites further investigation of the relationship between surgeon proficiency and operative outcomes. It proposes peer review as a potentially viable method of rating skill level, and it suggests a need for ongoing assessment of surgeons’ abilities even at the attending physician level. In affirming the relationship between operative volume and skill level, it also raises the question of whether surgeons who perform complex procedures should be expected to reach a minimum threshold of annual procedure volume.
While automated surgery may be far from reality, efforts to standardize operative performance may improve patient outcomes across surgeons. If we can find better ways to assess surgical performance, identify top performers, understand why they achieve better outcomes, and train less proficient surgeons to adopt those behaviors, we will be moving surgery toward a brighter, safer future.