While most algorithms work almost perfectly on their own, Peter Grantcharov explains that the black box OR is still not fully autonomous. For example, it is difficult to record audio through overhead microphones and thus obtain a reliable transcript to document whether each item on the surgical safety checklist has been completed. estimates that this algorithm has an error rate of 15%. Thus, before finalizing the result from each procedure, one of the Toronto analysts manually verifies compliance with the questionnaire. “It will require a human in the loop,” says Peter Grantcharov, but he estimates that the AI model has made the process of confirming compliance with the checklist 80% to 90% more efficient. He also emphasizes that the models are constantly being improved.
In total, an OR black box installation can cost about $100,000, and analysis costs run to $25,000 per year, according to Janet Donovan, an OR nurse who shared with MIT Technology Review an appreciation given to the staff of Brigham and Women’s Faulkner Hospital in Massachusetts. (Peter Grantcharov declined to comment on those numbers, writing in an email: “We don’t share specific rates; however, we can say it’s based on product mix and total room count, with a volume discount inherent in our pricing models.”)
“Big Brother is Watching”
Long Island Jewish Medical Center in New York, part of the Northwell Health System, was the first hospital for pilot OR black boxes, in February 2019. The mood was far from seamless, though not necessarily because of the technology.
“In the colon room, the cameras were tampered with,” recalls Northwell urologist Louis Kavoussi—they were turned around and deliberately unplugged. In his own OR, the staff kept quiet while he worked, worried they would say the wrong thing. “Unless you’re taking a golf or tennis lesson, you don’t want someone looking over there and watching what you’re doing,” says Cavucci, who has since joined the Scientific Advisory Board for Surgical Safety Technologies.
Grantcharov’s promises not to use the system to punish individuals have provided little comfort to some OR staff. When two black boxes were placed at Faulkner Hospital in November 2023, they threw the surgery department into crisis. “Everyone was very horrified by it,” says one surgeon who asked not to be named because he was not authorized to speak publicly. “We were being watched, and we felt like if we did something wrong, our jobs would be on the line.”
It wasn’t that he was doing anything illegal or spewing hate speech. she just wanted to joke with her friends, complain about the boss, and be herself without fear of administrators peeking over her shoulder. “You know very well that you are being watched. it’s not subtle at all,” he says. The early days were particularly difficult, with surgeons refusing to work in rooms equipped with black boxes and OR staff boycotting these operations: “It was definitely a struggle every morning.”
“In the rectal room, the cameras were undermined,” recalls Louis Cavousi. “Unless you’re taking a golf or tennis lesson, you don’t want someone looking over there and watching what you’re doing.”
At some level, identity protection is only half measures. Before 30-day-old records are automatically deleted, Grantcharov acknowledges that hospital administrators can still see the patient’s OR number, time of surgery and medical record number, so even if the OR staff is technically dismissed, they don’t he is truly anonymous. The result is a feeling that “Big Brother is watching,” says Christopher Mantyh, vice president of clinical operations at Duke University Hospital, which has black boxes in seven ORs. He will rely on aggregated data to talk generally about quality improvement at department meetings, but when specific issues arise, such as breaks in sterility or a cluster of infections, he will look at recordings and “go straight to the surgeons.”
In many ways, that’s what worries Faulkner Hospital nurse Donovan. She’s not convinced the hospital will protect staff members’ identities, and worries those recordings will be used against them — either through internal disciplinary action or in a patient malpractice lawsuit. In February 2023, she and nearly 60 others sent a letter to the hospital’s chief of surgery objecting to the black box. He has since filed a complaint with the state, with an arbitration proceeding scheduled for October.