MedTrack: Surgical Tool Tracking System for Robotic - Assisted Surgery

The Johnson & Johnson US team in ME 310 created MedTrack, the User-Centric Smart System, that serves registered nurses, scrub technicians, physician assistants, and surgeons to aid in operating room communication and manual tool tracking during robotic - assisted surgery. Through use of Graphical User Interfaces and Computer Vision, tool exchanges are automatically tracked and displayed to all members of the surgical team. The team conducted extensive need finding to learn about the different users in robotic surgery. This report provides the context and motivation for the project and documents the development of the design solution as well as the team's prototyping efforts throughout Winter quarter. This is the work of a team of students from both Stanford University and Norwegian University of Science and Technology who were sponsored by Johnson & Johnson MedTech.

There is a growing trend towards minimally invasive and robotic-assisted surgery (RAS). The main benefit of minimally invasive surgery (MIS) is the decreased recovery time for patients due to the smaller incision sizes. There is also greater precision, speed, and control with robotic techniques. While the technology and capabilities of robotic surgery are growing rapidly, these surgeries include a larger team of nurses, scrub techs, and physician assistants whose needs may be overlooked. Support for the communication needs of the operating room staff has not kept up pace with the improving robotic technology. This context provides the basis for a project prompt from Johnson and Johnson MedTech that was given to a team of graduate students from Stanford and Norwegian University of Science and Technology (NTNU): Identify alternatives to expensive surgical robotic interfaces without sacrificing safety and ergonomics, while providing high-quality feedback. The team conducted extensive interviews and operating room observations; from these they identified some particular problems to address. For example, teams today spend a lot of time counting and recounting surgical equipment that goes in and out of a patient and carefully documenting the logistics of the case. These processes can be tedious and inefficient. Moreover, despite these procedures there are still frequent count discrepancies. Approximately 1 in 8 surgeries involves a count discrepancy, which is defined as inconsistent counts of surgical materials before and after a case.

Teamwork is important in the OR, and there is a lot of communication between nurses and technicians regarding equipment and procedures, but unfortunately this is complicated by the sterile and non-sterile zones, and people are confined to either one or the other. Medical professionals in the OR are highly trained and follow strict means of communicating surgical information, especially in regards to the movement of instruments across the OR. The main means of having visual information in the OR is a non-sterile dry erase board, where counts of materials in and out of the body are recorded by the circulating nurse. Each time an instrument is placed in and out of the body, the circulating nurse must abandon their tasks and work station to travel across the room to mark the whiteboard, often having to carefully avoid sterile equipment in this cramped environment. This is a physical barrier to recording this information. Furthermore, personnel such as circulating nurses and scrub techs are shift-based employees with scheduled breaks, and shift changes often happening during cases. Changes in personnel can cause information to get lost between staff.

We propose a new system that reduces the time and effort spent in keeping track of equipment counts and also provides a communications interface for the different members of the team. Our system is made of three parts: the first is the MedTrack Dashboard, which is a ynamically updating user interface system to enable nurses to both display and record pertinent information relating to the phase of the case being completed. The second part is the MedTrack Cart, which uses a combination of computer vision and weight detection to determine when needles and sponges have left and returned to the mayo stand, where tools are organized in preparation for use. The third part of our system is a customized count of needles and sponges in the patient, allowing the surgeon to be privy to this information without averting their gaze. The three parts of our system, tailored to the specific needs of each OR personnel, bring documentation of surgical cases into the future. With MedTrack, we hope teams can track with confidence and collaborate with assurance. Through the completion of this project, a main takeaway we learned is that for this solution to be used in an OR it would initially need to be run in parallel with the current manual solution. This would allow surgical staff to get used to the system and build trust and confidence in MedTrack. There must also be a fail safe solution with the ability for people to take over or rectify any error, thereby ensuring a conservative design. The User Interface (UI) will ideally be customized for each hospital or nurse in order for MedTrack to be more widespread and fit individual user needs. In the future, MedTrack can be scoped beyond RAS and can include laparoscopic and traditional surgery. MedTrack could also be converted to be compatible with tablets or different screens to enable touchscreens.