Modern medicine radically changed with the introduction of laparoscopic surgery several decades ago. Instead of cutting open a patient, which can lead to serious complications and long recovery times, surgeons could instead insert specialized instruments through small incisions, also known as surgical ports, to perform minimally invasive procedures.
Hongrui Jiang
While the technique continues to improve year after year, there is still one problematic element: the camera that actually guides the procedure. In most cases, a separate camera is inserted into the abdomen via a dedicated port and is controlled by another surgeon, resident or nurse. Yet the camera’s field of view is narrow and it’s prone to smudging, fogging and spattering, which means it needs to be removed for cleaning. It can also sometimes get in the way of surgical instruments.
That’s why Hongrui Jiang, a professor of electrical and computer engineering at the University of Wisconsin-Madison, is leading the development of a new visualization system called EasyVis that could significantly improve the efficiency of laparoscopic surgery.
Recently, the National Institutes of Health renewed funding for this work, giving Jiang and his team $1.6 million over the next four years to improve EasyVis, an integrated, panoramic, flexible, immersive, 3D laparoscopic visualization system with a field of view much larger than that of traditional laparoscopic cameras.
“Our idea is really a paradigm shift to using multiple cameras mounted on the surgical port,” says Jiang. “There are lots of potential benefits including 3D rendition and much more flexible visualization of the surgical field.”
The EasyVis system consists of multiple 5-millimeter-diameter microcameras with liquid miniature lenses, tiny LED light sources, and miniature optical devices used to create 3D images. It’s deployed into the abdominal cavity on a device called a trocar, inserted through the same port that the surgical instruments use. Once the trocar enters the abdomen, it unfolds tiny mechanical arms that hold the imaging devices. After surgery, the arms fold up and the trocar is removed.
Yu Hen Hu
The system can be completely controlled by the operating surgeon, who wears a pair of virtual reality or augmented reality goggles and can zoom in, pan out, change viewing angles and get close-up 3D images of specific areas via voice control, eliminating the need for a second surgeon.
At least, that’s Jiang’s vision for how EasyVis will work when the project is complete.
There are still many elements of the system that need research and improvement—but the team has already spent several years making significant progress. The researchers have produced a prototype trocar that was tested by more than 20 different surgeons and residents during a laparoscopic training exercise called the “bean-drop test.” The team found that the EasyVis system led to a significant improvement in the speed and accuracy of the surgeons’ ability to pick up and manipulate beans inside the testing apparatus.
“After the first period of this project, we have this very nice folding mechanical system which you can deploy through the surgical port and flare the cameras out once it’s in there. We also developed computer vision algorithms to stitch together video views as well,” says Jiang. “For this grant renewal, we’re going to improve upon the hardware and the software.”
The goal of the second phase is development of a new type of miniaturized optical device and improving the trocar-camera assembly so that it can hold multiple microcameras and light sources, enabling auto-focus, zoom and other capabilities.
Charles Heise
On the software side, the team will work on real-time scene stabilization algorithms that will keep the video feed in constant focus, as well as algorithms that enable 3D visualization. The group will also create the voice command system and integrate it into the EasyVis user interface.
By the end of this grant period, Jiang says he expects EasyVis will be operational and ready for University of Wisconsin-Madison surgeons to validate through animal testing.
After that, Jiang hopes the concept will be ready for its next step in the medical industry. “What we do at universities is lay out the foundation for future commercialization,” he says. “And if we’re successful, hopefully this system will be picked up by established companies or commercialized by us. But we need to do the basic research and we need to get the foundations right first.”
Other senior investigators on the project include Yu Hen Hu, a professor of electrical and computer engineering, as well as Charles Heise, an MD in the Department of Surgery at the UW-Madison School of Medicine and Public Health, and Jacob Greenberg, formerly an MD in the Department of Surgery.
Hongrui Jiang is the Lynn H. Matthias Professor in Engineering and Vilas Distinguished Achievement Professor.