By watching videos, this robot becomes as competent as a human surgeon

© Anna Shvets/Pexels

Robotic surgery is a field that has been progressing at full speed in recent years. After its AASTR system (Autonomous System for Tumor Resection), Johns Hopkins University has struck again. One of their teams developed an imitation learning system that allows a robot to acquire surgical skills by watching videos of operations performed by experienced practitioners.

The results of their work were presented at the Conference on Robot Learning (CoRL) held in Munich from November 6 to 9.

Artificial intelligence is coming to the operating room

The device is based on the da Vinci surgical system; often referred to as a robot, it is actually a set of instruments, normally controlled remotely by a teleoperator. Robotic arms, miniaturized surgical instruments, high-definition 3D visualization system: a true technological gem. It is already widely deployed in hospitals around the world.

John Hopkins researchers have integrated a new approach to learning into this robot that shares its architecture with that of ChatGPT. However, while ChatGPT excels in manipulating words and text, this model ” speaks robot ” through kinematics, accurately converting surgical movements into complex mathematical formulas. It's like giving the robot an ” equation ” to tell it how to move.

To train it, scientists mined a goldmine of surgical data. The team collected and analyzed hundreds of video recordings captured by miniature cameras positioned on the articulated arms of da Vinci robots during real-world surgeries.

These sequences, initially archived for post-operative analysis, served as the training corpus for this da Vinci 3.0. The richness of this database is explained by the omnipresence of the da Vinci system: nearly 7,000 units are currently in service across the globe, handled daily by a community of more than 50,000 qualified surgeons.

The robot was therefore able to assimilate all the subtleties of surgical procedures by simple observation, as an intern would do alongside a senior physician. The recordings capture not only the technical movements, but also the variations and adaptations to different situations encountered in surgery.

Subscribe to Presse-citron

200% Deposit Bonus up to €3,000 180% First Deposit Bonus up to $20,000

From observation to practice: impressive results.

The tests focused on three fundamental gestures: needle manipulation, tissue lifting, and suturing. For each of these tasks, the robot demonstrated a skill comparable to that of human surgeons. “The model is fascinating: we only feed it images and it predicts the robotic movements needed for the procedure. For us, this is a breakthrough that paves the way for a new era in medical robotics ,” says Axel Krieger, assistant professor in the Department of Mechanical Engineering at Johns Hopkins.

The major innovation lies in the learning of relative movements (movements defined in relation to the current position of the tool) rather than absolute movements (movements defined in relation to a fixed point in space), making it possible to compensate for the inaccuracies inherent in the da Vinci system. Thus, by learning relative movements, the robot can adapt in real time to small variations and inaccuracies in the system, adapt more easily and gain in precision.

The researchers noted that the robot even develops abilities that were not programmed in advance. “If the needle falls, it automatically picks it up and continues the operation. This behavior has never been taught to it », says Professor Krieger. Capabilities that will completely transform the approach to surgical robotics. Instead of manually coding each step – a job that could take a decade for a single procedure – learning a new intervention now only takes a few days.

The team is currently continuing its work to extend this learning method to complete surgical interventions. If this system becomes democratized one day, the question of ethics arises. If an error occurs during a robot-performed procedure, who is to blame? The supervising surgeon, the robot manufacturer, the programmer, or the robot itself. How far should robots be allowed to act autonomously? Developing strong ethical and legal frameworks will be imperative to ensure the safety of such technology.A surgical robot has learned to operate by watching videos using an imitation learning system inspired by ChatGPT.

Capable of performing complex surgical procedures, its precision is comparable to that of human surgeons.

This advance is a small revolution for surgery, but is accompanied by ethical and legal issues that should not be underestimated.

📍 To not miss any Presse-citron news, follow us on Google News and WhatsApp.

[ ]