Team AVR - Control, Vision and Robotics Lab

STRAS - Robotization of flexible systems for minimally invasive surgery

From Team AVR - Control, Vision and Robotics Lab
Jump to: navigation, search


STRAS is a teleoperated robotic prototype for assistance to flexible endoscopic surgery. The current version of the prototype (v2) has already been used in more than 20 procedures on animal models, mainly for simulated ESD (Endoscopic Submucosa Dissection) procedures. These medical procedures will be of major importance for the treatment of early colorectal cancers in the coming years. They are conventionally realized using standard colonoscopes, but due to the difficulty of the task, at the limit between gastro-enterology and surgery, only a few experts are actually able to perform the procedure in western countries.

STRAS provides a large set of advantages for physicians during intraluminal colorectal procedures. The platform, based on Karl Storz® Anubis® system, allows the simultaneous standard use of two (and up to three) flexible instruments in a triangulated, laparoscopic-like, configuration. All mobilities (12 in total) of the endoscope and the instruments are robotized in a modular patented architecture, allowing easy setup and manipulation of the medical components. STRAS provides great stability of the endoscope and instruments and major improvements in the comfort, intuitivness, and precision of control thanks to dedicated, patent-pending, master interfaces. As demonstrated in many in vivo trials, STRAS can be easily teleoperated by a single surgeon for performing complex procedures, resulting in shorter and less tiring interventions.


Teleoperation of STRAS v2 with the dedicated master interfaces during pre-clinical trials
Slave system of STRAS v2


The video at this link shows some of the capabilities of the system.


Scope

Flexible endoscopy has long been used for diagnosis purposes only. But with the general trend in surgery to try to minimize invasiveness it has come a major tool for surgical procedures as shown by the development of specific flexible surgical platforms. Unfortunately these systems are not directly adapted to surgery. Several surgeons have to coordinate to manipulate the available DOFs, which result in difficult operations and potential high functioning costs. We envision a solution in the development of robotic tools for flexible endoscopic surgery. The aim of the ISIS project is to show that by using robotics it is possible for a single surgeon to perform surgical procedures intuitively with flexible systems. The chosen path to this goal is to build on existing asserted medical systems, such as flexible endoscopes manufactured by Karl Storz, rather than developing complete new systems. Issues arise at several levels (see the following for more details) :

  • Small-size mechanical design for developing light, compact and modular slave systems
  • Low-level control of cable-driven systems with significant non-linearities
  • Cartesian control of flexible instrument made difficult by imperfect models and lack of measurement solutions

Overall the project includes mechanical development, low-level control, high level strategies and measurement solutions development.

Keywords: Flexible endoscopy, continuum robots, teleoperation

Persons involved in the project

  • PhD students: R. Porto (starting December 2016)
  • Research engineers: L. Zorn (mechatronic design and realization), Ph. Zanne (Low-level software development, electrical design and realization), L. Goffin (GUI Software development and software validation)
  • Researchers: F. Nageotte, M. de Mathelin
  • Past engineers : F. LeBastard (Mechatronic design)
  • Past PhD students: P. Cabras (vision-based measurement), A. De Donno (teleoperation), B. Bardou (teleoperation and control), L. Ott (vision-based control)
  • Past Master students : N. Shepeleva (vision-based measurement), D. Goyard (learning and vision-based measurements), F. Le Bastard (mechanical transmission modelling), Z. Zarrouk (control)

Contacts

Florent Nageotte, Philippe Zanne, Lucile Zorn

Partners

The EASE / STRAS project is developed in collaboration with Karl Storz GmBH and surgeons from the IRCAD (Dr B. Dallemagne, Dr S. Peretta) and the IHU (A. Legner). EASE prototype has been partly outsourced to TelerobotLabs (Genova, Italy). Axilum Robotics provides support for software and regulations. P. Cabras was funded by CAMI labex and his thesis project was in collaboration with Agathe team (ISIR Paris). STRAS v2 realization was partly outsourced to Sandmann company (Bennwihr Gare, France) and Opticab.

In details

Visual control of flexible endoscopes

STRAS : a robotic system for intraluminal surgery

During the ISIS project, we have developed a robotic prototype based on the Anubis platform from Karl Storz. The proposed system is modular and consists of three kinds of modules:

  • an endoscope module, which motorizes the four-ways deflection of the main endoscope tip
  • instruments modules, which enables the two-ways deflection of instruments. They come in two flavours: electrical instruments modules for instruments needing electrical pluging, such as electro-coagulation tools and mechanical instruments needing an opening / closing capability, such as scisors or graspers.
  • Translation and Rotation modules (T/RMs) which allow the rotation and translation of the instruments inside the channels of the main endoscope.

TR/Ms and the endoscope modules are mounted onto passive positioning arms, which can be attached to the operating table. With this structure only two T/RMs are needed, whatever the number of instruments required for the surgery.

The whole motorization is located at the proximal side, which makes the robotic system easily compatible with medical constraints.

This slave system can be teleoperated using two master interfaces, enabling a single user to control all degrees of freedom. First tests have been performed in the lab, which have shown that STRAS allows a better and more comfortable control of the flexible instruments than in the manual version.

Overview of STRAS v1
Motorization on the proximal part of STRAS v1








Control of flexible systems

Flexible systems used in the STRAS prototype have very particular behaviours due to their kinematic properties (singularities at the straight position) and their mechanical actuation (cable driven systems) which makes their control difficult. We are investigating how to improve the control of these systems at two levels: At low level we try to understand the local behaviour of the flexible systems and at higher level we analyze the effect of the choices of master / slave control modes and mappings on the overall control capabilities.

Visual measurement

Because of frictions, backlash and deadzones, it is difficult to estimate the position of instruments using only internal sensors such as motor encoders. On the other hand there are no commercially available sensors which can be integrated inside small diameter instruments and which would allow to measure small enough radii of curvature. For improving the control of flexible instruments, we work on the use of the endoscopic images for measuring the positions and motions of the instruments. This information can be fused with encoders measurements to improve the obtained accuracy.

In the PhD thesis of Paolo Cabras, we developed image processing and computer vision tools allowing to measure the position of flexible instruments using mainly the embedded endoscopic camera. The approach is based on the use of color markers attached to the bendable part of the flexible instruments. We developed graph-based techniques which allow to extract and reconstruct the image structure of the instruments despite possible partial occultations, the strong specularities caused by direct lighting and numerous reflections onto organs. Because of the space necessary for allowing instruments to slide and rotate inside the guide channels, it is difficult to accurately modelize the position and orientation of instruments with respect to the embedded camera. Consequently, standard model-based approaches fail to estimate the position of instruments even when features are correctly extracted. Therefore we proposed an adaptive method, which allows to automatically adapt to change in position and orientation of the instrument in the channels and provide good estimation capabilities.

We also investigated the possibility to use learning-based approches for pose estimation. Learning techniques can avoid modeling at all the link between positions and images and learn it from demonstrations if a ground truth can be made available during a training stage. We adapted several learning techniques, such as RBF and LWR, to the estimation problem and were able to obtain similar accuracy than with models, but with the avantage of using simpler image features.

Major facts

  • Mai 2 2016: Presentation of STRAS to 250 engineers from Siemens, during Siemens days at IRCAD
  • Feb. 24 2016: PhD defense of Paolo Cabras on the pose estimation of flexible instruments of STRAS from monocular vision. The comitee was composed by A. Casals (President), G. Morel (reviewer), N. Andreff (reviewer), M. de Mathelin (examiner), C. Doignon (PhD supervisor), F. Nageotte (PhD Advisor)
  • Jun. 26 2015: STRAS successfully tested for single port procedures: Simulated appenditectomy, cholecystectomy, dissection of the gastro-oesophageal junction
  • May 11: STRAS presented to J.M Le Guen, State secretary for relations with the parliament and physician
  • Dec. 2014 : Success of in vivo trials of ESD on pigs
  • Sep. 2014 : Start of a new project, called "EASE", for developing a clinical version of STRAS. EASE is Funded by SATT Conectus with support of Karl Storz and IRCAD
  • Sep 2014 : new project for developing a clinical version of STRAS funded by SATT Conectus with support of Karl Storz
  • Sep. 1/2 2014 : in vivo experiments at the IRCAD
  • Feb. 12 2014 : Alsace 20 (TV broadcaster) has filmed STRAS v2 for broadcast program Lab20 to be seen on friday feb. 14 and on the website of Alsace 20 from feb. 17.
  • Feb. 5 2014 : In vivo tests of ESD with a novice surgeon
  • Jan. 30 2014 : Demonstration of STRAS v2 to M. F. Hollande, president of the republic during his visit at IRCAD
  • Jan. 30 2014 : Demonstration of STRAS v2 to Mrs Storz, CEO of Karl Storz
  • Jan. 10 2014 : Demonstration of STRAS v2 to a Karl Storz delegation
  • Jan. 7 2014 : Demonstration of STRAS v2 to a delegation of Covidien
  • Dec. 19 2013 : Second operation tests on animals
  • Dec. 13 2013 : A. De Donno successfully defended his PhD. The defense took place in the "Salle des Actes" at the faculty of Medicine in Strasbourg. The jury was composed by Ph. Cinquin (President), A. Menciassi (reviewer), J. Szewczyk (reviewer), M. de Mathelin and F. Nageotte.
  • Dec. 5 2013 : First in vivo tests of STRAS v2. Our medical partners at IRCAD successfully performed an ESD in an animal.
  • Nov. 4 2013 : F. Nageotte presented control strategies for STRAS at IROS in Tokyo.
  • Oct. 29 2013 : First ex vivo tests of STRAS v2. Successful fake ESD in a pig stomach.
  • Oct. 24 2013 : First lab tests of the new version of STRAS (v2) on chicken breast.
  • Oct 2013 : Second version of STRAS has been assembled
  • May 2013 : A. De Donno presented STRAS at IEEE ICRA in Karlsruhe
  • Presentation of STRAS robotic system during visits of Siemens HealthCare CEO (march 12), Mrs Sybil Storz, CEO of Karl Storz (march 13) and M. Louis Gallois (Commissaire général à l'investissement) (march 14).
  • Nov. 30 2012 : Presentation of STRAS robotic system at the IRCAD during Sanofi CEO visit
  • Nov. 29 2012 : first trials by IRCAD surgeons
  • oct. 2012 : presentation of surgical simulator at the "Fête de la Science" in Strasbourg
  • Oct. 2012 : Paolo Cabras has joined the project as PhD student working on image processing tools for system control
  • 2011 : Presentation of STRAS system at the IRCAD lab during visit of Laurent Wauquiez, minister of research and higher education