Team AVR - Control, Vision and Robotics Lab

Florent Nageotte Personal Web Page

From Team AVR - Control, Vision and Robotics Lab
Jump to navigation Jump to search
Associate Professor
in Medical Robotics

français | english

Florent nageotte id2.jpg

Internship offer

Computer vision for robotic flexible endoscopy

pdf file for internship proposal

Title : Environment reconstruction using a monocular endoscopic camera

Keywords : visual tracking, shape from motion, depth recovery, medical robotics

Duration : approximately 5 months (ideally between february and august 2021)

Grant : legal grant for training periods (~ 550 euros / month).

Location : ICube Robotic platform, at IHU Strasbourg

Context : This internship takes place in the scope of the assistance to medical procedures with robotic flexible endoscopes.

The AVR team of the ICube laboratory has developed a robotic platform for endoluminal surgery called STRAS (see photo below). This is a telemanipulated system equipped with an endoscopic camera and two articulated instruments, with 3 degrees of freedom each. In addition to the conventional telemanipulation control, we aim at including automatic modes to the robot, with the aim to perform tasks such as automated scanning, or automatic endoscope positioning. For reaching this aim, one of the difficulties to be tackled is the reconstruction of the shape of the environment with the only available sensor: a monocular endoscopic camera.

Automatic task viewed from the endoscopic camera
STRAS robotic system

Problem to be solved In this project, we aim at reconstructing the shape and position of the environment (tissues in in vivo environment, phantoms in laboratory setups) with respect to the endoscopic camera. The camera being monocular, shape and structure from motion will be primarily used to reconstruct the environment and motions up to a scale factor. Shape from shading could also be envisioned. The difficulties are the low quality of endoscopic images, the limited possible lateral displacement of the endoscope and the possible interactions of the instruments with the tissues creating disturbing motions and deformations. In a second step, we will try to reconstruct the metric shape and positions. This can be done by using odometric measurements on the endoscope. However, these measurements are known to be imprecise. Specific strategies, will thus be needed to recover the unknown scale factor, by using for instance Bayesian filtering approaches or machine learning techniques.

Work to be carried out The intern will have to develop algorithms for shape reconstruction from monocular images by relying on state of the art methods for tissues tracking in endoscopy (gastroenterology in particular). Algorithms have already been implanted for pure tracking and can serve as a basis. Techniques for depth estimation will then be developed, by focusing on the use of embedded measurements provided by the robot encoders. If needed a second miniature camera could be added to the setup. Tests will be carried out in the laboratory on phantoms and on in vivo images acquired during previous preclinical trials.

Work environment The internship will take place on the medical robotic platform of the ICube laboratory located at IHU (Institut Hospitalo Universitaire) in the heart of Strasbourg. The intern will be supervised by Florent Nageotte (associate professor in medical robotics) and Philippe Zanne (Engineer, responsible for the STRAS robotic system). The intern will have access to a computer for developing programs, to image acquisition systems, to in vivo images and to the robotic device for laboratory testing. Developments will be made in C / C++ or Python and possibly with Matlab for prototyping.

Covid19 conditions: In case of sanitary constraints that may prevent the internship to be realized on site, a large part of the work could be done at a distance by working on data acquired off-line. Only robotic testing will be made impossible. The intern will have to work on his/her own laptop either developing and running algorithms locally or at a distance on a connected machine.

Candidates profile We are looking for Master students in the second year or students in engineering school at the level of Master 2, with major in computer vision or robotics / computer science with a strong interest / experience in computer vision. Interest in medical applications is a plus. Proficiency in C/C++ or Python coding is mandatory.

Conditions 5 to 6 months between February 2021 and August / September 2021. The intern will receive the legal “gratification” (around 550€ / month)

Application Interested candidates should send CV / resume, master program and grades (if available) and motivation letter to, by mentioning “computer vision internship” in the email subject.

Pedagogic documents

  • Here is the course on registration for robot assisted medical intervention given in the scope of the PUF Houston/Strasbourg in 2009

Curriculum Vitae

  • 2000 : graduated from the ENSPS (Engineering school), Strasbourg.
  • 2000 : Master in Photonic, Image and Cybernetics, ULP, Strasbourg.
  • 2005 : PhD Thesis, ULP, Strasbourg.


Associate Professor at the University of Strasbourg, department of Physics and Engineering, Strasbourg [1].


  • School of engineering "Telecom Physique Strasbourg", Bio-engineering specialty
    • Course on computer vision for medical applications (pose estimation)
    • 3D rigid medical registration
  • Bachelor in applied physics (Electronic, Signal aud Automatic Control):
    • Courses and tutorials on automatic control (Analysis and control of analog systems)
    • Practical sessions in Automatic Control
  • Master MNE :
  • Master IRIV, IRMC speciality
    • Course on registration for medical robotics


  • Past supervised PhD students
    • Laurent Ott, defended in 2009
    • Bérengère Bardou, defended in 2011
    • Antonio De Donno, defended in 2013
    • Paolo Cabras
    • Laure-Anaïs Chanel
  • Currently supervised PhD theses
    • Rafael Aleluia Porto
  • Research projects
    • STRAS project: Robotic assistance to flexible surgical endoscopy
    • ProteCT: Robotic assistance to non-vascular inteventional radiology
    • Robotic HIFU

  • Past Master students
    • Zeineb Zarrouk (2011)
    • Maria Jakubowska (2011)
    • Florent LeBastard (2012)
    • David Goyard, KTH Stockholm / Ecole Centrale de Lille (2013)
    • Natalia Shepeleva, ViBot Master (2015)
    • Mohamed Amine Falek (2016)
  • Current Master students
    • Xuan Thao HA
    • Valentin Bordoux (advised by B. Rosa)
  • PhD thesis :

Robot and Computer assisted medical interventions :Suturing assistance in laparoscopic surgery


  • Vision control
  • Visual servoing
  • Parameter estimation using vision
  • Path planning
  • Medical and surgical robotics


downloadable publications

List of publications

<anyweb> </anyweb>

personal area

Seattle, WA (ICRA 2015)

Downtown from Lake Union
Welcome Dinner at the Experience Music Project / Science Fiction Museum
North view from Columbia Center

Tokyo (Medical robotics seminar at the french embassy)

Asakusa Shrine
Tokyo from Sunshine60
Shibuya by night

Texas (Computational Surgery 2011)

San Antonio Riverside
Fort Alamo
Texas Medical Center Houston

Minneapolis, MN (EMBC09)

Downtown Minneapolis
The largest Mall in the USA
Lake Calhoun)

Japan (Icra09, Kobe)

Kyoto - Kinkaku-Ji
Kobe in sunlight
... and at night

Scottsdale, AZ (Biorob08)

Scottsdale at sunset
The "Sun Valley" viewed from "Camel Moutain"
The "best student" rest

California (Icra08, pasadena)

Flock of Sealions
Spare vehicules
Santa Barbara

Beijing (Iros06)

Summer Palace
Turtle soup
The Great Wall in Grande muraille in mist

Ontario (visit by MDRobotics september 06)

Niagara falls
Toronto from CN tower
CN tower, Toronto

San Diego (Medical Imaging 05)

Balboa park
Dolphins in open sea

Chicago (Cars04)

[[Image:Photo 032.jpg|thumb|left|200px |