Master internship : Computer vision for robotic flexible endoscopy
Title : Environment reconstruction using a monocular endoscopic camera
Keywords : visual tracking, shape from motion, depth recovery, medical robotics
Duration : approximately 5 months (ideally between february and august 2021)
Grant : legal grant for training periods (~ 550 euros / month).
Location : ICube Robotic platform, at IHU Strasbourg
Context : This internship takes place in the scope of the assistance to medical procedures with robotic flexible endoscopes.
The AVR team of the ICube laboratory has developed a robotic platform for endoluminal surgery called STRAS (see photo below). This is a telemanipulated system equipped with an endoscopic camera and two articulated instruments, with 3 degrees of freedom each. In addition to the conventional telemanipulation control, we aim at including automatic modes to the robot, with the aim to perform tasks such as automated scanning, or automatic endoscope positioning. For reaching this aim, one of the difficulties to be tackled is the reconstruction of the shape of the environment with the only available sensor: a monocular endoscopic camera.
Problem to be solved In this project, we aim at reconstructing the shape and position of the environment (tissues in in vivo environment, phantoms in laboratory setups) with respect to the endoscopic camera. The camera being monocular, shape and structure from motion will be primarily used to reconstruct the environment and motions up to a scale factor. Shape from shading could also be envisioned. The difficulties are the low quality of endoscopic images, the limited possible lateral displacement of the endoscope and the possible interactions of the instruments with the tissues creating disturbing motions and deformations. In a second step, we will try to reconstruct the metric shape and positions. This can be done by using odometric measurements on the endoscope. However, these measurements are known to be imprecise. Specific strategies, will thus be needed to recover the unknown scale factor, by using for instance Bayesian filtering approaches or machine learning techniques.
Work to be carried out The intern will have to develop algorithms for shape reconstruction from monocular images by relying on state of the art methods for tissues tracking in endoscopy (gastroenterology in particular). Algorithms have already been implanted for pure tracking and can serve as a basis. Techniques for depth estimation will then be developed, by focusing on the use of embedded measurements provided by the robot encoders. If needed a second miniature camera could be added to the setup. Tests will be carried out in the laboratory on phantoms and on in vivo images acquired during previous preclinical trials.
Work environment The internship will take place on the medical robotic platform of the ICube laboratory located at IHU (Institut Hospitalo Universitaire) in the heart of Strasbourg. The intern will be supervised by Florent Nageotte (associate professor in medical robotics) and Philippe Zanne (Engineer, responsible for the STRAS robotic system). The intern will have access to a computer for developing programs, to image acquisition systems, to in vivo images and to the robotic device for laboratory testing. Developments will be made in C / C++ or Python and possibly with Matlab for prototyping.
Covid19 conditions: In case of sanitary constraints that may prevent the internship to be realized on site, a large part of the work could be done at a distance by working on data acquired off-line. Only robotic testing will be made impossible. The intern will have to work on his/her own laptop either developing and running algorithms locally or at a distance on a connected machine.
Candidates profile We are looking for Master students in the second year or students in engineering school at the level of Master 2, with major in computer vision or robotics / computer science with a strong interest / experience in computer vision. Interest in medical applications is a plus. Proficiency in C/C++ or Python coding is mandatory.
Conditions 5 to 6 months between February 2021 and August / September 2021. The intern will receive the legal “gratification” (around 550€ / month)
Application Interested candidates should send CV / resume, master program and grades (if available) and motivation letter to Nageotte@unistra.fr, by mentioning “computer vision internship” in the email subject.
Open PhD Position in Computer Vision/Deep Learning for Healthcare Sponsored by Intuitive Surgical
The operating room is a high-tech environment in which the surgical devices generate a lot of data about the underlying surgical activities. Our research group aims at making use of this large amount of multi-modal data coming from both cameras and surgical devices to develop an artificial intelligence system that can assist clinicians and staff in the surgical workflow. In this context, we currently have a new PhD position at the University of Strasbourg that will focus on developing machine learning and computer vision methods to understand the scene of the operating room, recognize the human activities, and analyze the workflow. The project will use as input multi-view RGB-D videos capturing surgical activities.
As this PhD position is funded by a fellowship from Intuitive Surgical, the successful candidate will have the opportunity to interact with researchers from Intuitive Surgical and also to conduct internships at the company in Sunnyvale, California.
More information is available here
Open Internship Positions within Project CAMMA
We are looking for motivated and talented students with knowledge in computer vision, machine learning and/or augmented reality who can contribute to the development of our computer vision system for the operating room.
Please feel free to contact Nicolas Padoy if you are interested to do your master's thesis or an internship with us (funding of ~500Euros/month will be provided during 4 to 6 months). The successful candidates will be part of a dynamic and international research group hosted within the IRCAD institute at the University Hospital of Strasbourg. They will thereby have direct contact with clinicians, industrial partners and also have access to an exceptional research environment. The CAMMA project is supported by the laboratory of excellence CAMI, the IdEx Unistra and the MixSurg Institute.
- Deep Learning for Activity Recognition in Large Video Databases
- Multi-view Human Body Tracking for the Operating Room using RGBD Cameras
- IHU Strasbourg / MIX-Surg Institute: http://www.ihu-strasbourg.eu/ihu/en/institut/presentation/
- IRCAD: http://www.ircad.fr/?lng=en
- AVR group: http://icube-avr.unistra.fr/en/index.php/Main_Page
- LabEx CAMI: http://cami-labex.fr/
Open Postdoc/Research Engineer Position in Computer Vision/Deep Learning within Project CAMMA
We are looking for a research engineer in Computer Vision/deep Learning to join the development of our clinical prototypes. The project involves the perception of the Operating Room through a set of RGB-D cameras mounted on the ceiling as well as the recognition of surgical activities using both RGB-D and endoscopic videos. More information is available here.