[PhD thesis] Deep learning based segmentation of thin structures

Supervision: Erwan Kerrien and Fabien Pierre (firstname.lastname@loria.fr)

Team: Magrit (https://magrit.loria.fr/index.en.html)

Overview

This doctoral work aims at designing deep learning-based image segmentation methods adapted to thin objects, in a medical context. Though highly efficient to segment objects from images in general, deep learning approaches have difficulties with thin structures, i.e. when at least one of the object dimensions is negligible. Moreover, annotated data are often scarce in a medical context. Limitations of deep learning methods against such conditions will be carried out and strategies will be proposed to raise them. This doctoral work will be applied to the segmentation of the diaphragm and intracranial blood vessels for validation.

Objectives and scientific context

Thin structures are objects with at least one very small dimension compared to the others. They are hardly preserved in a multiscale context, and deep convolutional neural networks (CNNs) thus have difficulties to accurately segment them in images [1,2,3]. However, variational methods were developed to precisely better preserve such structures. Their potential to improve the accuracy of CNNs has recentely been demonstrated [4].

The main objective of this doctoral work is to develop methods for the segmentation of thin structures, that both leverage the high performances of CNNs, and the geometric accuracy of variational methods so as to improve the segmentation of thin structures. Various levels of coupling will be investigated: post-processing, tailoring the last layers, down to designing new pooling layers. The integration or not of such operations in the training phase will also be adressed.

This work will be applied in a medical context to the segmentation from 3D images of the diaphragm (thin organ, attached to the lungs, 1 tiny dimension) and intracerebral blood vessels (2 tiny dimensions). PDE methods have been considered for diaphragm segmentation [5], but this problem was not addressed by deep learning approaches. Similarly, variational methods have inspired numerous works in vascular segmentation [6, 7, 8, 9]. Here, deep learning methods were also investigated but only in 2D (retinal images, X-ray angiography) [10].

Annotated data are difficult to collect in a medical context. A second objective will investigate patch-based approaches so as to demultiply the training data and enable segmentation in 3D.

Challenges

The candidate will have the opportunity to rely on recent works in Magrit team [11, 4], including a Masters’ internship about diaphragm segmentation. 3 goals will be pursued:

  • Critical review of the literature on prior works to improve the geometric accuracy of segmentation using deep learning methods [1, 2, 3].
  • Coupling a CNN with a variational approach to preserve thin structures.
  • Validation on medical images (3D rotational angiography for blood vessels, and abdominal CT for diaphragm). Our long-term collaboration with the Interventional Neuroradiology department of the University Hospital in Nancy [12] gives us priviledged access to medical data.

Requested skills

The applicant mush hold a Master of Science (or equiv.) in Applied Maths or Computer Science.
We are looking for a highly motivated student, with a solid background in mathematics (PDE), and/or computer science (computer vision). Strong knowledge in machine learning, as well as high skills within deep learning environments such as Keras, tensorflow and/or PyTorch, and proficiency in Python programming in general. Interest in multidisciplinary research and medical applications will be appreciated.

How to apply

Send resume and motivation letter to Erwan Kerrien and Fabien Pierre (firstname.lastname@loria.fr) before 05/21/2020.

Starting date: 10/01/2020 (3 years)

References

[1] O. Ronneberger, P. Fischer, and T. Brox. U-net : Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention, pages 234-241. Springer, 2015.
[2] Liang-Chieh Chen, George Papandreou, Iasonas Kokkinos, Kevin Murphy, and Alan L Yuille. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE transactions on pattern analysis and machine intelligence, 40(4) :834-848, 2017.
[3] Liang-Chieh Chen, George Papandreou, Florian Schroff, and Hartwig Adam. Rethinking atrous convolution for semantic image segmentation. arXiv preprint arXiv :1706.05587, 2017.
[4] Thomas Mouzon, Fabien Pierre, and Marie-Odile Berger. Joint CNN and variational model for fully-automatic image colorization. In International Conference on Scale Space and Variational Methods in Computer Vision, pages 535546. Springer, 2019.
[5] Rangaraj M Rangayyan, Randy H Vu, and Graham S Boag. Automatic delineation of the diaphragm in computed tomographic images. Journal of digital imaging, 21(1) :134-147, 2008.
[6] Liana M Lorigo, Olivier Faugeras, W Eric L Grimson, Renaud Keriven, Ron Kikinis, and Carl-Fredrik Westin. Co-dimension 2 geodesic active contours for mra segmentation. In Biennial International Conference on Information Processing in Medical Imaging, pages 126139. Springer, 1999.
[7] Rodrigo Moreno, Chunliang Wang, and Örjan Smedby. Vessel wall segmentation using implicit models and total curvature penalizers. In Scandinavian Conference on Image Analysis, pages 299-308. Springer, 2013.
[8] Laurent D Cohen and Thomas Deschamps. Segmentation of 3d tubular objects with adaptive front propagation and minimal tree extraction for 3d medical imaging. Computer methods in biomechanics and biomedical engineering, 10(4) :289-305, 2007.
[9] Wei Liao, Stefan Wörz, Chang-Ki Kang, Zang-Hee Cho, and Karl Rohr. Progressive minimal path method for segmentation of 2d and 3d line structures. IEEE transactions on pattern analysis and machine intelligence, 40(3) :696-709, 2017.
[10] Sara Moccia, Elena De Momi, Sara El Hadji, and Leonardo S Mattos. Blood vessel segmentation algorithms – review of methods, datasets and evaluation metrics. Computer methods and programs in biomedicine, 158 :71-91, 2018.
[11] Erwan Kerrien, Ahmed Yureidini, Jeremie Dequidt, Christian Duriez, René Anxionnat, and Stéphane Cotin. Blood vessel modeling for interactive simulation of interventional neuroradiology procedures. Medical image analysis, 35 :685-698, 2017.
[12] René Anxionnat, Marie-Odile Berger, and Erwan Kerrien. Time to go augmented in vascular interventional neuroradiology ? In Workshop on Augmented Environments for Computer-Assisted Interventions, pages 3-
8. Springer, 2012.

Logo d'Inria