Link to paper

The full paper is available here.

You can also find the paper on PapersWithCode here.

Abstract

  • Developing tools for automated cortical segmentation requires topologically correct segmentations.
  • Accurate cortical segmentation is difficult due to image artifacts and the highly convoluted anatomy of the cortex.
  • A novel deep learning-based cortical segmentation method is proposed which incorporates prior knowledge about the geometry of the cortex.
  • A loss function is designed which uses Laplace’s equation to penalize unresolved boundaries between tightly folded sulci.
  • The approach is demonstrated to outperform baseline segmentation networks.

Paper Content

Introduction

  • Segmentation of the cerebral cortex from MRI is important for understanding neurological disorders.
  • Automated segmentation methods are challenged by artifacts such as image noise, partial volume effects, and intensity inhomogeneities.
  • The cerebral cortex is defined as the space between two cortical surfaces: the pial surface and the white matter surface.
  • The cortex has a complex geometry and is often modelled as a highly folded 2D sheet.
  • Geometrically accurate segmentation of the cortex requires accurate reconstruction of both the WM and pial cortical surfaces.
  • Imaging artifacts often introduce topological defects in the resulting surface reconstructions.
  • Several state-of-the-art methods have been developed to address this problem.

Methods

Backbone segmentation network

  • Proposed Laplacian solver is compatible with any semantic segmentation network.
  • Experiments conducted using two backbone networks for 3D image segmentation.

Differentiable laplacian solver

  • Laplacian field is computed based on a given input patch
  • Laplace’s equation is a second-order partial differential equation
  • Boundary conditions are set for the Laplacian field
  • Laplacian field is approximated using the finite-difference method
  • Successive Over Relaxation (SOR) algorithm is used to solve for the Laplacian field
  • Initial values for all voxels in an image are given
  • SOR algorithm accelerates the rate of convergence
  • Laplacian solution is updated in two steps
  • Computations used to generate the Laplacian field must be differentiable
  • Boundary conditions are initialized by taking a weighted sum of GM, WM and background probability maps
  • SOR update step is reformulated as a 1x1 convolutional layer
  • Maximum number of iterations for the Laplacian solver is set to 60

Loss function

  • Model is trained by comparing predicted tissue segmentation to ground truth segmentation
  • Additional loss term compares Laplacian field computed from predicted tissue segmentation to solution of Laplace’s equation applied to ground truth segmentation
  • Laplacian field is converted to multi-label segmentation using thresholding functions
  • Loss terms are equally weighted using combined Dice and cross-entropy loss

Experiments

Dataset

  • 27 brain donors used for ex vivo images
  • Images obtained from National Disease Research Interchange and University of Pennsylvania Center for Neurodegenerative Disease Research
  • Pre-consent and next-of-kin consent given
  • Images scanned on 9.4 Tesla 31 cm bore MRI scanner
  • Resolution of 0.2 x 0.2 x 0.2 mm3
  • Semi-automatic interpolation technique used to generate 3D segmentations of the MTL cortex
  • Iterative finite-differences approach used to solve Laplace’s equation
  • Source and sink boundary conditions semi-manually labeled

Implementation details

  • Used Pytorch 1.9.1 and Nvidia Quadro RTX 500 GPUs to train models
  • Implemented differentiable Laplacian solver within standardized training framework
  • Made modifications to default training parameters
  • Set ignore_label parameter to 0
  • Increased oversample_foreground parameter
  • Used input patch size of 96 x 96 x 96
  • Trained with batch size of 2 or 4 for 250 epochs
  • Tested performance with 5 or 10 class labels
  • Used β = 10 and thresholds spanning [0,1] range
  • Tested effect of increasing weight given to Laplacian segmentation loss

Evaluation

  • Compared performance of approach with corresponding backbone segmentation networks
  • Measured segmentation accuracy by computing DSC and HD between predicted and ground truth segmentations
  • Re-computed Laplacian field for both ground truth and predicted segmentations
  • Evaluated effect of introducing Laplacian constraint on downstream cortical thickness measures
  • Applied nnU-Net models to dataset of 36 temporal lobe specimens
  • Measured MTL thickness at 6 manually identified landmarks
  • Compared thickness measurements obtained using baseline nnU-Net and proposed model in terms of correlation and ICC

Results and discussion

Segmentation accuracy

  • Proposed framework improves Laplacian segmentation accuracy compared to baseline networks
  • Laplacian term implicitly includes information about the location of the sulcus
  • DSC metrics of Laplacian segmentations show improvement in anterior and posterior MTL
  • Proposed network is able to detect buried sulcus in cortical fold not included in ground truth

Downstream thickness measurements

  • Proposed deep learning-based solution for cortical segmentation improves accuracy of segmentations across the whole length of MTL
  • Correlations between automated segmentation-based cortical thickness measurements and reference measurements are stronger with proposed network
  • Proposed network captures layered geometry of cortex by locally imposing Laplacian mappings between predicted WM and pial cortical surfaces
  • Long run-time of iterative solver during training, but only 3-5 minutes per scan at inference time
  • Need for sulci to be well delineated in training data
  • Can be extended to other high-resolution neuroimaging datasets and in vivo image segmentation tasks