Link to paper
The full paper is available here.
You can also find the paper on PapersWithCode here.
Abstract
- Developing tools for automated cortical segmentation requires topologically correct segmentations.
- Accurate cortical segmentation is difficult due to image artifacts and the highly convoluted anatomy of the cortex.
- A novel deep learning-based cortical segmentation method is proposed which incorporates prior knowledge about the geometry of the cortex.
- A loss function is designed which uses Laplace’s equation to penalize unresolved boundaries between tightly folded sulci.
- The approach is demonstrated to outperform baseline segmentation networks.
Paper Content
Introduction
- Segmentation of the cerebral cortex from MRI is important for understanding neurological disorders.
- Automated segmentation methods are challenged by artifacts such as image noise, partial volume effects, and intensity inhomogeneities.
- The cerebral cortex is defined as the space between two cortical surfaces: the pial surface and the white matter surface.
- The cortex has a complex geometry and is often modelled as a highly folded 2D sheet.
- Geometrically accurate segmentation of the cortex requires accurate reconstruction of both the WM and pial cortical surfaces.
- Imaging artifacts often introduce topological defects in the resulting surface reconstructions.
- Several state-of-the-art methods have been developed to address this problem.
Methods
Backbone segmentation network
- Proposed Laplacian solver is compatible with any semantic segmentation network.
- Experiments conducted using two backbone networks for 3D image segmentation.
Differentiable laplacian solver
- Laplacian field is computed based on a given input patch
- Laplace’s equation is a second-order partial differential equation
- Boundary conditions are set for the Laplacian field
- Laplacian field is approximated using the finite-difference method
- Successive Over Relaxation (SOR) algorithm is used to solve for the Laplacian field
- Initial values for all voxels in an image are given
- SOR algorithm accelerates the rate of convergence
- Laplacian solution is updated in two steps
- Computations used to generate the Laplacian field must be differentiable
- Boundary conditions are initialized by taking a weighted sum of GM, WM and background probability maps
- SOR update step is reformulated as a 1x1 convolutional layer
- Maximum number of iterations for the Laplacian solver is set to 60
Loss function
- Model is trained by comparing predicted tissue segmentation to ground truth segmentation
- Additional loss term compares Laplacian field computed from predicted tissue segmentation to solution of Laplace’s equation applied to ground truth segmentation
- Laplacian field is converted to multi-label segmentation using thresholding functions
- Loss terms are equally weighted using combined Dice and cross-entropy loss
Experiments
Dataset
- 27 brain donors used for ex vivo images
- Images obtained from National Disease Research Interchange and University of Pennsylvania Center for Neurodegenerative Disease Research
- Pre-consent and next-of-kin consent given
- Images scanned on 9.4 Tesla 31 cm bore MRI scanner
- Resolution of 0.2 x 0.2 x 0.2 mm3
- Semi-automatic interpolation technique used to generate 3D segmentations of the MTL cortex
- Iterative finite-differences approach used to solve Laplace’s equation
- Source and sink boundary conditions semi-manually labeled
Implementation details
- Used Pytorch 1.9.1 and Nvidia Quadro RTX 500 GPUs to train models
- Implemented differentiable Laplacian solver within standardized training framework
- Made modifications to default training parameters
- Set ignore_label parameter to 0
- Increased oversample_foreground parameter
- Used input patch size of 96 x 96 x 96
- Trained with batch size of 2 or 4 for 250 epochs
- Tested performance with 5 or 10 class labels
- Used β = 10 and thresholds spanning [0,1] range
- Tested effect of increasing weight given to Laplacian segmentation loss
Evaluation
- Compared performance of approach with corresponding backbone segmentation networks
- Measured segmentation accuracy by computing DSC and HD between predicted and ground truth segmentations
- Re-computed Laplacian field for both ground truth and predicted segmentations
- Evaluated effect of introducing Laplacian constraint on downstream cortical thickness measures
- Applied nnU-Net models to dataset of 36 temporal lobe specimens
- Measured MTL thickness at 6 manually identified landmarks
- Compared thickness measurements obtained using baseline nnU-Net and proposed model in terms of correlation and ICC
Results and discussion
Segmentation accuracy
- Proposed framework improves Laplacian segmentation accuracy compared to baseline networks
- Laplacian term implicitly includes information about the location of the sulcus
- DSC metrics of Laplacian segmentations show improvement in anterior and posterior MTL
- Proposed network is able to detect buried sulcus in cortical fold not included in ground truth
Downstream thickness measurements
- Proposed deep learning-based solution for cortical segmentation improves accuracy of segmentations across the whole length of MTL
- Correlations between automated segmentation-based cortical thickness measurements and reference measurements are stronger with proposed network
- Proposed network captures layered geometry of cortex by locally imposing Laplacian mappings between predicted WM and pial cortical surfaces
- Long run-time of iterative solver during training, but only 3-5 minutes per scan at inference time
- Need for sulci to be well delineated in training data
- Can be extended to other high-resolution neuroimaging datasets and in vivo image segmentation tasks