3D segmentation of the whole heart vasculature using improved multi-threshold Otsu and white top-hat scale space hessian based vessel filter

Quantification of vessel density helps to know the stage of the disease during diagnosis and patient's response to treatment. However, this requires presence of all vessels in the image. The available segmentation techniques that are manual based are prone to errors, tiresome and slow, while some that are automated do face difficulty in distinguishing the vessel tissue from the non-vessel tissue due to the presence of intensity inhomogeneity and noise in images. Therefore, there is need for improved segmentation methods that can extract all sizes of vessels for better quantification of the vessel density and improved decision making during diagnosis. In this paper, a 3D hybrid approach for segmentation has been developed, based on white top hat scale space hessian vessel enhancement filter and multi-threshold Otsu method. The hybrid method can address the intensity inhomogeneity, as a result, more vessels of different sizes are detected. The method is also robust and able to detect abnormalities in the vessels.


INTRODUCTION
Segmentation of human heart blood vessels plays an important role in a number of clinical procedures such as in the quantification of vessel density, arteriogenesis, stenosis, and tortuosity which requires all the blood vessels (tubular structures) to be detected for proper diagnosis [23]. The complex nature of medical images has made extraction of the tubular structure difficult. The presence of intensity inhomogeneity and noise has led to difficulty in distinguishing vessels from the nonvessels, difficulty in separating arteries from the veins and because of this, most of the researchers have resorted to segmenting 2D image planes rather than 3D whole image which lead to inconsistencies and non-smooth vessel surface in the results. Despite the fact that 2D image segmentation is easy, 3D segmentation provides full information about anatomic structure of the vessels as compared to 2D based segmentation. There are 3D segmentation techniques that segment the whole human heart vessels. Most of these segmentation techniques are model based [29] [31] [32] [34] [42] . Model based segmentation techniques are slow, involve human intervention and find hardship in detecting small vessels and abnormalities in the vessels [28]. Some of the 3D based pattern recognition techniques [44] [45], segment parts rather than the whole human heart vasculature yet all vessels are needed during quantification of vessel density for better diagnosis. Other pattern recognition techniques claim to be 3D based yet they segment 2D image planes and then later reconstruct to form 3D image [35] [36] [37] [46]. A few 3D based pattern recognition techniques that segment the whole heart are slow, computationally expensive and face difficulty in detecting different sizes of vessels due to the presence of noise and intensity inhomogeneity [33].
In this paper, a 3D hybrid approach has been developed, based on white top hat scale space hessian vessel enhancement filter and multi-threshold Otsu method for the extraction of the whole human heart vasculature in medical images that suffer from intensity inhomogeneity problem.
The rest of the paper is structured as follows: Section II presents the related segmentation methods; Section III discusses the proposed methodology; Section IV presents the experimental results and discussions; Section V concludes the paper. II.

RELATED WORKS
This section briefly reviews some segmentation algorithms. Segmentation techniques employed in the extraction of vessels from medical images are categorized into three classes namely: model based segmentation techniques [24] [46]. Among the classes of the segmentation, model based segmentation method is a popular and widely used method in the extraction of vessels from the heart. However techniques that employ model based segmentation techniques are time consuming as they require human intervention during parameter setting and model fitting [37] [47] [48]. Model based segmentation techniques find hardship in detecting small vessels and abnormalities in the vessels [28]. Although tracking techniques have the ability to estimate vessel diameter with a high performance, they are slow, involve human intervention and cannot handle vessel crossover points [26] [27].
Pattern recognition methods are associated with high accuracy and are fast methods as they do not involve a lot of human intervention. However, some of the pattern recognition methods segment parts of the vessels instead of the whole heart vessels and are 2D based although they claim to be 3D based. These techniques analyse 2D image planes and later reconstruct into 3D image which leads to loss of information and nonsmooth surface in the segmentation results [30]. Although 3D vessel segmentation is difficult and slow, it provides full information about anatomic vascular structure as compared to 2D vessel segmentation [29] [30].
There are very few pattern recognition techniques that are 3D based and segment the whole human heart. However these techniques are computationally intensive, slow and still find difficulty in distinguishing the vessel tissue from the non-vessel tissue due to presence of intensity inhomogeneity in images. Therefore there is need for 3D pattern recognition techniques that are capable of addressing the intensity inhomogeneity problem in order to distinguish vessel tissue from non-vessels.
Among the pattern recognition methods, Frangi hessian based vessel enhancement filter is popular and widely used for segmentation of vessels because of its capability to detect vessels even in the presence of noise. It also facilitates measurement of vessels such as extraction of diameters of vessels [1] [15]. However Frangi hessian based vessel enhancement filter causes blurriness at the edges of the vessels in magnetic resonance angiogram (MRA) images, slow and face difficulty in segmenting vessels in image with intensity inhomogeneity such as magnetic resonance imaging (MRI) images especially if applied in 3D segmentation [1] [17] [20], improved Frangi hessian based vessel enhancement filter by introducing level set and incorporating an external constrained term based variance into the level set in order to increase the speed of hessian based vesselness filter and also to address the blurriness problem, the method can only work on high contrast images and it is still slow.
According to [10] [11], top hat morphological transformation is good at enhancing the contrast of the image, and better results can be obtained if a multiscale approach is employed with application of appropriate structuring element whose size increases with increase in the scale size. Top hat morphological transform is capable of not only separating different organs but also in enhancing edges of the region of interest, correcting illumination in order to uncover the objects rendered absent in the images and also get rid of the unwanted tissues in MRI image [10] [11]. Recently, [49] improved Frangi hessian based vessel enhancement filter by combining top-hat morphological transform and hessian matrix. The method produced promising results. However it was applied on a 2D image planes and it misses small vessel. It also leads to blurriness in the edges of the vessels due to the use of Gaussian convolution [17] [19], and does not maintain the edges of the small vessels as the vessels (especially small vessels) are split as shown in figure 1c.
Further improvement could include the use of the second order information (Hessian) of the white top-hat morphological transform to analyse the local behaviour of the image to address the problems (such as blurriness) associated with Gaussian convolution and intensity inhomogeneity problem in order to detect vessels.
According to [50] [47], visibility of the smalls vessels would be enhanced if Frangi hessian based vessel enhancement filter is combined with anisotropic diffusion, since anisotropic diffusion is capable of reducing noise, preserving geometry, topology and localization of the small vessels. More vessels (especially big vessels) could be extracted using Otsu thresholding [18].
Otsu thresholding is not only capable of extracting big vessels but it also has an advantage of smaller storage, fast processing speed and ease of manipulation [12] [14] [39] [40]. Otsu thresholding is also computationally inexpensive, [ [38]. 3D Otsu is more resistant to noise as compared to 1D and 2D based Otsu thresholding and utilizes not only grey levels of pixels but also spatial information [8].
A combination of improved multi-threshold Otsu thresholding, anisotropic diffusion, improved Frangi hessian based vessel enhancement filter and bwareaopen morphological operator would yield better results with improved multithreshold Otsu thresholding used to extract big vessels, bwareaopen morphological operator used to remove the small objects, Anisotropic diffusion used to maintain the edges of the small vessels and improved Frangi hessian based vessel enhancement filter used to extract capillaries and small vessels.

METHOD AND MATERIALS
In order to avoid false detections during segmentation of the heart vessels, the images are segmented while considering the information about the heart vessel structure and also the image modality (MRI). In our approach, segmentation is carried out two times to enable extraction of vessels of different sizes. Improved multi-threshold Otsu and improved Frangi hessian based vessel enhancement filter are used to extract big and medium vessels and small vessels respectively with anisotropic diffusion filter used to maintain the small vessels and bwareaopen morphological operator used to get rid of the very small objects. Multi-thresholding Otsu technique is extended to 3D. One level is utilized to improve processing speed and efficiency in [18]. The efficiency of Frangi hessian based vessel enhancement filter is improved by incorporating white top-hat morphological transform in the hessian based vesselness filter instead of the Gaussian filter in order to increase the speed and efficiency of Frangi hessian based vesselness on images (such as MRI, CT (computed tomography) faced by intensity inhomogeneity. We call the improved Frangi hessian based vessel enhancement filter a white top-hat scale space hessian based vessel enhancement filter. Figure 2 shows the proposed methodology for extraction of the heart vessels. Our approach consist of seven stages, during the first stage, different sizes of vessels are segmented using the modified multi-threshold Otsu thresholding. The resulting image is denoised using bwareaopen morphological operator in the second stage.
Since multi-threshold Otsu in [18] is not so good at segmenting small vessels and has a tendency of cutting vessels [2], we then employ white top-hat scale space hessian based vessel enhancement filter for the segmentation of the small vessels during the fourth stage. We chose to apply the white tophat scale space hessian based vessel enhancement filter to the original image to ensure extraction of small vessels and medium vessels. Since small vessels have weak edges, we used anisotropic diffusion before white top-hat scale space hessian based vessel enhancement filter to detect and maintain edges in the third stage in order to reduce noise, preserve the geometry, topology and localization of the small vessels [47]. We then got rid of the small objects in the image in the fifth stage.
In order to obtain a final segmented vessel image, the results obtained using multi-threshold Otsu and bwareaopen morphological operator are then added to results obtained with white top-hat scale space hessian based vessel enhancement filter, anisotropic diffusion and bwareaopen morphological operator during the sixth stage. The result is then post processed to remove unnecessary objects using bwareaopen morphological operator during the seventh stage.

A. Improved Multi-threshold Otsu Method
2D multi-threshold Otsu thresholding of [18] is extended to 3D mult-level Otsu method and it is used during the iinitial segmentation of vessels of different sizes. 3D multi-threshold Otsu method is different in such a way that, we just need to adjust the gray levels until we get the optimal threshold. The adjustment of thresholds is done at the beginning of segmentation and there is no human intervention during segmentation after an optimal threshold is obtained. However the threshold is sensitive to gray values. The improved Otsu utilizes single level during segmentation in order to reduce on processing time and avoid loss of the vessels.

B. White Top-Hat Scale Space Hessian based Vessel Enhancement Filter
A multiscale approach is employed with the aim of extracting detailed information at various scales. The size of the structuring element depends on the scale (scale specified in Frangi hessian based vessel enhancement). Increase in the scale leads to increase in the structuring element allowing detailed extraction of information at various scales [16] [22]. Given Frangi vesselness scale space of levels n= 1, 2……, L.
Where a level with index L corresponds to a top-hat transform with structuring element size where increases with L.
A white top-hat scale space morphological operator is given by:-

= − ( ∘ )
Where I is the image, denote the structuring element used at level n. In this research, we used a square structuring element.
In the Frangi hessian based vessel enhancement filter, the local behavior of the image is analysed using the second order information of the Gaussian, we choose to analyse the local behaviour of an image using the second order information (Hessian) of the white top-hat morphological transform. The eigenvalues obtained from the hessian matrix are used to extract tubular structures (vessels). The hessian matrix in the point x at scale n, is given by:-To distinguish tubular structures from blob-like or plate-like structures and background. A dissimilarity measure in equation is used [2]. is the ratio of the two biggest eigenvalues that indicates whether the local structure is more plate-like or tube-like, the term accounts for blob-like structures and the term S deals with the difference between vessel and background areas.
The decomposition of the local second order structure of the image extracts the eigenvalues and sort them in ascending order (| 1 | ≤ | 2 | ≤ | 3 |) and principal direction (̂1,̂2,̂3).
The above vesselness measure uses the filter responses at different scales taking the maximum response:-

IV. EXPERIMENTAL RESULTS
We tested the proposed 3D segmentation method on 5 original MRI images of different sizes obtained from Josef Ehling. The images used in this paper are of size 256x256x300. 240 MRA images from http://www.osirix-viewer.com/datasets/ of (size= 576x448) captured by MRI 3T Prisma [49]. In all images, vessels and cerebral aneurysm were automatically segmented. Matlab 2016a is used during simulation.  Otsu thresholds of 32, 49 and 50 are used to get rid of the outer tissue, segment out the whole heart and the heart vessels respectively. Otsu Threshold 33 is used in the segmentation of cerebral aneurysm and associated vessels such as cerebral vessels and carotid artery as shown in figure 6, and 7. A scale size (n = 1, 3) are used in top-hat scale space based hessian vessel enhancement filter and the parameters used for heart vessel segmentation include α=0.3, β=60, c=140 since the vessels give higher response at those scales and parameter. The parameter that were used for anisotropic diffusion for heart vessel segmentation include threshold of 10, 3 iterations and delta_t of 0.02. Different parameters are used because medical images captured by different scan have different features. During post processing, bwareaopen morphological operator was used to remove small objects with pixels below 26. The segmentation results were normalized to the range [0, 255]. Figure 3 show the image before segmentation, removal of the outer tissue surrounding the heart and segmentation of the heart. Figure 4 is a 3D segmentation of the whole heart. Figure 5 shows segmentation results of cerebral aneurysm using Osirix (left), and segmentation results of proposed method (right). Figure 6 shows segmented cerebral aneurysm and associated vessels. Figure 7 shows segmentation results of the cerebral vessels, carotids and aortic arc.

C. Validation and Evaluation
In medical image analysis, it is very hard to validate, qualitatively and quantitatively compare segmentation methods as it requires ground truth to which the segmentation results can be compared. The ground truth is obtained by an expert who manually analyses and segments the medical images. Unfortunately the ground truth data is not available.
In our study, a visual evaluation of segmentation results is necessary to strongly assess the contribution of this method in segmentation of heart vessels

D. Discussions
Based on the above results, the proposed method is capable of dealing with intensity homogeneity and noise. Multithreshold Otsu get rid of unwanted tissue and most of the noise in the image. A white top-hat hessian based vessel enhancement is capable of dealing with the intensity inhomogeneity in images, More vessels were detected and different vessels are separated to enable further analysis. The method is capable of also detecting the tumor vessels and the organs associated with the image as shown in figure 6 and figure 7.
The proposed method is better than osirix method and the model based method in [49] as it is capable of detecting more vessels than osirix however the method works well on the MRA, MRI and CT images.
The above results can be used for diagnosis. Abnormalities (such tortuous and stenosis) in the vessels are evident. The method led to over segmentation in the MRI heart images since the white top-hat scale space hessian based vesselness filter was applied on the original image. The next step is to improve the efficiency of the segmentation method and also to quantify vessel density in order to detect abnormal growth in vessels.

IV. CONCLUSION
A 3D robust segmentation approach is presented to deal with the intensity inhomogeneity and noise in MRI images to enable extraction of vessels. A white top-hat hessian based vessel enhancement is used to deal with the intensity inhomogeneity in medical images to enable extraction of vessels. 2D multithreshold Otsu is extended and improved in order to reduce the processing speed during extraction.
The method is capable of dealing with intensity homogeneity and noise. More vessels were detected and different vessels are separated to enable further analysis. The new method is capable of detecting organs associated with the vessels. The method is also faster and detects different sizes of vessels as compared to 3D segmentation methods. Results are obtained in 260 seconds.
The segmentation method can be used for diagnosis. Abnormalities (such tortuous and stenosis) in the vessels are Cerebral aneurysm evident. The next step is to improve the efficiency of the segmentation method (speed) and also to quantify vessel density in order to detect abnormal growth in vessels.
ACKNOWLEDGEMENT Faiza Bukenya acknowledges the Islamic Development Bank (IDB) Merit scholarship scheme and University of Nottingham for their generous support. We also thank Joseph Ehling who provided the MRI images for this study.