Hilbert curve
The Hilbert curve, H(t), or Hilbert space-filling curve, is a continuous fractal space-filling curve [22], a surjective mapping from the interval of the real number [0, 1] to the plane of the real number [0, 1] × [0, 1]. That is, given a point (x0, y0) on the plane unit square, the parameter t0 can be found using H(t) as follows:
$$H\left( {t_{0} } \right) = \left( {x_{0} , y_{0} } \right)$$
An illustration of the Hilbert curve with levels 1 to 8 is presented in Fig. 1.
The original Hilbert curve provided a mapping between one-dimensional (1D) and 2D space, preserving the locality fairly well [23]; when traversing 2D pixels by the Hilbert curve, pixels adjacent to a certain pixel in 2D space were in close proximity to that pixel in the corresponding 1D space (Fig. 2).
Because of the locality property, the Hilbert curve effectively reduces dimensionality [24], which is the description of the information in the N-dimensional space using the (N-1)-dimensional space.
The 2D-Hilbert curve provides a mapping in which all the 2D pixels are expanded into a 1D space (which can be stretched into a straight line); each pixel’s locality with its 2D neighbors is preserved after expansion [25]. As shown in Fig. 3a, an image with four pixels from P0 to P1 could be traversed by a level 1 Hilbert curve by expanding the image with a width of two and a height of two to a 1D space with four points. The level 2 Hilbert curve enables filling a 16-pixel image, transforming a 2D image with a width of four and a height of four to a 1D space with 16 points (Fig. 3b). As the Hilbert curve iteration increases, the size of the image that can be filled increases correspondingly. As the iteration approaches infinity (that is, a 2D plane with an infinite number of pixels), the space will be filled by the Hilbert curve.
Further, an object in 3D space, expressed by a 3D Hilbert curve, could be expanded to 2D space, and the neighboring properties of spatially adjacent voxels would be maintained on the 2D image (Fig. 3c).
Therefore, in image analysis, with the help of a 3D Hilbert curve, current intra-tumoral heterogeneity analysis techniques could be employed on 2D images to interpret information inherent in the 3D volume by characterizing the spatial correlation into 2D images. When using a 3D Hilbert curve for dimensionality reduction, local voxel adjacency in 3D space is well preserved on the corresponding 2D image after Hilbert expansion, as shown in Fig. 3c.
Dimensionality reduction for voxel expansion
CT scans of patients from the open-access Lung Image Database Consortium image collection (LIDC-IDRI) database [26] were used in this study for dimensionality reduction. Manual segmentation of the lung tumor was performed by one to four radiologists, and the intersection of the radiologists was used. One of the CT scans including a lung tumor spreading across 43,068 voxels in 31 slices in total, was used to illustrate the procedure. Informed consent was not required for the data, and the lung tumor is presented in Fig. 4a.
In this study, a level 6 3D Hilbert curve was defined to store the lung tumor voxels, which we called the Hilbert volume, to reduce the 3D tumor volume dimensionality to a 2D matrix. The size of the Hilbert volume was \(\left( {2^{6} , 2^{6} , 2^{6} } \right)\). We rescaled the tumor’s maximum diameter to \(< 2^{6}\) on the axial, sagittal, and coronal planes, respectively. The real gray intensity of the tumor voxels on CT was used in the Hilbert volume defined, and the others were marked as zero.
Next, a 2D Hilbert curve with the same number of points as the 3D Hilbert volume (Fig. 4b) was defined to store all the pixels transformed from the 3D Hilbert volume voxels, which we called the Hilbert matrix. The size of the Hilbert matrix was \(2^{9} \times 2^{9}\) as shown in Fig. 4c.
Then, we simulated reducing the 3D space to 2D space. During dimensionality reduction, each image layer on the cross-sectional axis was sequentially input into the Hilbert transformation. Slice layers pulled from the 3D Hilbert volume were pushed to the corresponding positions on the 2D Hilbert matrix where the previous slice was located to demonstrate the dimensionality reduction vividly; the points of the previous slice were moved outward step-wise along the continuous fractal space-filling curve. This process was iterated until all image layers were pushed to the Hilbert matrix. Finally, the Hilbert matrix was filled in, and all voxels fell on the corresponding pixel position, as defined by the Hilbert curve.
To demonstrate the spatial locality after the Hilbert curve-based voxel expansion, the following experiments were performed:
-
1.
Three Hilbert volumes with level 6 were constructed, consisting of only a single image on the axial, coronal, and sagittal planes; other voxels were marked as zero, as shown in Fig. 5a. The Hilbert volumes, containing only one image on the traditional planes, were used to illustrate the expansion results of the proposed Hilbert curve-based spatial correspondence mapping approach for the single slice on the traditional planes. All three Hilbert volumes were then expanded to the corresponding Hilbert matrices.
-
2.
A Hilbert volume with level 6 consisting of four 3D blocks with sizes of \(\left( {2^{4} , 2^{4} , 2^{4} } \right)\) was constructed; other voxels were marked as zero, as shown in Fig. 6a. The Hilbert volume constructed here was used to clarify the difference of the expansion results by the proposed approach between the 3D blocks and the slices. The location of the blocks varied inside the volume.
-
3.
A Hilbert volume with level 6 with a lung tumor inside was expanded into the Hilbert matrix, as shown in Fig. 6c. The Hilbert volume constructed with the lung tumor was used to indicate the intra-tumor spatial heterogeneity decoded by the proposed Hilbert curve-based spatial correspondence mapping approach.
-
4.
According to the latest LIDC-IDRI nodule list released [27], all the 2635 lung nodules, including 14,266 CT images, were used to evaluate the performance of the Hilbert matrix image on the task of classification into benign or malignant masses. The LIDC-IDRI dataset was used because all the nodules were diagnosed by at least one radiologist, and scoring as malignant or benign was provided by the radiologists. In addition, all the lung nodules were manually delineated by at least one radiologist. The averaged malignancy rating for each nodule and the intersection of the segmentation of each nodule from the radiologists were used in this study. As described by a previous study [28], nodules with an average score < 3 were classified as benign; those with an average score > 3 as malignant. A state-of-the-art network for classification, named Inception-V4 [29], was used to test both the Hilbert matrix images of lung nodules and the original lung nodule CT images. Lung nodules with a minimum diameter of 5 mm were used and divided into training, validation, and test datasets (80%:10%:10%); the difference of classification accuracy was evaluated by the McNemar's test. Furthermore, to validate the robustness of the proposed approach, we used the manual nodule segmentation in the test dataset only performed by Radiologist 1 of the LIDC-IDRI dataset. The Hilbert curve-based spatial correspondence mapping was implemented to the segmented nodules, and the Hilbert matrix images were obtained. All the images were then input into the well-trained Inception-V4 model to verify the potential bias caused by segmentation.
The source code for Hilbert curve-based spatial correspondence mapping in this study is publicly available at https://github.com/JD910/HilbCurv_Spatial_Heterogeneity. Hilbert volume data for this study, consisting of axial plane images, 3D blocks, and the illustrated lung tumor, are also available at the above repository to facilitate the reproduction of our results. Appropriate institutional review board approval was obtained for this study.