|

Semantic Segmentation for Neutrophils on HUVEC Monolayers: A Machine Learning Approach

ByDan Ahmad

Introduction

Phase-contrast microscopy is widely utilised in live cell imaging, with pixel segmentation protocols in particular finding widespread usage as a means to computationally isolate cells from a tissue-culture background. Computer segmentation allows for the analysis of individual cell properties such as state or spatial location through automated or semi-automated techniques, ultimately attempting to avoid the tedium associated with manual counting. With respect to leukocyte-endothelium interactions, the µSIM physiological systems utilized by our group provides an added layer of complexity to tissue-culture model image analysis due to facilitating neutrophil seeding on a layer of endothelial cells.

Computational segmentation of these images through traditional methods is impractical, if not impossible due to difficulties in accurately separating neutrophils from the endothelial background. Furthermore, traditional computer vision segmentation techniques on grayscale images typically rely on brightness and contrast variations in order to delineate objects from one another. While this is sufficient for neutrophils that are on the luminal side of the µSIM device, any robust probing or transmigration from the neutrophil results in a loss of segmentation capability due to shifts in 8 bit grayscale value that mimic values for the human umbilical vein endothelial cell monolayer (HUVEC) (Figure 1).

Figure 1: Typical approaches to segmentation include binarization, which seeks to threshold pixel values as either “0” or “255” in an 8 bit grayscale configuration, leading to either totally black or totally white pixels respectively. Note, in an 8 bit configuration a grayscale image can have a pixel value between 0 and 255, with 1-254 representing shades of grays. A) For phase bright neutrophils (depicted as blue, HUVECs as white), binarization at a threshold of 0.65 (scale 0-1) is sufficient for separating neutrophils from the HUVEC background. B) When transmigration (darker color) or probing occurs (light gray), neutrophils are lost to the background with a similar binarization threshold.

The inability to computationally isolate a neutrophil as it undergoes transmigration effectively cripples our ability to automatically assess neutrophil state (spatially located either luminal/abluminal) in response to chemokine or cytokine stimulation. Furthermore, individual cell tracking is made difficult by an inability to assess the transmigrated state, necessitating manual tracking. While cell tracking can be enhanced through the usage of fluorescent staining, it is important to note that live cell staining still fails to address bulk assessment of neutrophil state.

To this end, computer vision techniques have steadily improved over the last decade. Semantic segmentation via machine learning presents unique advantages with respect to pixel level segmentation of biological imaging samples. This work utilizes a FIJI implementation of the Waikato Environment for Knowledge Analysis (WEKA).

Materials and Methods

FIJI supports the WEKA segmentation plugin that allows for ground truth labeling of images, feature selection, model training, model generation, and subsequent classification through a headless interface via terminal (or command prompt). All of the image processing performed for this work was done with multiple computers. As WEKA does not support graphics processing unit (GPU) acceleration, central processing units (CPU) were exclusively used for computational processing. The main computer used (CPU/RAM/GPU/OS) is as follows:

Ryzen 9 3950x, 64gb DDR4 RAM, Nvidia/EVGA RTX 2070 Super, Windows 10

Beanshell, ImageJ Macro language, and Batch were utilised for scripting, implementation, and automation of the machine learning process via WEKA. Mathematica via Wolframscript was used for data analysis, video correction (described in: ) and plotting. Ground truth labeling was performed in FIJI, with labeled regions saved via “ROI Manager” (Figure 2). Four classes were established in order to differentiate neutrophils and HUVECs, and images were sourced from brightness corrected frames from all experimental conditions (Pos/Neg control, Luminal/Abluminal TNF-alpha, and fMLP). Neutrophils could be in “phase bright” (bright white), “probing” (gray), or “phase dark” (transmigrated, black) while HUVECs were given one class: “endothelial background”. Emphasis was placed on labeling boundaries between classes in ambiguous cases. The four classes were pixel balanced in order to prevent overfitting for one class and multiple features were utilized in order to enhance the model. These include: Gaussian blurs, Sobel filters, Hessian matrices, Difference of Gaussian, Membrane Projections, Mean, Minimum, Maximum, Variance, Entropy, and Neighbors. The mathematical descriptions of these convolutions are described in the ImageJ wiki: (https://imagej.net/Trainable_Weka_Segmentation.html#Training_features_.282D.29).

Figure 2: An example of the ground truth labeling GUI. Red is for “phase bright”, green is for “probing”, purple is for “phase dark”, and yellow is for “endothelial background”. Neutrophils were given three classes based on distinct grayscale profiles that are evident upon observation. Neutrophils that are at rest appear bright white (hence phase bright), while neutrophils actively proving the membrane surface appear gray. A neutrophil that has transmigrated across the HUVEC monolayer will appear dark.

After labeling, models were trained and saved as both “.arff” files (containing ground truth labels/traces) and a “.model” file that was used for video classification. A beanshell script was adapted from the ImageJ wiki in order to process frames of a video separately in order to mitigate memory issues.

Results

After training, the model reported an error of 1.58%, indicating an overall accuracy of 98.42%. This is tested by applying the trained model on the ground truth labeled images, and assessing pixel detection accuracy against manually labeled pixels. The trained model can then be used on microscopy videos, with an example shown below (Condition: Position Control).

merged_Postive_Control_2_new

Results from a classification procedure include a segmentation map that assigns a pixel one of four colors (correlating to the four classes) based which class has the highest probability of occurring and a probability map of each class (Figure 3).

Figure 3: Results from applying the trained model to a video of our neutrophil transmigration study (condition: Positive Control). The color segmentation map represents the four possible classes and each pixel is assigned a class based on the highest probability of occurrence. Probability maps are also generated, which use 8-bit grayscale values (0 to 255, correlating linearly to P = 0 and P = 1 respectively) to give the probability of a pixel being a certain class. These images are taken from the first frame of the video input. False colors are shown in the “Phase Bright”, “Probing”, and “Phase Dark” probability maps.

Discussion/Conclusion

Machine learning based segmentation via WEKA allows for robust pixel level classification and computational separation of neutrophils from the HUVEC monolayer. The overall accuracy of this model is high, and this facilitates state analysis and tracking (which will be discussed in another post). There are distinct limitations to note, however. First is the spurious detections that can be seen throughout the video. These are more related to aberrations generated during video recording/correction, and may be fixed soon with a migration to a newer microscope. The second limitation to note is that phase dark segmentation is evident, but not as robust as phase bright/probing. This should not limit our ability to analyze state or track cells, and better performance may be possible with implementation of a deep learning based solution called “Unet”. There will be more on this front later on.

Similar Posts