Lesion Classification and Segmentation from Multispectral Images

Problem: H&E histology is the tried and true method for diagnosing cancer and differentiating different types. But an increasing number of alternative modalities are being developed to image tissue. Some don’t require staining, some don’t need slides, and some can even be done in vivo.

Solution: This project involved guiding the development of a tissue classification model using a pathologist’s assessment of H&E as ground truth. The ground truth labels enabled training a weakly supervised model. The multispectral perspective can provide additional information but made transfer learning more challenging. A variety of ways for integrating the different channels were evaluated. Regardless of the machine learning approach, good study design was an essential starting point to ensure that all models developed will generalize to lesions from different patients.