Fig. 3: Development and validation of DL systems for MSI status recognition.

a Overview of the MSI status recognition process by the DL system. Left to right, extracting tumor tiles from WSI, tile-level MSS/MSI classification, fusion of tile-level MSI results, and patient-level MSI status diagnosis. b Pie graph of patient cohorts for MSI status recognition. MSS and MSI denote the patients belonging to the MSS and MSI classes, respectively. There are 436 patients in total. c Pie graph of tumor tiles for MSI status recognition. MSS and MSI denote the tumor tiles belonging to the MSS and MSI classes, respectively. There are approximately three hundred thousand tumor tiles in total. d ROC curve plot for tile-level MSI recognition in testing cohorts. e Confusion matrix plot for patient-level MSI recognition in testing cohorts. The performance of the two methods are shown. Left panel, raw classifier comparing the ratio of MSI tiles to the threshold of 50%. Right panel, optimal classifier by comparing the ratio of MSI tiles to the threshold of 30.7%. f Gradient-weighted class activation map for MSS. Upper, original HE staining tiles for MSS tumor tiles. Down, class activation maps for the HE staining tiles. Hot regions correspond to key features for DL models to recognize MSS. The red and blue colors indicate greater importance and less importance, respectively. g Gradient-weighted class activation map for MSI. Hot regions correspond to key features for DL models to recognize MSI.