Extended Data Fig. 2: Shifts of attention scores from unimodal to multimodal model.
From: Prediction of recurrence risk in endometrial cancer with multimodal deep learning
a, Model using only H&E WSI (unimodal) and a corresponding example of the normalized attention scores shown as overlaid on the H&E WSI as a heatmap where red is high attention score and blue low attention score. b, The two-arm model with H&E WSI and image-based molecular class predicted by im4MEC, and a corresponding example of the normalized attention scores shown as overlaid on the H&E WSI. c, The multimodal three-arm HECTOR model with H&E WSI, image-based molecular class, and stage, and a corresponding example of the normalized attention scores shown as overlaid on the H&E WSI. d, Density plot of the normalized attention scores of the heatmap shown in a,b,c for each model. e, Quantitative analysis of the distribution shift between the three models in the internal test set (n = 353 patients) using the WSI-level skewness and median of the normalized attention scores.
