Fig. 1: Heatmaps (Layer-wise Relevance Propagation/LRP for convolutional networks and attention rollout for Vision Transformer) for positive COVID-19 and Pneumonia X-rays and photographs, extracted from the synthetically biased test datasets (biased test). | Nature Communications

Fig. 1: Heatmaps (Layer-wise Relevance Propagation/LRP for convolutional networks and attention rollout for Vision Transformer) for positive COVID-19 and Pneumonia X-rays and photographs, extracted from the synthetically biased test datasets (biased test).

From: Improving deep neural network generalization and robustness to background bias via layer-wise relevance propagation optimization

Fig. 1

Last row displays classifier trained without the synthetic bias (and analyzing images without the bias), for reference. The image’s true class is stated above the figures, and the DNN that produced the heatmap is identified on the left. The triangle (background bias) indicates the classes COVID-19, smiling or Pug. The circle pneumonia, high cheekbones, and Tibetan Mastiff. The square rosy cheeks and Pekingese. Red colors in the LRP maps indicate areas the DNN associated to the image’s true class, while blue colors are areas that reduced the network confidence for the class. For attention rollout, red shows the DNN attention. White represents areas with little influence over the classifiers. DNN focus on the images’ foregrounds (dogs, faces, or lungs), which results in whiter heatmap backgrounds, is desirable. For privacy, the face picture was substituted by a representation of the face (gray) and bias (white) locations, but classifiers received the real picture.

Back to article page