Table 1 Accuracy and bias in the training data and CNN classifications

From: How human–AI feedback loops alter human perceptual, emotional and social judgements

Labels

Objective ranking

(accuracy = 100%;

bias = 0%)

Objective ranking + minor bias

(accuracy = 97%;

bias = 3%)

Participant classifications

(accuracy = 63%;

bias = 3%)

Random labels + minor bias

(accuracy = 50%;

bias = 3%)

Accuracy − objective labels

96%

94%

66%

50%

Accuracy – training labels

96%

92%

69%

53%

Bias

1%

3%

15%

50%

  1. Training was conducted using four different label sets: (1) objective (based on morphing ranking scores); (2) objective with a 3% bias; (3) participant classifications; and (4) random labels with a 3% bias. The predictions of the model were assessed on an out-of-sample test set of 300 arrays. Accuracy and bias were evaluated with respect to the objective labels and with respect to the labels the models were trained on (training labels).
Back to article page