Table 2 Inter-rater agreement for annotations of the EAR dataset (Study 2).

From: The paucity of morality in everyday talk

 

Sample 2A

Sample 2B

Sample 2C

Sample 2D

All Samples

Instances

17,307

9824

16,874

6953

50,961

Annotators

4

4

4

3

10

Label

Prevalence-adjusted bias-adjusted Kappas

Moral

0.73

0.75

0.85

0.91

0.80

Care

0.92

0.96

0.96

0.97

0.95

Harm

0.96

0.98

0.97

0.99

0.97

Fairness

0.98

0.96

0.99

1.00

0.98

Cheating

0.97

0.95

0.99

0.99

0.98

Loyalty

0.99

1.00

0.97

0.99

0.99

Betrayal

0.99

0.99

0.99

1.00

0.99

Authority

0.95

0.98

0.98

0.98

0.97

Subversion

1.00

1.00

1.00

1.00

1.00

Purity

0.97

0.98

0.99

0.99

0.98

Degradation

0.90

0.92

0.98

0.99

0.94