From Tournesol
Jump to navigation Jump to search

Crowdsourcing is the idea of relying on a large crowd to solve tasks.

Wisdom of the crowds

The wisdom of the crowds is a curious phenomenon where the aggregation of crowds' estimates can be remarkably accurate for certain tasks.

However, the crowds may have highly unreliable aggregate opinions on some topics.§

The collaboration of a homogenous crowd has been found to sometimes lead to group polarization.

BeckerPC-19 shows limited group polarization when judging verifiable facts. It is noteworthy, however, that their study has a small sample (35 participants per trial) and incentivizes accuracy with financial rewards. Such results may not generalize to settings without financial rewards, and wher§e the claims to assess are less factual or verifiable.

Bias from experts

BhuiyanZSM-20 report a bias between crowd and expert ratings, when judging the credibility of a sample of 50 news article on climate science. Interestingly, there are also biases between expert journalists and scientific experts. Rater demographics and political leaning are shown to influence the ratings. However, given the sample size and the specificity of rated news, the generalizability of the study may be limited.