fter a number of false starts, Google and Facebook finally figured out what the rest
of the world already knew: it's impractical for humans to curate the Internet.
Their solution: find a computer algorithm to detect fake news, thus giving themselves
plausible deniability when the result turns out to be blatantly political.
I described here and here why these algorithms can't work, even in principle. To summarize, detecting fake news is equivalent to deciding the truth of an empirical statement. If a computer could do that, I could let my PC do my experiments and I could lounge around drinking piña coladas all day.
Now Jimmy Wales of Wikipedia has a new idea: to call upon the vast collective intelligence of a cluster of carbon units, letting the wisdom of crowds decide what's true and what isn't.
Economists have long held that voting is a good decision process because uninformed votes cancel each other out, but James Surowiecki, author of The Wisdom of Crowds, is often credited for the idea of “wisdom of crowds” (WOC). He said that for a crowd to make good decisions, there must be diversity of opinion, each opinion-holder must formulate a decision independently of the others, and there must be a mechanism for calculating the result.
You can see the problems right away. Even Surowiecki touched on it: herd instinct and peer pressure can sabotage the decision. But there's a more fundamental problem: if the knowledge held by crowds is based on inaccurate (or malicious) information, its collective judgment will also be wrong.
The dubious wisdom of relying on the wisdom of crowds to decide political topics, is, or ought to be, readily apparent, but to see why it fails we only have to look at Wikipedia. Even for uncontroversial scientific questions, it is difficult for crowds to avoid making errors. I occasionally check to see what WP has to say about topics I'm expert on, and I always find errors—sometimes egregious ones.
We'd love for there to be such a thing as a wise crowd. The idea that crowds can be smart appeals to our belief, or hope, that democracy is not simply mob rule. It makes us confident that we can safely follow the herd and not be faced with the difficulty of deciding for ourselves. But a wise crowd is a thing that can never be.
Assumptions in the wisdom of crowds
The basis for the wisdom of crowds comes from statistics. For statistical processes, like measuring radioactivity, averaging many different results increases the sample size and thereby improves the accuracy. Even for physical measurements, like weighing something, multiple measurements = more accuracy. But for this to work, several assumptions must hold:
- There must be some quantitative way of discriminating between good, careful measurements and bad ones.
- The question must be unambiguously defined.
- The problem must be unique, not a composite of two different questions, possibly based on conflicting assumptions.
- There must be one and only one correct answer, to which repeated measurements will converge. This is another way of saying it must be a question of ‘fact’, not opinion.
- All the votes must be honest. If half the measurers have their thumb on the scale, no computational process can eliminate their effect.
Assumption #1 doesn't always hold in science. For a variety of reasons, when comparing measurements from different instruments, a simple mean or even a geometric mean of the votes is inadequate. It's become a hot topic these days to devise new tests, called robust statistical tests, to reduce the impact of wildly wrong answers, which are called outliers.
Assumptions #2, 3, and 4 don't hold for political topics, which (at best) are derivable from individual value systems. At worst, they're derived from group values. Because of groupthink, a group will doggedly adhere to the wrong solution. The bad will drive out the good.
Assumption #5 doesn't hold for fake news.
The WOC theory is based on the assumption that all the participants express honestly held beliefs. This assumption is not valid with so-called fake news. Using it would change truth-seeking into a numbers game. There is no way of weighting votes that is not subjective. Worse, there is no way to ensure that a majority of voters are honest or that the uninformed voters will really cancel each other out. Fake news is all about groupthink Even if people wanted to be fair, they would honestly believe that stories supporting viewpoints that contradict their worldview are fake.
If there is no way to ensure that the contributors are acting in good faith, the WOC theory is invalid. Voting would reduce to using politics to decide facts, which was the very reason the fake news was considered a problem. Wikitribune would be substituting one political judgment for another.
In today's polarized political climate, one man's fake news is another man's gospel. Even if people wanted to be fair, they would honestly believe that news items supporting viewpoints that contradict their worldview are fake. If Wikipedia joins in this groupthink, it would not be part of the solution; it would become part of the problem.
At best it would create another layer of politics between us and the truth. At worst, it would create a problem where none existed before.
created apr 29 2017; last edited apr 29 2017, 8:42 pm
