Viral hit: we all suffer from an inbuilt psychological bug, exacerbated by the internet. Photo: Marcelo Graciolli on Flickr, via Creative Commons
Show Hide image

Omniscience bias: how the internet makes us think we already know everything

The internet is an answer machine, it doesn’t help us ask better questions. It feeds the illusion that we already know everything we need to know to be well-informed.

Last week, Washington DC was hit by an earthquake. The Republican congressional leader Eric Cantor lost his seat to a Tea Party upstart with the suggestive name of David Brat. This wasn’t just a surprise: it was a shock. Nobody saw it coming – not even Nate Silver.

The political press quickly concluded that Cantor had committed the ultimate political sin of losing touch with the voters. Spending his days and nights in the unreal world of the nation’s capital, absorbed in the high politics of the Capitol, Cantor had forgotten about the people who put him there.

No doubt this is true. But as David Carr, the media correspondent of the New York Times, suggests, certain members of the press might want to get an appointment with an optician to see about that log in their eye.

Carr points out that the only journalists who got even a sniff of the trouble that Cantor was in were from local newspapers. Jim McConnell is a staff reporter at the Chesterfield Reporter, which serves the district in question. He didn’t call Brat’s victory, but he did predict it was going to be very close at a time when everyone assumed Cantor was a shoo-in.

He was able to do this by employing the sophisticated journalistic technique of leaving the office and talking to people. “You could tell wherever you went that Cantor was incredibly unpopular, that people saw him as arrogant,” he told Carr. Meanwhile, members of more prestigious and well-funded national newspapers completely missed the big story about to explode in a district less than two hours drive from Capitol Hill.

Carr blames the internet, at least in part. The web is a tremendous boon to reporters: the world’s information is now accessible from a desk or smartphone. But it can also seduce journalists into thinking that they know everything worth knowing. As Carr puts it, “the always-on data stream is hypnotic, giving us the illusion of omniscience.”

Take another story that seemed to come out of the blue: the current violence in Iraq. There’s no shortage of pundits pronouncing with impressive confidence on its causes and ramifications. The real experts tend to be more cautious; they know how little we know about ISIS and its aims. They may have also have been left wondering why editors only got interested in this story once pictures started to show up in their Twitter streams.

Actually, I think Carr puts his finger on something with implications far beyond the media. We all suffer from an inbuilt psychological bug, which is exacerbated by the internet. Call it “omniscience bias”: the illusion that we know everything we need to.

In 1987, researchers at the University of Oklahoma ran an experiment in which they gave students a series of problems to solve, and asked them to generate as many solutions as they could. The researchers deliberately gave their subjects a very limited amount of information on each problem. One problem was how to provide enough parking spaces on the university campus, given the limited space available. The students came up with different solutions, including reducing demand for parking space by raising fees or using the space more efficiently.

After the students had generated their answers they were asked to estimate what percentage of possible good solutions they thought they had come up with, while, separately, a panel of experts were asked to compile a database of the possible solutions. It turned out that the average participant generated only about one in three of the best solutions – yet when asked, participants guessed that they had landed on three out of four possible solutions. Not only had they missed most of the best ideas, but they found it hard to imagine there were many alternatives they hadn’t covered.

Psychologists have replicated this or similar effects in different ways: we tend to be over-confident that we have the right information we need to form opinions or make judgements. The modern internet feeds this tendency by persuading you that everything you need to know is a click away or coming soon from a feed near you. Google never says, “I don’t know.” It is an answer machine, but it doesn’t help us ask better questions.

Even those paid to be intellectual explorers are can be stymied by the apparent certainties of the web. James Evans, a sociologist at the University of Chicago, assembled a database of 34 million scholarly articles published between 1945 and 2005. He analysed the citations included in the articles to see if patterns of research have changed as journals shifted from print to online.

His working assumption was that he would find a more diverse set of citations, as scholars used the web to broaden the scope of their research. Instead, he found that as journals moved online, scholars actually cited fewer articles than they had before. A broadening of available information had led to “a narrowing of science and scholarship”.

Explaining his finding, Evans noted that Google has a ratchet effect, making popular articles even more popular, thus quickly establishing and reinforcing a consensus about what’s important and what isn’t. Furthermore, the efficiency of hyperlinks means researchers bypass many of the “marginally related articles” print researchers would routinely stumble upon as they flipped the pages of a printed journal or book. Online research is faster and more predictable than library research, but precisely because of this it can have the effect of shrinking the scope of investigation.

According to the psychologist Daniel Kahneman, “our comforting conviction the world makes sense rests on a secure foundation: our almost unlimited ability to ignore our ignorance.” It’s never been easier to go through life assuming you know everything you need to know. But that leaves you more vulnerable to information earthquakes. Just ask Eric Cantor.

Ian Leslie is the author of Curious: The Desire to Know and Why Your Future Depends on It (Quercus, £10.99)

 

Ian Leslie is a writer, author of CURIOUS: The Desire to Know and Why Your Future Depends On It, and writer/presenter of BBC R4's Before They Were Famous.

Getty
Show Hide image

A quote-by-quote analysis of how little Jeremy Hunt understands technology

Can social media giants really implement the health secretary’s sexting suggestions? 

In today’s “Did we do something wrong? No, it was social media” news, Health Secretary Jeremy Hunt has argued that technology companies need to do more to prevent sexting and cyber-bullying.

Hunt, whose job it is to help reduce the teenage suicide rate, argued that the onus for reducing the teenage suicide rate should fall on social media companies such as Facebook and Twitter.

Giving evidence to the Commons Health Committee on suicide prevention, Hunt said: “I think social media companies need to step up to the plate and show us how they can be the solution to the issue of mental ill health amongst teenagers, and not the cause of the problem.”

Pause for screaming and/or tearing out of hair.

Don’t worry though; Hunt wasn’t simply trying to pass the buck, despite the committee suggesting he direct more resources to suicide prevention, as he offered extremely well-thought out technological solutions that are in no way inferior to providing better sex education for children. Here’s a quote-by-quote analysis of just how technologically savvy Hunt is.

***

“I just ask myself the simple question as to why it is that you can’t prevent the texting of sexually explicit images by people under the age of 18…”

Here’s Hunt asking himself a question that he should be asking the actual experts, which is in no way a waste of anybody’s time at all.

“… If that’s a lock that parents choose to put on a mobile phone contract…”

A lock! But of course. But what should we lock, Jeremy? Should teenager’s phones come with a ban on all social media apps, and for good measure, a block on the use of the camera app itself? It’s hard to see how this would lead to the use of dubious applications that have significantly less security than giants such as Facebook and Snapchat. Well done.

“Because there is technology that can identify sexually explicit pictures and prevent it being transmitted.”

Erm, is there? Image recognition technology does exist, but it’s incredibly complex and expensive, and companies often rely on other information (such as URLs, tags, and hashes) to filter out and identify explicit images. In addition, social media sites like Facebook rely on their users to click the button that identifies an image as an abuse of their guidelines, and then have a human team that look through reported images. The technology is simply unable to identify individual and unique images that teenagers take of their own bodies, and the idea of a human team tackling the job is preposterous. 

But suppose the technology did exist that could flawlessly scan a picture for fleshy bits and bobs? As a tool to prevent sexting, this still is extremely flawed. What if two teens were trying to message one another Titian’s Venus for art or history class? In September, Facebook itself was forced to U-turn after removing the historical “napalm girl” photo from the site.

As for the second part of Jezza’s suggestion, if you can’t identify it, you can’t block it. Facebook Messenger already blocks you from sending pornographic links, but this again relies on analysis of the URLs rather than the content within them. Other messaging services, such as Whatsapp, offer end-to-end encryption (EE2E), meaning – most likely to Hunt’s chagrin – the messages sent on them are not stored nor easily accessed by the government.

“I ask myself why we can’t identify cyberbullying when it happens on social media platforms by word pattern recognition, and then prevent it happening.”

Jeremy, Jeremy, Jeremy, Jeremy, can’t you spot your problem yet? You’ve got to stop asking yourself!

There is simply no algorithm yet intelligent enough to identify bullying language. Why? Because we call our best mate “dickhead” and our worst enemy “pal”. Human language and meaning is infinitely complex, and scanning for certain words would almost definitely lead to false positives. As Labour MP Thangam Debbonaire famously learned this year, even humans can’t always identify whether language is offensive, so what chance does an algorithm stand?

(Side note: It is also amusing to imagine that Hunt could even begin to keep up with teenage slang in this scenario.)

Many also argue that because social media sites can remove copyrighted files efficiently, they should get better at removing abusive language. This is a flawed argument because it is easy to search for a specific file (copyright holders will often send social media giants hashed files which they can then search for on their databases) whereas (for the reasons outlined above) it is exceptionally difficult for algorithms to accurately identify the true meaning of language.

“I think there are a lot of things where social media companies could put options in their software that could reduce the risks associated with social media, and I do think that is something which they should actively pursue in a way that hasn’t happened to date.”

Leaving aside the fact that social media companies constantly come up with solutions for these problems, Hunt has left us with the burning question of whether any of this is even desirable at all.

Why should he prevent under-18s from sexting when the age of consent in the UK is 16? Where has this sudden moral panic about pornography come from? Are the government laying the ground for mass censorship? If two consenting teenagers want to send each other these aubergine emoji a couple of times a week, why should we stop them? Is it not up to parents, rather than the government, to survey and supervise their children’s online activities? Would education, with all of this in mind, not be the better option? Won't somebody please think of the children? 

“There is a lot of evidence that the technology industry, if they put their mind to it, can do really smart things.

Alas, if only we could say the same for you Mr Hunt.

Amelia Tait is a technology and digital culture writer at the New Statesman.