Can a dating site tell if you're a secret racist?

How the OkCupid website, started by four Harvard geeks, used statistics to unearth its users’ secret

When it comes to love and sex, how do you find out what people really feel, rather than what they say they do? Well, there are easier ways, but you could always start a dating website. Every time a user responds to a message, or clicks on a profile, she is telling you who she finds attractive, and who she's interested in starting a relationship with. Multiply that by a million or more and you have one hell of a database to plunder for insights.

That's what OkCupid did. Four Harvard graduates - Chris Coyne, Max Krohn, Christian Rudder and Sam Yagan - started the dating website in 2004. They'd previously run thespark.com, which offered personality quizzes such as the Purity Test (sample questions: "Have you ever fantasised about a family member? Have you ever fantasised about your own member?"), and decided to take the same slant with their dating venture. OkCupid users answer some, lots or occasionally all of the 4,000 questions written by the site or submitted by others about what they are looking for in a partner. They also give feedback on how useful the question was and how much weight they would give to the answer when setting up a date. That allows every candidate to create their own algorithm - the mathematical "secret sauce" that finds you matches.

While the site users were clicking away, trying to find someone to go for a drink with on Friday night, the founders were busy crunching their data. In June 2009 the first blog post popped up on OkTrends. "Since we went online in 2004, we've collected an enormous amount of data on human interactions," it said. "This blog was started as a way to share some of the things we've learned about people."

And boy, did they discover some interesting things. Some findings were quirky: that users of both genders added two inches on average to their height - even though shorter women got more messages. Or that using ur instead of you're or your in a first approach shrank the reply rate from 32 per cent to around 6 per cent.

In October 2009 a post titled "How Your Race Affects the Messages You Get" appeared. "We've processed the messaging habits of over a million people and are about to basically prove that, despite what you might've heard from the Obama campaign and organic cereal commercials, racism is alive and well," Rudder wrote. "It would be awesome if the other major online dating players would go out on a limb and release their own race data, too. I can't imagine they will: multimillion-dollar enterprises rarely like to admit the people paying them those millions act like turds."

Rudder showed that the percentages of matches were roughly even across all races. But white men got the most responses from almost all ethnic groups; white, Asian and Hispanic women preferred them to the exclusion of everyone else. Black women, on the other hand, get a bum deal - even though they reply more often than any other group to messages from every race, including their own, their messages get by far the fewest replies. "Essentially every race - including other blacks - singles them out for the cold shoulder," Rudder wrote.

He contrasted users' actions with their words: only 6 per cent overall said that interracial marriage was a bad idea, and 38 per cent that they would "strongly prefer" to date someone of their own racial background. (Among white users it was 45 per cent and among non-whites, 20.)

After that, the blog tackled such taboos as rape fantasies (these are deemed much more acceptable in Nevada than in New England, and in Lithuania than actual England) and even questioned the extent to which bisexuality exists. Noting that 80 per cent of self-identified bisexuals were interested in only one gender, Rudder concluded: "This suggests that bisexuality is often either a hedge for gay people or a label adopted by straights to appear more sexually adventurous to their (straight) matches."

Strange combinations

Last April, however, postings on the blog abruptly stopped. What happened? As the OkCupid number-crunchers would say, correlation does not imply causation, but it's hard to feel it wasn't anything to do with the site being acquired for $50m in February by its paid-for rival match.com. Was it intending to suffocate its free competitor? When I asked Yagan, the OkCupid chief executive, he said the blog would return but refused to be drawn further.

Then again, running a dating site exposes you to bits of humanity that maybe are best hidden. OkCupid was rare in making its data public, but our hidden prejudices and preferences are clearly well known to those in the industry. When I emailed Markus Frind, founder of Plenty of Fish, to ask about his rivals at OkCupid he said his matching system was better, because "we look at trends or patterns in couples . . . A female doctor is never going to date a carpenter. There are many, many combinations of relationships that will never happen or are very unstable." And he had the data to prove it.

What OkCupid showed was that, when it comes to choosing our partners, none of us is as progressive as we think. Perhaps the world is a better place for not revealing our deepest, darkest secrets?

True love: A neon sign by artist Chris Bracey. Photograph: Getty Images

Helen Lewis is deputy editor of the New Statesman. She has presented BBC Radio 4’s Week in Westminster and is a regular panellist on BBC1’s Sunday Politics.

This article first appeared in the 12 March 2012 issue of the New Statesman, The weaker sex

Getty Images.
Show Hide image

David Davis interview: The next Conservative leader will be someone nobody expects

The man David Cameron beat on why we should bet on a surprise candidate and what the PM needs to do after the referendum. 

“I’m tired,” says David Davis when I greet him. The former Conservative leadership candidate is running on three hours’ sleep after a Question Time appearance the night before. He is cheered, however, by the coverage of his exchange with Ed Miliband. “Which country would it be be like?” the former Labour leader asked of a post-EU UK. “The country we’re going to be like is Great Britain,” the pro-Brexit Davis retorted

The 67-year-old Haltemprice and Howden MP is at Hull University to debate constituency neighbour Alan Johnson, the head of the Labour In campaign. “As far as you can tell, it’s near to a dead heat,” Davis said of the referendum. “I think the run of events will favour Brexit but if I had to bet your salary, I wouldn’t bet mine, I’d place it on a very narrow victory for Brexit.”

Most economists differ only on how much harm a Leave vote would do. Does Davis believe withdrawal is justified even if it reduces growth? “Well, I think that’s a hypothetical question based on something that’s not going to happen ... One of the arguments for Brexit is that it will actually improve our longer-run economic position. In the short-run, I think Stuart Rose, the head of Remain, had a point when he said there would be very small challenges. In a few years probably nothing.

“The most immediate thing would likely be wage increases at the bottom end, which is very important. The people in my view who suffer from the immigration issue are those at the bottom of society, the working poor, which is why I bridle when people ‘oh, it’s a racist issue’. It’s not, it’s about people’s lives.”

More than a decade has passed since David Cameron defeated Davis by 68-32 in the 2005 Conservative leadership contest. The referendum has pitted the two men against each other once more. I asked Davis whether he agreed with the prime minister’s former strategist, Steve Hilton, that Cameron would be a Brexiter were he not in No.10.

“I think it might be true, I think it might be. When you are in that position you’re surrounded by lot of people: there’s the political establishment, the Whitehall establishment, the business establishment, most of who, in economic parlance, have a ‘sunk cost’ in the current set-up. If changes they stand to lose things rather than gain things, or that’s how they see it.

“Take big business. Big business typically gets markets on the continent, maybe distribution networks, supply networks. They’re going to think they’re all at risk and they’re not going to see the big opportunities that exist in terms of new markets in Brazil, new markets in China and so on, they’re naturally very small-C Conservative. Whitehall the same but for different reasons. If you’re a fast-track civil servant probably part of your career will be through the Commission or maybe the end of your career. Certainly in the Foreign Office. When I ran the European Union department in the Foreign Office, everybody wanted a job on the continent somewhere. They were all slanted that way. If all your advice comes from people like that, that’s what happens.”

Davis told me that he did not believe a vote to Leave would force Cameron’s resignation. “If it’s Brexit and he is sensible and appoints somebody who is clearly not in his little group but who is well-equipped to run the Brexit negotiations and has basically got a free hand, there’s an argument to say stability at home is an important part of making it work.”

He added: “I think in some senses the narrow Remain is more difficult for him than the narrow Brexit. You may get resentment. It’s hard to make a call about people’s emotional judgements under those circumstances.”

As a former leadership frontrunner, Davis avoids easy predictions about the coming contest. Indeed, he believes the victor will be a candidate few expect. “If it’s in a couple of years that’s quite a long time. The half life of people’s memories in this business ... The truth of the matter is, we almost certainly don’t know who the next Tory leader is. The old story I tell is nobody saw Thatcher coming a year in advance, nobody saw Major coming a year in advance, nobody saw Hague coming a year in advance, nobody saw Cameron coming a year in advance.

“Why should we know two years in advance who it’s going to be? The odds are that it’ll be a Brexiter but it’s not impossible the other way.”

Does Davis, like many of his colleagues, believe that Boris Johnson is having a bad war? “The polls say no, the polls say his standing has gone up. That being said, he’s had few scrapes but then Boris always has scrapes. One of the natures of Boris is that he’s a little bit teflon.”

He added: “One thing about Boris is that he attracts the cameras and he attracts the crowds ... What he says when the crowd gets there almost doesn’t matter.”

Of Johnson’s comparison of the EU to Hitler, he said: “Well, if you read it it’s not quite as stern as the headline. It’s always a hazardous thing to do in politics. I think the point he was trying to make is that there’s a long-running set of serial attempts to try and unify Europe not always by what you might term civilised methods. It would be perfectly possible for a German audience to turn that argument on its head and say isn’t it better whether we do it this way.”

Davis rejected the view that George Osborne’s leadership hopes were over (“it’s never all over”) but added: “Under modern turbulent conditions, with pressure for austerity and so on, the simple truth is being a chancellor is quite a chancy business ... The kindest thing for Dave to do to George would be to move him on and give him a bit of time away from the dangerous front.”

He suggested that it was wrong to assume the leadership contest would be viewed through the prism of the EU. “In two years’ time this may all be wholly irrelevant - and probably will be. We’ll be on to some other big subject. It’’ll be terrorism or foreign wars or a world financial crash, which I think is on the cards.”

One of those spoken of as a dark horse candidate is Dominic Raab, the pro-Brexit justice minister and Davis’s former chief of staff. “You know what, if I want to kill somebody’s chances the thing I would do is talk them up right now, so forgive me if I pass on that question,” Davis diplomatically replied. “The reason people come out at the last minute in these battles is that if you come out early you acquire enemies and rivals. Talking someone up today is not a friendly thing to do.” But Davis went on to note: “They’re a few out there: you’ve got Priti [Patel], you’ve got Andrea [Leadsom]”.

Since resigning as shadow home secretary in 2008 in order to fight a by-election over the issue of 42-day detention, Davis has earned renown as one of parliament’s most redoubtable defenders of civil liberties. He was also, as he proudly reminded me, one of just two Tory MPs to originally vote against tax credit cuts (a record of rebellion that also includes tuition fees, capital gains tax, child benefit cuts, House of Lords reform, boundary changes and Syria).

Davis warned that that any attempt to withdraw the UK from the European Convention on Human Rights would be defeated by himself and “a dozen” other Conservatives (a group known as the “Runnymede Tories” after the meadow where Magna Carta was sealed).

“They’ve promised to consult on it [a British Bill of Rights], rather than bring it back. The reason they did that is because it’s incredibly difficult. They’ve got a conundrum: if they make it non-compliant with the ECHR, it won’t last and some of us will vote against it.

“If they make it compliant with the ECHR it is in essence a rebranding exercise, it’s not really a change. I’d go along with that ... But the idea of a significant change is very difficult to pull off. Dominic Raab, who is working on this, is a very clever man. I would say that, wouldn’t I? But I think even his brain will be tested by finding the eye of the needle to go through.”

Davis is hopeful of winning a case before the European Court of Justice challenging the legality of the bulk retention of communications data. “It’s a court case, court cases have a random element to them. But I think we’ve got a very strong case. It was quite funny theatre when the ECJ met in Luxembourg, an individual vs. 15 governments, very symbolic. But I didn’t think any of the governments made good arguments. I’m lucky I had a very good QC. Our argument was pretty simple: if you have bulk data collected universally you’ve absolutely got to have an incredibly independent and tough authority confirming this. I would be surprised if the ECJ doesn’t find in my favour and that will have big implications for the IP [Investigatory Powers] bill.”

Davis launched the legal challenge in collaboration with Labour’s deputy leader Tom Watson. He has also campaigned alongside Jeremy Corbyn, last year travelling to Washington D.C. with him to campaign successfully for the release of Shaker Aamer, the final Briton to be held in Guantanamo Bay.

“I like Jeremy,” Davis told me, “but the long and the short of it is that not having been on the frontbench at all shows. I’m not even sure that Jeremy wanted to win the thing. He’s never been at the Despatch Box. He’s up against a PM who’s pretty good at it and who’s been there for quite a long time. He’s playing out of his division at the moment. Now, he may get better. But he’s also got an incredibly schismatic party behind him, nearly all of his own MPs didn’t vote for him. We had a situation a bit like that with Iain Duncan Smith. Because we’re a party given to regicide he didn’t survive it. Because the Labour Party’s not so given to regicide and because he’d be re-elected under the system he can survive it.”

At the close of our conversation, I returned to the subject of the EU, asking Davis what Cameron needed to do to pacify his opponents in the event of a narrow Remain vote.

“He probably needs to open the government up a bit, bring in more people. He can’t take a vengeful attitude, it’s got to be a heal and mend process and that may involve bringing in some of the Brexiters into the system and perhaps recognising that, if it’s a very narrow outcome, half of the population are worried about our status. If I was his policy adviser I’d say it’s time to go back and have another go at reform.”

Davis believes that the UK should demand a “permanent opt-out” from EU laws “both because occasionally we’ll use it but also because it will make the [European] Commission more sensitive to the interests of individual member states. That’s the fundamental constitutional issue that I would go for.”

He ended with some rare praise for the man who denied him the crown.

“The thing about David Cameron, one of the great virtues of his premiership, is that he faces up to problems and deals with them. Sometimes he gets teased for doing too many U-turns - but that does at least indicate that he’s listening.”

George Eaton is political editor of the New Statesman.