“Hey! I’m Charlotte. I’m here to ask how you feel about AI imagery.” The voice that comes down the phone is crisp and English-accented, a little stiff, like a mediocre actor. Though it makes several human-like sounds to indicate listening (“hmmm”), it is clearly AI. Charlotte is a virtual interviewer created by Prime Radiant, a public opinion research company. I’m on a call with co-founder Jon Nash, who is demonstrating how it works. He has a pleasant back-and-forth with Charlotte about his feelings towards brands using AI-generated models to advertise their clothes. She responds to each of his answers and uses them to tailor the next question.
Virtual interviewers like these are becoming more common in opinion research. More dynamic than a fixed question list, they can prompt respondents to elaborate on opinions. “You can triple the richness of the text data you’re getting out of people,” says Fintan Smith, the co-founder of Convergent Opinion, a polling and research start-up which also uses voice agent surveyors. But some respondents do not take kindly to being questioned by an AI. “You do have people saying: ‘no, I don’t want to talk to a fucking chatbot.’”
In spite of these bursts of resistance, AI is already in every part of the public opinion research process, whether it’s being used to gather the data or analyse it. Survey respondents, too, are using LLMs to speed up their answers. But perhaps the most controversial of AI techniques employed in modern polling is the use of synthetic respondents, also known as silicon sampling. In March, an article in Axios cited a statistic indicating that the majority of people trust their doctors and nurses. The figure in question had come from Aaru, a billion-dollar New York start-up that makes predictions, including for election results and vote turnout, using a simulation. In other words, it gauges the sentiments of AI-generated personas instead of real people.
The statistic’s inclusion in the story prompted a round of criticism, partly because it initially went unlabelled as having originated in a simulated survey. But it also represented a breakthrough moment, in which more people realised that “silicon sampling” is no longer in its experimental phase and is in use across consultancies, businesses and research institutions. As the authors of an op-ed in the New York Times wrote, “it’s suddenly everywhere”.
In politics, campaigns are already using these tools to tailor their gameplans. Aaru – certainly a company with a rather brash communications strategy – boasts that its technology is “used to select candidates for office”. Nash tells me that his company, Prime Radiant, is working on tools for “an upcoming UK mayoral election” that can generate synthetic respondents to help canvassers be more effective. For example, door-knockers could ask what topics to bring up on their rounds and be given street by street suggestions. The responses generated are grounded in “a load of socio-economic data”, he says, and the underlying model can be updated daily in a way that real-life surveys would struggle to match.
Though synthetic respondents offer many opportunities to speed up and lower the cost of research, several people I spoke to working on cutting-edge polling were wary of it. Smith, who was formerly a researcher for YouGov and Labour Together before setting up Convergent Opinion earlier this year, has mixed feelings about silicon sampling. He notes that it could be used for a “fairly bog-standard market research tracker”, but that the approach might fail to account for anything “novel”.
“Imagine it’s a general election; what if the major party leaders decide to dress up in a chicken costume and run around Westminster, smacking people around the face or something bizarre, right? I don’t think an LLM really knows – it has no sort of reference point for how that goes down, what happens.”
A deliberately absurd example, but the point is that not everything can be modelled based on previous events. Another concern is that using the tech in this way further divorces candidates from the people they seek to represent. “If you want to be an effective person in democracy, you should speak to real people.” says James He, the co-founder of one of the buzziest startups in this space, Artificial Societies. A Y Combinator alumnus with several million pounds in funding, the company creates AI personas that mimic real-world attitudes, using an approach the founders say is rooted in behavioural science.
Yet while this form of market research is sought after by large businesses, He has reservations about it being used for political campaigning, and certainly doesn’t think it should replace connecting with voters. “If you want to participate in a democracy, you need to feel the pain and the joy of the people you’re representing.”
The academic picture is similarly mixed, with studies finding promising results from using AI in some instances but shortfalls in others. In one study, led by Lisa P Argyle, the AI respondents replicated even the foibles of human participants. When tasked with writing four words to describe Democrats or Republicans, GPT-3 mostly complied but “much like humans – it occasionally responds with long phrases, mini-essays or nothing at all”.
The results of this test showed a high level of what the authors call “algorithmic fidelity”. The human-like output of language models “stems from human-like underlying concept associations”, they wrote. “This means that given basic human demographic background information, the model exhibits underlying patterns between concepts, ideas, and attitudes that mirror those recorded from humans with matching backgrounds”.
Yet a different study, led by Shibani Santurkar, finds that language model opinions are misaligned with the US populace, and that attempts to refine them can even make things worse. When fine-tuned with human feedback, the results aligned more with liberal, educated and wealthy demographics than the base output. The authors also found certain groups which were poorly represented by all models, including Mormons and widows.
In short, silicon sampling is unlikely to be able to match the accuracy of human surveys, though there may be more of a case for blending the two. “There is, right now, a pretty well-established framework that people can use to increase the power of existing surveys,” says Manoel Horta Ribeiro, an assistant professor at Princeton. “This is something that is done not as a purely silicon survey, but as some sort of combination of synthetic respondents and real respondents. And it’s kind of like magic.”
Magic because it means you can increase a sample size without (theoretically) reducing the accuracy. Horta Ribeiro leads the Humans and Machines Lab at Princeton, and we chat as he is walking his dog, the lab’s official mascot, Cannoli. As part of his work, he teaches a Machine Behaviour course that tries to unpick how well AI tools can simulate and predict human behaviour. It is still an open question, he says – not just in terms of how well the tech works, but whether the public will accept it.
“Now they’re starting to make it into the mainstream, this trust with the public hasn’t been negotiated yet, so I think it’s something very much in the making.” Using silicon sampling internally is one thing. If the results are wrong, the consequences fall on the company that has launched a dud product, or the campaign that loses the election. Where things get sticky is if these methods are used to inform how an incumbent government acts. “In a policy context, there’s very little appetite for anything synthetic because it does still suffer from a lack of credibility,” says Nash. “When you’re in policy, you’re trying to be bulletproof.”
Nevertheless, there seems to be appetite for this kind of tech in government. In Britain, some of the seeds were sown by Dominic Cummings during his tenure at No 10. In a job advert that went viral at the time, he said there were huge opportunities in “the frontiers of the science of prediction” and using superforecasting tournaments to “improve policy and project management”. The Spectator reported last year that Downing Street’s then-chief of staff Morgan McSweeney was experimenting with synthetic voters. Electric Twin, a start-up that creates synthetic audiences, was founded by a former No 10 adviser.
It’s easy to see the attraction of a fully simulated public. Each time the Budget rolls around, every line seems to be leaked in advance for a sense of what the populace, or certain vocal interest groups, will accept. Yet this leads to the impression that the government is making an embarrassing U-turn, even on a policy it has never officially announced. Businesses already use synthetic audiences to test ideas they don’t want leaked, so why not ministers? Advocates also argue that a system like this would help governments tackle the sheer complexity of policymaking, maybe even making it easier to get to grips with the possible consequences. “It is my hope that policymakers would consult an artificial society as part of their stack for considering policy that will affect hundreds of millions of people,” says He. “But naturally, talking to the people who are going to be part of the hundreds of millions of people affected should also be part of the stack.”
Despite the criticisms, this tech is unlikely to fade away – not least because the market for it is hotting up, especially on the commercial side. Some old-school polling companies have also dipped their toes in. YouGov snapped up New Zealand-founded synthetic responses pioneer Yabble back in 2024. Savanta recently launched Virtual Personas, which its clients can use to access on-demand individual personas or focus groups representative of their audiences.
Still, people don’t like to feel manipulated by technology. The Cambridge Analytica scandal showed that. Pollsters, campaign strategists and policymakers will have to tread carefully when dabbling with this tech if they want to avoid a reputational risk.
“I think right now, there’s a typical gut reaction,” says Joseph Wright, co-founder of Semilattice, an AI start-up aiming to simulate complex systems. He adds that the word “synthetic” does the field no favours. “It’s not helpful that the label it’s got means fake.” That’s not a fair association, he argues, given the relative accuracy of many of these models.
Most people I spoke with insisted that human research would never go away, despite the ongoing challenges in official statistics. Those working on this technology repeatedly told me they saw it as filling in gaps, or adding to the field rather than replacing it. In a world where everyone has access to a big polling model, human data also becomes a competitive advantage, as stalwart poll analyst Nate Silver pointed out in his take on the Aaru/Axios debacle: “If I were running a campaign, I’d invest more in going the extra mile to find a representative sample of these voters. And then I’d hire some smart quants – perhaps with help from Claude et al. – to figure out the implications for campaign strategy based on that proprietary data that my competitors didn’t have access to”.
Just as bosses are using their proprietary data to make general AI models more useful to their businesses, political operatives might feed the information they gather on the ground into their database of virtual voters. As for the public perception of the practice, Wright thinks the silicon samplers will just have to continue proving they can be accurate over and over again until the technology is more widely trusted, or at least becomes as normal as the weather forecast. “Everyone notices that when it says it’ll rain, it mostly does.”
[Further reading: The silent coup]
Content from our partners
Related
Join the debate
Subscribe here to comment