Show Hide image

Now you see it, now you don't: what optical illusions tell us about our brains

Illusions can offer insights into how the visual system processes images.

Maurits Escher: where do the staircases lead?

The human brain is a network of about 20 billion neurons – nerve cells – linked by several trillion connections. Not to mention glial cells, which scientists used to think were inactive scaffolding, but increasingly view as an essential part of how the brain works. Our brains give us movement, language, senses, memories, consciousness and personality. We know a lot more about the brain than we used to, but it still seems far too complicated for human understanding.

Fortunately, the brain contains many small networks of neurons that carry out some specific function: vision, hearing, movement. It makes sense to tackle these simple modules first. Moreover, we have good mathematical models of nerve cell behaviour. In 1952, Alan Hodgkin and Andrew Huxley wrote down the “Hodgkin-Huxley equations” for the transmission of a nerve impulse, which won them the 1963 Nobel Prize in Medicine. We also have effective techniques for understanding small networks’ components and how they are linked.

Many of these simple networks occur in the visual system. We used to think that the eye was like a camera, taking a “snapshot” of the outside world that was stored in the brain like a photo stuck in an album. It uses a lens to focus an image on to the retina at the back of the eye, which functions a bit like a roll of film – or, in today’s digital cameras, a charge-coupled device, storing an image pixel by pixel. But we now know that when the retina sends information to the brain’s visual cortex, the similarity to a camera ends.

Although we get a strong impression that what we are seeing is “out there” in front of us, what determines that perception resides inside our own heads. The brain decomposes images into simple pieces, works out what they are, “labels” them with that information, and reassembles them. When we see three sheep and two pigs in a field, we “know” which bits are sheep, which are pigs, and how many of each there are. If you try to program a computer to do that, you quickly realise how tricky the process is. Only very recently have computers been able to distinguish between faces, let alone sheep and pigs.

Probing the brain’s detailed activity is difficult. Rapid progress is being made, but it still takes a huge effort to get reliable information. But when science cannot observe something directly, it infers it, working indirectly. An effective way to infer how something functions is to see what it does when it goes wrong. It may be hard to understand a bridge while it stays up, but you can learn a lot about strength of materials when it collapses.

The visual system can “go wrong” in several interesting ways. Hallucinogenic drugs can change how neurons behave, producing dramatic images such as spinning spirals, which originate not in the eye, but in the brain. Some images even cause the brain to misinterpret what it’s seeing without outside help. We call them optical illusions.

One of the earliest was discovered in Renaissance Italy in the 16th century. Giambattista della Porta was the middle of three surviving sons of a wealthy merchant nobleman who became secretary to the Holy Roman emperor Charles V. The father was an intellectual, and Giambattista grew up in a house in Naples that hosted innumerable mathematicians, scientists, poets and musicians. He became an outstanding polymath, with publications on secret codes (including writing on the inside of eggshells), physiology, botany, agriculture, engineering, and much else. He wrote more than 20 plays.

Della Porta was particularly interested in the science of light. He made definitive improvements to the camera obscura, a device that projects an image of the outside world into a darkened room; he claimed to have invented the telescope before Galileo, and very likely did. His De refractione optices of 1593 contained the first report of a curious optical effect. He arranged two books so that one was visible to one eye only and the other to the other eye. Instead of seeing a combination of the two images, he perceived them alternately. He discovered that he could select either image at will by consciously switching his attention. This phenomenon is known today as binocular rivalry.

Two other distinct but related effects are impossible figures and visual illusions. In rivalry, each image appears unambiguous, but the eyes are shown conflicting images. In the other two phenomena, both eyes see the same image, but in one case it doesn’t make sense, and in the other it makes sense but is ambiguous.

Impossible figures at first sight seem to be entirely normal, but depict things that cannot exist – such as Roger Shepard’s 1990 drawing of an elephant in which everything above the knees makes sense, and everything below the knees makes sense, but the two regions do not fit together correctly. The Dutch artist Maurits Escher made frequent use of this kind of visual quirk.

In 1832, the Swiss crystallographer Louis Necker invented his “Necker cube” illusion, a skeletal cube that seems to switch its orientation repeatedly. An 1892 issue of the humorous German magazine Fliegende Blätter contains a picture with the caption “Which animals are most like each other?” and the answer “Rabbit and duck”. In a 1915 issue of the American magazine Puck, the cartoonist Ely William Hill published “My wife and my mother-in-law”, based on an 1888 German postcard. The image can be seen either as a young lady looking back over her shoulder, or as an elderly woman facing forwards. Several of Salvador Dalí’s paintings include illusions; especially Slave Market With the Apparition of the Invisible Bust of Voltaire, where a number of figures and everyday objects, carefully arranged, combine to give the impression of the French writer’s face.

Illusions offer insights into how the visual system processes images. The first few stages are fairly well understood. The top layer in the visual cortex detects edges of objects and the direction in which they are pointing. This information is passed to lower layers, which detect places where the direction suddenly changes, such as corners. Eventually some region in the cortex detects that you are looking at a human face and that it belongs to Aunt Matilda. Other parts of the brain are alerted, and you belatedly remember that tomorrow is her birthday and hurry off to buy a present.

These things don’t happen by magic. They have a very definite rationale, and that’s where the mathematics comes in. The top layer of the visual cortex contains innumerable tiny stacks of nerve cells. Each stack is like a pile of pancakes, and each pancake is a network of neurons that is sensitive to edges that point in one specific direction: one o’clock, two o’clock and so on.

For simplicity, call this network a cell; it does no harm to think of it as a single neuron. Roughly speaking, the cell at the top of the stack senses edges at the one o’clock position, the next one down corresponds to the two o’clock angle, and so on. If one cell receives a suitable input signal, it “fires”, telling all the other cells in its stack: “I’ve seen a boundary in the five o’clock direction.” However, another cell in the same stack might disagree, claiming the direction is at seven o’clock. How to resolve this conflict?

Neurons are linked by two kinds of connection, excitatory and inhibitory. If a neuron activates an excitatory connection, those at the other end of it are more likely to fire themselves. An inhibitory connection makes them less likely to fire. The cortex uses inhibitory connections to reach a definite decision. When a cell fires, it sends inhibitory signals to all of the other cells in its stack. These signals compete for attention. If the five o’clock signal is stronger than the seven o’clock one, for instance, the seven o’clock one gets shut down. The cells in effect “vote” on which direction they are detecting and the winner takes all.

Many neuroscientists think that something very similar is going on in visual illusions and rivalry. Think of the duck and rabbit with two possible interpretations. Hugh R Wilson, a neuroscientist at the Centre for Vision Research at York University, Toronto, proposed the simplest model, one stack with just two cells. Rodica Curtu, a mathematician at the University of Iowa, John Rinzel, a biomathematician then at the National Institutes of Health, and several other scientists have analysed this model in more detail. The basic idea is that one cell fires if the picture looks like a duck, the other if it resembles a rabbit. Because of the inhibitory connections, the winner should take all. Except that, in this illusion, it doesn’t quite work, because the two choices are equally plausible. That’s what makes it an illusion. So both cells want to fire. But they can’t, because of those inhibitory connections. Yet neither can they both remain quiescent, because the incoming signals encourage them to fire.

One possibility is that random signals coming from elsewhere in the brain might introduce a bias of perception, so that one cell still wins. However, the mathematical model predicts that, even without such bias, the signals in both cells should oscillate from active to inactive and back again, each becoming active when the other is not. It’s as if the network is dithering: the two cells take turns to fire and the network perceives the image as a duck, then as a rabbit, and keeps switching from one to the other. Which is what happens in reality.

Generalising from this observation, Wilson proposed a similar type of network that can model decision-making in the brain – which political party to support, for instance. But now the network consists of several stacks. Maybe one stack represents immigration policy, another unemployment, a third financial regulation, and so on. Each stack consists of cells that “recognise” a distinct policy feature. So the financial regulation stack has cells that recognise state regulation by law, self-regulation by the industry, or free-market economics.

The overall political stance of any given political party is a choice of one cell from each stack – one policy decision on each issue. Each prospective voter has his or her preferences, and these might not match those of any particular party. If these choices are used as inputs to the network, it will identify the party that most closely fits what the voter prefers. That decision can then be passed to other areas of the brain. Some voters may find themselves in a state akin to a visual illusion, vacillating between Labour and Liberal Democrat, or Conservative and Ukip.

This idea is speculative and it is not intended to be a literal description of how we decide whom to vote for. It is a schematic outline of something more complex, involving many regions of the brain. However, it provides a simple and flexible model for decision-making by a neural network, and in particular it shows that simple networks can do the job quite well. Martin Golubitsky of the Mathematical Biosciences Institute at Ohio State University and Casey O Diekman of the University of Michigan wondered whether Wilson’s networks could be used to model more complex examples of rivalry and illusions. Crucially, the resulting models allow specific predictions about experiments that have not yet been performed, making the whole idea scientifically testable.

The first success of this approach helped to explain an experiment that had already been carried out, with puzzling results. When the brain reassembles the separate bits of an image, it is said to “bind” these pieces. Rivalry provides evidence that binding occurs, by making it go wrong. In a rivalry experiment carried out in 2006 by S W Hong and S K Shevell, the subject’s left eye is shown a horizontal grid of grey and pink lines while the right eye sees a vertical grid of grey and green lines. Many subjects perceive an alternation between the images, just as della Porta did with his books. But some see two different images alternating: pink and green vertical lines, and pink and green horizontal lines – images shown to neither eye. This effect is called colour misbinding; it tells us that the reassembly process has matched colour to grid direction incorrectly. It is as if della Porta had ended up seeing another book altogether.

Golubitsky and Diekman studied the simplest Wilson network corresponding to this experiment. It has two stacks: one for colour, one for grid direction. Each stack has two cells. In the “colour” stack one cell detects pink and the other green; in the “orientation” stack one cell detects vertical and the other horizontal. As usual, there are inhibitory connections within each stack to ensure a winner-takes-all decision.

Following Wilson’s general scheme, they also added excitatory connections between cells in distinct stacks, representing the combinations of colour and direction that occur in the two “learned” images – those actually presented to the two eyes. Then they used recent mathematical techniques to list the patterns that arise in such a network. They found two types of oscillatory pattern. One corresponds to alternation between the two learned images. The other corresponds precisely to alternation between the two images seen in colour misbinding.

Colour misbinding is therefore a natural feature of the dynamics of Wilson networks. Although the network is “set up” to detect the two learned images, its structure produces an unexpected side effect: two images that were not learned. The rivalry experiment reveals hints of the brain’s hidden wiring. The same techniques apply to many other experiments, including some that have not yet been performed. They lead to very specific predictions, including more circumstances in which subjects will observe patterns that were not presented to either eye.

Similar models also apply to illusions. However, the excitatory connections cannot be determined by the images shown to the two eyes, because both eyes see the same image. One suggestion is that the connections may be determined by what your visual system already “knows” about real objects.

Take the celebrated moving illusion called “the spinning dancer”. Some observers see the solid silhouette of a dancer spinning anticlockwise, others clockwise. Sometimes, the direction of spin seems to switch suddenly.

We know that the top half of a spinning dancer can spin either clockwise or anticlockwise. Ditto for the bottom half. In principle, if the top half spins one way but the bottom half spins the other way, you would see the same silhouette, as if both were moving together. When people are shown “the spinning dancer”, no one sees the halves moving independently. If the top half spins clockwise, so does the bottom half.

Why do our brains do this? We can model that information using a series of stacks that correspond to different parts of the dancer’s body. The brain’s prior knowledge sets up a set of excitatory connections between all cells that sense clockwise motion, and another set of excitatory connections between all “anticlockwise” cells. We can also add inhibitory connections between the “clockwise” and the “anticlockwise” cells. These connections collectively tell the network that all parts of the object being perceived must spin in the same direction at any instant. Our brains don’t allow for a “half and half” interpretation.

When we analyse this network mathematically, it turns out that the cells switch repeatedly between a state in which all clockwise cells are firing but the anticlockwise ones are quiescent, and a state in which all anticlockwise cells are firing but the clockwise ones are quiescent. The upshot is that we perceive the whole figure of the dancer switching directions. Similar networks provide sensible models for many other illusions, including some in which there are three different inputs.

These models provide a common framework for both rivalry and illusion, and they unify many experiments, explain otherwise puzzling results and make new predictions that can be tested. They also tell us that in principle the brain can carry out some apparently complex tasks using simple networks. (What it does in practice is probably different in detail, but could well follow the same general lines.)

This could help make sense of a real brain, as new experiments improve our ability to observe its “wiring diagram”. It might not be as ambitious as trying to model the whole thing on a computer, but modesty can be a virtue. Since simple networks behave in strange and unexpected ways, what incomprehensible quirks might a complicated network have?

Perhaps Dalí, and Escher, and the spinning dancer can help us find out. 

Ian Stewart is Emeritus Professor of Mathematics and Digital Media Fellow at the University of Warwick

Show Hide image

Why Jeremy Corbyn is a new leader for the New Times

In an inspired election campaign, he confounded his detractors and showed that he was – more than any other leader – in tune with the times.

There have been two great political turning points in postwar Britain. The first was in 1945 with the election of the Attlee government. Driven by a popular wave of determination that peacetime Britain would look very different from the mass unemployment of the 1930s, and built on the foundations of the solidaristic spirit of the war, the Labour government ushered in full employment, the welfare state (including the NHS) and nationalisation of the basic industries, notably coal and the railways. It was a reforming government the like of which Britain had not previously experienced in the first half of the 20th century. The popular support enjoyed by the reforms was such that the ensuing social-democratic consensus was to last until the end of the 1970s, with Tory as well as Labour governments broadly operating within its framework.

During the 1970s, however, opposition to the social-democratic consensus grew steadily, led by the rise of the radical right, which culminated in 1979 in the election of Margaret Thatcher’s first government. In the process, the Thatcherites redefined the political debate, broadening it beyond the rather institutionalised and truncated forms that it had previously taken: they conducted a highly populist campaign that was for individualism and against collectivism; for the market and against the state; for liberty and against trade unionism; for law and order and against crime.

These ideas were dismissed by the left as just an extreme version of the same old Toryism, entirely failing to recognise their novelty and therefore the kind of threat they posed. The 1979 election, followed by Ronald Reagan’s US victory in 1980, began the neoliberal era, which remained hegemonic in Britain, and more widely in the West, for three decades. Tory and Labour governments alike operated within the terms and by the logic of neoliberalism. The only thing new about New Labour was its acquiescence in neoliberalism; even in this sense, it was not new but derivative of Thatcherism.

The financial crisis of 2007-2008 marked the beginning of the end of neoliberalism. Unlike the social-democratic consensus, which was undermined by the ideological challenge posed by Thatcherism, neoliberalism was brought to its knees not by any ideological alternative – such was the hegemonic sway of neoliberalism – but by the biggest financial crisis since 1931. This was the consequence of the fragility of a financial sector left to its own devices as a result of sweeping deregulation, and the corrupt and extreme practices that this encouraged.

The origin of the crisis lay not in the Labour government – complicit though it was in the neoliberal indulgence of the financial sector – but in the deregulation of the banking sector on both sides of the Atlantic in the 1980s. Neoliberalism limped on in the period after 2007-2008 but as real wages stagnated, recovery proved a mirage, and, with the behaviour of the bankers exposed, a deep disillusionment spread across society. During 2015-16, a populist wave of opposition to the establishment engulfed much of Europe and the United States.

Except at the extremes – Greece perhaps being the most notable example – the left was not a beneficiary: on the contrary it, too, was punished by the people in the same manner as the parties of the mainstream right were. The reason was straightforward enough. The left was tarnished with the same brush as the right: almost everywhere social-democratic parties, albeit to varying degrees, had pursued neoliberal policies. Bill Clinton and Tony Blair became – and presented themselves as – leaders of neoliberalism and as enthusiastic advocates of a strategy of hyper-globalisation, which resulted in growing inequality. In this fundamental respect these parties were more or less ­indistinguishable from the right.

***

The first signs of open revolt against New Labour – the representatives and evangelists of neoliberal ideas in the Labour Party – came in the aftermath of the 2015 ­election and the entirely unpredicted and overwhelming victory of Jeremy Corbyn in the leadership election. Something was happening. Yet much of the left, along with the media, summarily dismissed it as a revival of far-left entryism; that these were for the most part no more than a bunch of Trots. There is a powerful, often overwhelming, tendency to see new phenomena in terms of the past. The new and unfamiliar is much more difficult to understand than the old and familiar: it requires serious intellectual effort and an open and inquiring mind. The left is not alone in this syndrome. The right condemned the 2017 Labour Party manifesto as a replica of Labour’s 1983 manifesto. They couldn’t have been more wrong.

That Corbyn had been a veteran of the far left for so long lent credence to the idea that he was merely a retread of a failed past: there was nothing new about him. In a brilliant election campaign, Corbyn not only gave the lie to this but also demonstrated that he, far more than any of the other party leaders, was in tune with the times, the candidate of modernity.

Crises, great turning points, new conjunctures, new forms of consciousness are by definition incubators of the new. That is one of the great sources of their fascination. We can now see the line of linkage between the thousands of young people who gave Corbyn his overwhelming victory in the leadership election in 2015 and the millions of young people who were enthused by his general election campaign in 2017. It is no accident that it was the young rather than the middle-aged or the seniors who were in the vanguard: the young are the bearers and products of the new, they are the lightning conductors of change. Their elders, by contrast, are steeped in old ways of thinking and doing, having lived through and internalised the values and norms of neoliberalism for more than 30 years.

Yet there is another, rather more important aspect to how we identify the new, namely the way we see politics and how politics is conceived. Electoral politics is a highly institutionalised and tribal activity. There have been, as I argued earlier, two great turning points in postwar politics: the social-democratic era ushered in by the 1945 Labour government and the neoliberal era launched by the Tory government in 1979.

The average Tory MP or activist, no doubt, would interpret history primarily in terms of Tory and Labour governments; Labour MPs and activists would do similarly. But this is a superficial reading of politics based on party labels which ignores the deeper forces that shape different eras, generate crises and result in new paradigms.

Alas, most political journalists and columnists are afflicted with the same inability to distinguish the wood (an understanding of the deeper historical forces at work) from the trees (the day-to-day manoeuvring of parties and politicians). In normal times, this may not be so important, because life continues for the most part as before, but at moments of great paradigmatic change it is absolutely critical.

If the political journalists, and indeed the PLP, had understood the deeper forces and profound changes now at work, they would never have failed en masse to rise above the banal and predictable in their assessment of Corbyn. Something deep, indeed, is happening. A historical era – namely, that of neoliberalism – is in its death throes. All the old assumptions can no longer be assumed. We are in new territory: we haven’t been here before. The smart suits long preferred by New Labour wannabes are no longer a symbol of success and ambition but of alienation from, and rejection of, those who have been left behind; who, from being ignored and dismissed, are in the process of moving to the centre of the political stage.

Corbyn, you may recall, was instantly rejected and ridiculed for his sartorial style, and yet we can now see that, with a little smartening, it conveys an authenticity and affinity with the times that made his style of dress more or less immune from criticism during the general election campaign. Yet fashion is only a way to illustrate a much deeper point.

The end of neoliberalism, once so hegemonic, so commanding, is turning Britain on its head. That is why – extraordinary when you think about it – all the attempts by the right to dismiss Corbyn as a far-left extremist failed miserably, even proved counterproductive, because that was not how people saw him, not how they heard him. He was speaking a language and voicing concerns that a broad cross-section of the public could understand and identify with.

***

The reason a large majority of the PLP was opposed to Corbyn, desperate to be rid of him, was because they were still living in the neoliberal era, still slaves to its ideology, still in thrall to its logic. They knew no other way of thinking or political being. They accused Corbyn of being out of time when in fact it was most of the PLP – not to mention the likes of Mandelson and Blair – who were still imprisoned in an earlier historical era. The end of neoliberalism marks the death of New Labour. In contrast, Corbyn is aligned with the world as it is rather than as it was. What a wonderful irony.

Corbyn’s success in the general election requires us to revisit some of the assumptions that have underpinned much political commentary over the past several years. The turmoil in Labour ranks and the ridiculing of Corbyn persuaded many, including on the left, that Labour stood on the edge of the abyss and that the Tories would continue to dominate for long into the future. With Corbyn having seized the political initiative, the Tories are now cast in a new light. With Labour in the process of burying its New Labour legacy and addressing a very new conjuncture, then the end of neoliberalism poses a much more serious challenge to the Tories than it does the Labour Party.

The Cameron/Osborne leadership was still very much of a neoliberal frame of mind, not least in their emphasis on austerity. It would appear that, in the light of the new popular mood, the government will now be forced to abandon austerity. Theresa May, on taking office, talked about a return to One Nation Toryism and the need to help the worst-off, but that has never moved beyond rhetoric: now she is dead in the water.

Meanwhile, the Tories are in fast retreat over Brexit. They held a referendum over the EU for narrowly party reasons which, from a national point of view, was entirely unnecessary. As a result of the Brexit vote, the Cameron leadership was forced to resign and the Brexiteers took de facto command. But now, after the election, the Tories are in headlong retreat from anything like a “hard Brexit”. In short, they have utterly lost control of the political agenda and are being driven by events. Above all, they are frightened of another election from which Corbyn is likely to emerge as leader with a political agenda that will owe nothing to neoliberalism.

Apart from Corbyn’s extraordinary emergence as a leader who understands – and is entirely comfortable with – the imperatives of the new conjuncture and the need for a new political paradigm, the key to Labour’s transformed position in the eyes of the public was its 2017 manifesto, arguably its best and most important since 1945. You may recall that for three decades the dominant themes were marketisation, privatisation, trickle-down economics, the wastefulness and inefficiencies of the state, the incontrovertible case for hyper-globalisation, and bankers and financiers as the New Gods.

Labour’s manifesto offered a very different vision: a fairer society, bearing down on inequality, a more redistributive tax system, the centrality of the social, proper funding of public services, nationalisation of the railways and water industry, and people as the priority rather than business and the City. The title captured the spirit – For the Many Not the Few. Or, to put in another way, After Neoliberalism. The vision is not yet the answer to the latter question, but it represents the beginnings of an answer.

Ever since the late 1970s, Labour has been on the defensive, struggling to deal with a world where the right has been hegemonic. We can now begin to glimpse a different possibility, one in which the left can begin to take ownership – at least in some degree – of a new, post-neoliberal political settlement. But we should not underestimate the enormous problems that lie in wait. The relative economic prospects for the country are far worse than they have been at any time since 1945. As we saw in the Brexit vote, the forces of conservatism, nativism, racism and imperial nostalgia remain hugely powerful. Not only has the country rejected continued membership of the European Union, but, along with the rest of the West, it is far from reconciled with the new world that is in the process of being created before our very eyes, in which the developing world will be paramount and in which China will be the global leader.

Nonetheless, to be able to entertain a sense of optimism about our own country is a novel experience after 30 years of being out in the cold. No wonder so many are feeling energised again.

This article first appeared in the 15 June 2017 issue of the New Statesman, Corbyn: revenge of the rebel

Martin Jacques is the former editor of Marxism Today. 

This article first appeared in the 15 June 2017 issue of the New Statesman, Corbyn: revenge of the rebel

0800 7318496