Show Hide image

Now you see it, now you don't: what optical illusions tell us about our brains

Illusions can offer insights into how the visual system processes images.

Maurits Escher: where do the staircases lead?

The human brain is a network of about 20 billion neurons – nerve cells – linked by several trillion connections. Not to mention glial cells, which scientists used to think were inactive scaffolding, but increasingly view as an essential part of how the brain works. Our brains give us movement, language, senses, memories, consciousness and personality. We know a lot more about the brain than we used to, but it still seems far too complicated for human understanding.

Fortunately, the brain contains many small networks of neurons that carry out some specific function: vision, hearing, movement. It makes sense to tackle these simple modules first. Moreover, we have good mathematical models of nerve cell behaviour. In 1952, Alan Hodgkin and Andrew Huxley wrote down the “Hodgkin-Huxley equations” for the transmission of a nerve impulse, which won them the 1963 Nobel Prize in Medicine. We also have effective techniques for understanding small networks’ components and how they are linked.

Many of these simple networks occur in the visual system. We used to think that the eye was like a camera, taking a “snapshot” of the outside world that was stored in the brain like a photo stuck in an album. It uses a lens to focus an image on to the retina at the back of the eye, which functions a bit like a roll of film – or, in today’s digital cameras, a charge-coupled device, storing an image pixel by pixel. But we now know that when the retina sends information to the brain’s visual cortex, the similarity to a camera ends.

Although we get a strong impression that what we are seeing is “out there” in front of us, what determines that perception resides inside our own heads. The brain decomposes images into simple pieces, works out what they are, “labels” them with that information, and reassembles them. When we see three sheep and two pigs in a field, we “know” which bits are sheep, which are pigs, and how many of each there are. If you try to program a computer to do that, you quickly realise how tricky the process is. Only very recently have computers been able to distinguish between faces, let alone sheep and pigs.

Probing the brain’s detailed activity is difficult. Rapid progress is being made, but it still takes a huge effort to get reliable information. But when science cannot observe something directly, it infers it, working indirectly. An effective way to infer how something functions is to see what it does when it goes wrong. It may be hard to understand a bridge while it stays up, but you can learn a lot about strength of materials when it collapses.

The visual system can “go wrong” in several interesting ways. Hallucinogenic drugs can change how neurons behave, producing dramatic images such as spinning spirals, which originate not in the eye, but in the brain. Some images even cause the brain to misinterpret what it’s seeing without outside help. We call them optical illusions.

One of the earliest was discovered in Renaissance Italy in the 16th century. Giambattista della Porta was the middle of three surviving sons of a wealthy merchant nobleman who became secretary to the Holy Roman emperor Charles V. The father was an intellectual, and Giambattista grew up in a house in Naples that hosted innumerable mathematicians, scientists, poets and musicians. He became an outstanding polymath, with publications on secret codes (including writing on the inside of eggshells), physiology, botany, agriculture, engineering, and much else. He wrote more than 20 plays.

Della Porta was particularly interested in the science of light. He made definitive improvements to the camera obscura, a device that projects an image of the outside world into a darkened room; he claimed to have invented the telescope before Galileo, and very likely did. His De refractione optices of 1593 contained the first report of a curious optical effect. He arranged two books so that one was visible to one eye only and the other to the other eye. Instead of seeing a combination of the two images, he perceived them alternately. He discovered that he could select either image at will by consciously switching his attention. This phenomenon is known today as binocular rivalry.

Two other distinct but related effects are impossible figures and visual illusions. In rivalry, each image appears unambiguous, but the eyes are shown conflicting images. In the other two phenomena, both eyes see the same image, but in one case it doesn’t make sense, and in the other it makes sense but is ambiguous.

Impossible figures at first sight seem to be entirely normal, but depict things that cannot exist – such as Roger Shepard’s 1990 drawing of an elephant in which everything above the knees makes sense, and everything below the knees makes sense, but the two regions do not fit together correctly. The Dutch artist Maurits Escher made frequent use of this kind of visual quirk.

In 1832, the Swiss crystallographer Louis Necker invented his “Necker cube” illusion, a skeletal cube that seems to switch its orientation repeatedly. An 1892 issue of the humorous German magazine Fliegende Blätter contains a picture with the caption “Which animals are most like each other?” and the answer “Rabbit and duck”. In a 1915 issue of the American magazine Puck, the cartoonist Ely William Hill published “My wife and my mother-in-law”, based on an 1888 German postcard. The image can be seen either as a young lady looking back over her shoulder, or as an elderly woman facing forwards. Several of Salvador Dalí’s paintings include illusions; especially Slave Market With the Apparition of the Invisible Bust of Voltaire, where a number of figures and everyday objects, carefully arranged, combine to give the impression of the French writer’s face.

Illusions offer insights into how the visual system processes images. The first few stages are fairly well understood. The top layer in the visual cortex detects edges of objects and the direction in which they are pointing. This information is passed to lower layers, which detect places where the direction suddenly changes, such as corners. Eventually some region in the cortex detects that you are looking at a human face and that it belongs to Aunt Matilda. Other parts of the brain are alerted, and you belatedly remember that tomorrow is her birthday and hurry off to buy a present.

These things don’t happen by magic. They have a very definite rationale, and that’s where the mathematics comes in. The top layer of the visual cortex contains innumerable tiny stacks of nerve cells. Each stack is like a pile of pancakes, and each pancake is a network of neurons that is sensitive to edges that point in one specific direction: one o’clock, two o’clock and so on.

For simplicity, call this network a cell; it does no harm to think of it as a single neuron. Roughly speaking, the cell at the top of the stack senses edges at the one o’clock position, the next one down corresponds to the two o’clock angle, and so on. If one cell receives a suitable input signal, it “fires”, telling all the other cells in its stack: “I’ve seen a boundary in the five o’clock direction.” However, another cell in the same stack might disagree, claiming the direction is at seven o’clock. How to resolve this conflict?

Neurons are linked by two kinds of connection, excitatory and inhibitory. If a neuron activates an excitatory connection, those at the other end of it are more likely to fire themselves. An inhibitory connection makes them less likely to fire. The cortex uses inhibitory connections to reach a definite decision. When a cell fires, it sends inhibitory signals to all of the other cells in its stack. These signals compete for attention. If the five o’clock signal is stronger than the seven o’clock one, for instance, the seven o’clock one gets shut down. The cells in effect “vote” on which direction they are detecting and the winner takes all.

Many neuroscientists think that something very similar is going on in visual illusions and rivalry. Think of the duck and rabbit with two possible interpretations. Hugh R Wilson, a neuroscientist at the Centre for Vision Research at York University, Toronto, proposed the simplest model, one stack with just two cells. Rodica Curtu, a mathematician at the University of Iowa, John Rinzel, a biomathematician then at the National Institutes of Health, and several other scientists have analysed this model in more detail. The basic idea is that one cell fires if the picture looks like a duck, the other if it resembles a rabbit. Because of the inhibitory connections, the winner should take all. Except that, in this illusion, it doesn’t quite work, because the two choices are equally plausible. That’s what makes it an illusion. So both cells want to fire. But they can’t, because of those inhibitory connections. Yet neither can they both remain quiescent, because the incoming signals encourage them to fire.

One possibility is that random signals coming from elsewhere in the brain might introduce a bias of perception, so that one cell still wins. However, the mathematical model predicts that, even without such bias, the signals in both cells should oscillate from active to inactive and back again, each becoming active when the other is not. It’s as if the network is dithering: the two cells take turns to fire and the network perceives the image as a duck, then as a rabbit, and keeps switching from one to the other. Which is what happens in reality.

Generalising from this observation, Wilson proposed a similar type of network that can model decision-making in the brain – which political party to support, for instance. But now the network consists of several stacks. Maybe one stack represents immigration policy, another unemployment, a third financial regulation, and so on. Each stack consists of cells that “recognise” a distinct policy feature. So the financial regulation stack has cells that recognise state regulation by law, self-regulation by the industry, or free-market economics.

The overall political stance of any given political party is a choice of one cell from each stack – one policy decision on each issue. Each prospective voter has his or her preferences, and these might not match those of any particular party. If these choices are used as inputs to the network, it will identify the party that most closely fits what the voter prefers. That decision can then be passed to other areas of the brain. Some voters may find themselves in a state akin to a visual illusion, vacillating between Labour and Liberal Democrat, or Conservative and Ukip.

This idea is speculative and it is not intended to be a literal description of how we decide whom to vote for. It is a schematic outline of something more complex, involving many regions of the brain. However, it provides a simple and flexible model for decision-making by a neural network, and in particular it shows that simple networks can do the job quite well. Martin Golubitsky of the Mathematical Biosciences Institute at Ohio State University and Casey O Diekman of the University of Michigan wondered whether Wilson’s networks could be used to model more complex examples of rivalry and illusions. Crucially, the resulting models allow specific predictions about experiments that have not yet been performed, making the whole idea scientifically testable.

The first success of this approach helped to explain an experiment that had already been carried out, with puzzling results. When the brain reassembles the separate bits of an image, it is said to “bind” these pieces. Rivalry provides evidence that binding occurs, by making it go wrong. In a rivalry experiment carried out in 2006 by S W Hong and S K Shevell, the subject’s left eye is shown a horizontal grid of grey and pink lines while the right eye sees a vertical grid of grey and green lines. Many subjects perceive an alternation between the images, just as della Porta did with his books. But some see two different images alternating: pink and green vertical lines, and pink and green horizontal lines – images shown to neither eye. This effect is called colour misbinding; it tells us that the reassembly process has matched colour to grid direction incorrectly. It is as if della Porta had ended up seeing another book altogether.

Golubitsky and Diekman studied the simplest Wilson network corresponding to this experiment. It has two stacks: one for colour, one for grid direction. Each stack has two cells. In the “colour” stack one cell detects pink and the other green; in the “orientation” stack one cell detects vertical and the other horizontal. As usual, there are inhibitory connections within each stack to ensure a winner-takes-all decision.

Following Wilson’s general scheme, they also added excitatory connections between cells in distinct stacks, representing the combinations of colour and direction that occur in the two “learned” images – those actually presented to the two eyes. Then they used recent mathematical techniques to list the patterns that arise in such a network. They found two types of oscillatory pattern. One corresponds to alternation between the two learned images. The other corresponds precisely to alternation between the two images seen in colour misbinding.

Colour misbinding is therefore a natural feature of the dynamics of Wilson networks. Although the network is “set up” to detect the two learned images, its structure produces an unexpected side effect: two images that were not learned. The rivalry experiment reveals hints of the brain’s hidden wiring. The same techniques apply to many other experiments, including some that have not yet been performed. They lead to very specific predictions, including more circumstances in which subjects will observe patterns that were not presented to either eye.

Similar models also apply to illusions. However, the excitatory connections cannot be determined by the images shown to the two eyes, because both eyes see the same image. One suggestion is that the connections may be determined by what your visual system already “knows” about real objects.

Take the celebrated moving illusion called “the spinning dancer”. Some observers see the solid silhouette of a dancer spinning anticlockwise, others clockwise. Sometimes, the direction of spin seems to switch suddenly.

We know that the top half of a spinning dancer can spin either clockwise or anticlockwise. Ditto for the bottom half. In principle, if the top half spins one way but the bottom half spins the other way, you would see the same silhouette, as if both were moving together. When people are shown “the spinning dancer”, no one sees the halves moving independently. If the top half spins clockwise, so does the bottom half.

Why do our brains do this? We can model that information using a series of stacks that correspond to different parts of the dancer’s body. The brain’s prior knowledge sets up a set of excitatory connections between all cells that sense clockwise motion, and another set of excitatory connections between all “anticlockwise” cells. We can also add inhibitory connections between the “clockwise” and the “anticlockwise” cells. These connections collectively tell the network that all parts of the object being perceived must spin in the same direction at any instant. Our brains don’t allow for a “half and half” interpretation.

When we analyse this network mathematically, it turns out that the cells switch repeatedly between a state in which all clockwise cells are firing but the anticlockwise ones are quiescent, and a state in which all anticlockwise cells are firing but the clockwise ones are quiescent. The upshot is that we perceive the whole figure of the dancer switching directions. Similar networks provide sensible models for many other illusions, including some in which there are three different inputs.

These models provide a common framework for both rivalry and illusion, and they unify many experiments, explain otherwise puzzling results and make new predictions that can be tested. They also tell us that in principle the brain can carry out some apparently complex tasks using simple networks. (What it does in practice is probably different in detail, but could well follow the same general lines.)

This could help make sense of a real brain, as new experiments improve our ability to observe its “wiring diagram”. It might not be as ambitious as trying to model the whole thing on a computer, but modesty can be a virtue. Since simple networks behave in strange and unexpected ways, what incomprehensible quirks might a complicated network have?

Perhaps Dalí, and Escher, and the spinning dancer can help us find out. 

Ian Stewart is Emeritus Professor of Mathematics and Digital Media Fellow at the University of Warwick

Jeremy Corbyn. Photo: Getty
Show Hide image

Lexit: the EU is a neoliberal project, so let's do something different when we leave it

Brexit affords the British left a historic opportunity for a decisive break with EU market liberalism.

The Brexit vote to leave the European Union has many parents, but "Lexit" – the argument for exiting the EU from the left – remains an orphan. A third of Labour voters backed Leave, but they did so without any significant leadership from the Labour Party. Left-of-centre votes proved decisive in determining the outcome of a referendum that was otherwise framed, shaped, and presented almost exclusively by the right. A proper left discussion of the issues has been, if not entirely absent, then decidedly marginal – part of a more general malaise when it comes to developing left alternatives that has begun to be corrected only recently, under Jeremy Corbyn and John McDonnell.

Ceding Brexit to the right was very nearly the most serious strategic mistake by the British left since the ‘70s. Under successive leaders Labour became so incorporated into the ideology of Europeanism as to preclude any clear-eyed critical analysis of the actually existing EU as a regulatory and trade regime pursuing deep economic integration. The same political journey that carried Labour into its technocratic embrace of the EU also resulted in the abandonment of any form of distinctive economics separate from the orthodoxies of market liberalism.

It’s been astounding to witness so many left-wingers, in meltdown over Brexit, resort to parroting liberal economics. Thus we hear that factor mobility isn’t about labour arbitrage, that public services aren’t under pressure, that we must prioritise foreign direct investment and trade. It’s little wonder Labour became so detached from its base. Such claims do not match the lived experience of ordinary people in regions of the country devastated by deindustrialisation and disinvestment.

Nor should concerns about wage stagnation and bargaining power be met with finger-wagging accusations of racism, as if the manner in which capitalism pits workers against each other hasn’t long been understood. Instead, we should be offering real solutions – including a willingness to rethink capital mobility and trade. This places us in direct conflict with the constitutionalised neoliberalism of the EU.

Only the political savvy of the leadership has enabled Labour to recover from its disastrous positioning post-referendum. Incredibly, what seemed an unbeatable electoral bloc around Theresa May has been deftly prized apart in the course of an extraordinary General Election campaign. To consolidate the political project they have initiated, Corbyn and McDonnell must now follow through with a truly radical economic programme. The place to look for inspiration is precisely the range of instruments and policy options discouraged or outright forbidden by the EU.

A neoliberal project

The fact that right-wing arguments for Leave predominated during the referendum says far more about today’s left than it does about the European Union. There has been a great deal of myth-making concerning the latter –much of it funded, directly or indirectly, by the EU itself.

From its inception, the EU has been a top-down project driven by political and administrative elites, "a protected sphere", in the judgment of the late Peter Mair, "in which policy-making can evade the constraints imposed by representative democracy". To complain about the EU’s "democratic deficit" is to have misunderstood its purpose. The main thrust of European economic policy has been to extend and deepen the market through liberalisation, privatisation, and flexiblisation, subordinating employment and social protection to goals of low inflation, debt reduction, and increased competitiveness.

Prospects for Keynesian reflationary policies, or even for pan-European economic planning – never great – soon gave way to more Hayekian conceptions. Hayek’s original insight, in The Economic Conditions of Interstate Federalism, was that free movement of capital, goods, and labour – a "single market" – among a federation of nations would severely and necessarily restrict the economic policy space available to individual members. Pro-European socialists, whose aim had been to acquire new supranational options for the regulation of capital, found themselves surrendering the tools they already possessed at home. The national road to socialism, or even to social democracy, was closed.

The direction of travel has been singular and unrelenting. To take one example, workers’ rights – a supposed EU strength – are steadily being eroded, as can be seen in landmark judgments by the European Court of Justice (ECJ) in the Viking and Laval cases, among others. In both instances, workers attempting to strike in protest at plans to replace workers from one EU country with lower-wage workers from another, were told their right to strike could not infringe upon the "four freedoms" – free movement of capital, labour, goods, and services – established by the treaties.

More broadly, on trade, financial regulation, state aid, government purchasing, public service delivery, and more, any attempt to create a different kind of economy from inside the EU has largely been forestalled by competition policy or single market regulation.

A new political economy

Given that the UK will soon be escaping the EU, what opportunities might this afford? Three policy directions immediately stand out: public ownership, industrial strategy, and procurement. In each case, EU regulation previously stood in the way of promising left strategies. In each case, the political and economic returns from bold departures from neoliberal orthodoxy after Brexit could be substantial.

While not banned outright by EU law, public ownership is severely discouraged and disadvantaged by it. ECJ interpretation of Article 106 of the Treaty on the Functioning of the European Union (TFEU) has steadily eroded public ownership options. "The ECJ", argues law professor Danny Nicol, "appears to have constructed a one-way street in favour of private-sector provision: nationalised services are prima facie suspect and must be analysed for their necessity". Sure enough, the EU has been a significant driver of privatisation, functioning like a ratchet. It’s much easier for a member state to pursue the liberalisation of sectors than to secure their (re)nationalisation. Article 59 (TFEU) specifically allows the European Council and Parliament to liberalise services. Since the ‘80s, there have been single market programmes in energy, transport, postal services, telecommunications, education, and health.

Britain has long been an extreme outlier on privatisation, responsible for 40 per cent of the total assets privatised across the OECD between 1980 and 1996. Today, however, increasing inequality, poverty, environmental degradation and the general sense of an impoverished public sphere are leading to growing calls for renewed public ownership (albeit in new, more democratic forms). Soon to be free of EU constraints, it’s time to explore an expanded and fundamentally reimagined UK public sector.

Next, Britain’s industrial production has been virtually flat since the late 1990s, with a yawning trade deficit in industrial goods. Any serious industrial strategy to address the structural weaknesses of UK manufacturing will rely on "state aid" – the nurturing of a next generation of companies through grants, interest and tax relief, guarantees, government holdings, and the provision of goods and services on a preferential basis.

Article 107 TFEU allows for state aid only if it is compatible with the internal market and does not distort competition, laying out the specific circumstances in which it could be lawful. Whether or not state aid meets these criteria is at the sole discretion of the Commission – and courts in member states are obligated to enforce the commission’s decisions. The Commission has adopted an approach that considers, among other things, the existence of market failure, the effectiveness of other options, and the impact on the market and competition, thereby allowing state aid only in exceptional circumstances.

For many parts of the UK, the challenges of industrial decline remain starkly present – entire communities are thrown on the scrap heap, with all the associated capital and carbon costs and wasted lives. It’s high time the left returned to the possibilities inherent in a proactive industrial strategy. A true community-sustaining industrial strategy would consist of the deliberate direction of capital to sectors, localities, and regions, so as to balance out market trends and prevent communities from falling into decay, while also ensuring the investment in research and development necessary to maintain a highly productive economy. Policy, in this vision, would function to re-deploy infrastructure, production facilities, and workers left unemployed because of a shutdown or increased automation.

In some cases, this might mean assistance to workers or localities to buy up facilities and keep them running under worker or community ownership. In other cases it might involve re-training workers for new skills and re-fitting facilities. A regional approach might help launch new enterprises that would eventually be spun off as worker or local community-owned firms, supporting the development of strong and vibrant network economies, perhaps on the basis of a Green New Deal. All of this will be possible post-Brexit, under a Corbyn government.

Lastly, there is procurement. Under EU law, explicitly linking public procurement to local entities or social needs is difficult. The ECJ has ruled that, even if there is no specific legislation, procurement activity must "comply with the fundamental rules of the Treaty, in particular the principle of non-discrimination on grounds of nationality". This means that all procurement contracts must be open to all bidders across the EU, and public authorities must advertise contracts widely in other EU countries. In 2004, the European Parliament and Council issued two directives establishing the criteria governing such contracts: "lowest price only" and "most economically advantageous tender".

Unleashed from EU constraints, there are major opportunities for targeting large-scale public procurement to rebuild and transform communities, cities, and regions. The vision behind the celebrated Preston Model of community wealth building – inspired by the work of our own organisation, The Democracy Collaborative, in Cleveland, Ohio – leverages public procurement and the stabilising power of place-based anchor institutions (governments, hospitals, universities) to support rooted, participatory, democratic local economies built around multipliers. In this way, public funds can be made to do "double duty"; anchoring jobs and building community wealth, reversing long-term economic decline. This suggests the viability of a very different economic approach and potential for a winning political coalition, building support for a new socialist economics from the ground up.

With the prospect of a Corbyn government now tantalisingly close, it’s imperative that Labour reconciles its policy objectives in the Brexit negotiations with its plans for a radical economic transformation and redistribution of power and wealth. Only by pursuing strategies capable of re-establishing broad control over the national economy can Labour hope to manage the coming period of pain and dislocation following Brexit. Based on new institutions and approaches and the centrality of ownership and control, democracy, and participation, we should be busy assembling the tools and strategies that will allow departure from the EU to open up new political-economic horizons in Britain and bring about the profound transformation the country so desperately wants and needs.

Joe Guinan is executive director of the Next System Project at The Democracy Collaborative. Thomas M. Hanna is research director at The Democracy Collaborative.

This is an extract from a longer essay which appears in the inaugural edition of the IPPR Progressive Review.

 

 

0800 7318496