In a report published yesterday, former Cameron advisor James O’Shaughnessy identified an important problem. The conversion of thousands of schools into academies – which are free from local authority control – has left a vacuum with nobody to oversee school improvement at a local level. Michael Gove is finding that he cannot reliably monitor thousands of individual schools from his office in Whitehall. In the words of the report, this centralised system is "simply not viable" as a strategy for improving schools.

O’Shaughnessy’s first answer is for underperforming schools to be forced into academy chains. These are groups of schools that operate under an umbrella organisation which can monitor their performance and help them improve. There is some merit in this idea as many academy chains have proven to be successful at improving schools. Indeed it already happens to a large extent with failing schools – the report is just recommending an expansion of this approach to include schools that are mediocre, rather than plain bad.

O’Shaughnessy’s second answer is much more problematic. He argues that for-profit providers are best placed to take over the running of these schools and chains. According to him, only private companies – driven by the desire to make a profit – will have an incentive to turn around these schools. Without them the system will not be able to do the job.

There are good reasons why new providers can help our schools to improve – but they don’t have to be commercial companies. England already has a vibrant charitable independent sector and there is no shortage of organisations – like Harris and Ark – who are prepared to run our schools on a not-for-profit basis. Indeed academy chains in England are expanding at a far faster rate than the US.

Neither is there international evidence that commercial companies will improve results. As a recent IPPR report showed, profit-making companies have been brought in to run schools in Chile, Sweden and the US with little impact on standards.

Rather than turning to tired and unproven ideas around the power of the private sector, the government should adopt a different strategy for improving schools based on world class systems such as Canada and Finland. These countries can teach England three lessons on how to improve schools.

First, they rely on building the capacity of their teaching profession. In Finland, teachers are drawn from the top third of graduates, and those who work with the toughest children have masters degrees. In England, the government has taken the opposite approach – deregulating the sector and giving schools the freedom to recruit people who haven’t even qualified or trained as a teacher.

Second, these countries place schools in clusters where they collaborate with each other - sharing the best teachers, observing each other’s performance, spreading good practice and challenging each other to do improve. This sort of collaboration is hard to foster in the sort of market advocated by O’Shaughnessy - where companies have an incentive to compete for profit and market share rather than work together.  

Third, these countries all have structures in place to monitor the performance of schools and drive improvement at the local level. In Canada, school superintendents help to spot problems early and help tackle them before they escalate – they don’t leave it for distant bodies such as Ofsted or government ministers to do. O’Shaughnessy acknowledges the importance of this function in his report – and calls for a local schools commissioner to fill the role. But under his model this job would be put out to tender so that any organisation – public or private – would be responsible for assessing whether schools should be forced to change management. A far better model would be for school commissioners to be separate but accountable to local authorities, as IPPR had argued.

O’Shaughnessy's report has exposed a gap at the heart of the government’s school improvement agenda. The academies programme has created thousands of individual schools with little oversight or support to improve. Rather than putting his faith in commercial companies to provide the answer, Michael Gove should adopt a strategy that builds the capacity of the teaching profession, fosters collaboration between schools, and holds them to account for their performance through more democratic means.

Jonathan Clifton is a senior research fellow at IPPR. Follow him on Twitter: @jp_clifton