Opening up government

Jeni Tennison, technical director of the Open Data Institute, discusses progress on releasing data, how it feeds into wider Whitehall technology reform and why companies should follow the government's lead.

It’s three years since the launch of and we now have access to datasets covering topics as diverse as HS2, broadband and GP prescriptions. The UK has won many plaudits as a result and is often touted as a world leader in the field of open data.

Time then, perhaps, to give some credit to the people behind these achievements, one of whom is Jeni Tennison.

Tennison has a number of roles: primarily as technical director of the Open Data Institute, but she also sits on the Cabinet Office’s Open Data User Group (ODUG) and the Government Digital Service’s Open Standards Board, to name but a couple.

Tennison agrees that major progress has been made. She says:

We’re getting to a stage now in the UK where, happily, a lot of data that can easily be released has already been published.

However, she is far from complacent. Tennison says, “The challenges are around data that is still being made available for money, where a business case has to be made to demonstrate that there is wider economic benefit against the short-term cost of losing that revenue. There are also challenges around the formats in which information is made available and the regularity and guarantees around that data.”

She explains that a key part of ODUG’s work is representing user requirements for data into government and helping to prioritise datasets for release.

Tennison adds, “It’s just trying to unpick those requirements and represent them into government. Obviously we’ve also had campaigns that ODUG has run around open address information, and ongoing discussions with Ordnance Survey over its licensing terms for example.

“So basically ODUG is a campaigning body that’s close to government and that tries to represent the need for data, spanning a broad range of different interests including big businesses, small businesses and civil society.”

Privacy trade-offs

Regarding concerns raised about open data, particularly around security and privacy, Tennison explains, “I think that you'll find that within the open data community there is a very strong recognition of the need to protect privacy but it’s more complicated than baldly saying 'open data is not personal data'. There are grey areas where personal information needs to be anonymised or aggregated in order to provide the benefits of openness without intruding on people's lives.

“And we're doing that very slowly, very gradually and are very much aware of the risks. The worst thing that could possibly happen for the open data community would be for some personal data to be released under the open data agenda because it would set it back. So it's not in anyone's interest. It's something we feel very strongly about in fact.”

She adds, “It's not 'everything should be out there', it's much more 'what data can we best make use of', and where are the trade-offs?”

As an example of these concerns in practice, Tennison points to a recent HM Revenue & Customs consultation about the VAT register, which contains information on businesses.

However, she explains:

A large proportion of businesses in the VAT register, especially if they aren't in Companies House already, will be named individuals as sole traders. So there is personal information within the register, certainly personally identifiable information.

On the other hand there's a big transparency and open data story about being able to get VAT registration information because it helps you to tie up all of the spending that local government does and central government does with particular companies. So there are lots of benefits economically, and in terms of service delivery and transparency, of making that information available. But then you have this personal data. So you have to balance these up.

And the argument that we made in response to that was that there was a subset of fields that didn't reveal too much but provided enough information to be useful for the 80% of things that didn't need all of this information to be made available. And so, just opening that up would be sufficient to address most of the requirements. It doesn't address all of them but in this case that was the right place to draw the line.

Tennison suggests that organisations follow ICO (Information Commissioner’s Office) advice, which says “it’s open by default, but if you have personal information in your data then you should conduct a Privacy Impact Assessment: you consult with any stakeholders that would be affected by revealing this information, and you do a considered analysis of what the impact will be of releasing that data.

“And when you've done that considered analysis then you may find there are certain bits that shouldn't be released, other bits that can be released, and it allows you to make those trade-offs in a risk-assessment-like process.”

Expanding the audience

Tennison says that one of her worries is about trying to explain what open data means for the general public. She says, “People often don’t know they want open data until they need it, for example about their local schools and hospitals.”

“Certainly one of the things that we concentrate really strongly on at the ODI is trying to expand the set of people that think about and understand what open data is and what it means beyond the existing open data community and the geeks. And the kind of technique that we're using at the moment is lots of storiesto try to bring home how open data enables new things to happen.”

Tennison is also keen to expand an open data ethos throughout Whitehall. She regularly visits the Government Digital Service (GDS) for meetings of the Open Standards Board, but recently did a presentation during one of their monthly meetings.

She says, “The pitch that I made to GDS was that open data should be very much at the heart of what they are doing, and they get that. It’s been very hard for them because they’ve had to focus on getting GOV.UK running, getting the government departments in, and now getting the public sector agencies in too.

“They are starting now, really excitingly, to look more at how they make data available as open data and to build that into their thinking. GDS should be one of the greatest beneficiaries of open data from the public sector because they should be able to build any open data that is available into their own services and the transactions that they are supporting.

“And so I’d really like GDS to be making a stronger stand around asking for the data that they need to be made available as open data by departments. To be asking for that themselves. They’re beginning to do some really great things. They are starting to build in micro APIs to the GOV.UK back end, so if you’re building an application that depends on the VAT rate, you can have the API call that goes and gets whatever the current VAT rate is and uses that.

“That’s always been what they intended to do at GDS: making it wholesale, enabling other people to build in services and data that they’re making available. So it’s really good to see them starting to move in that direction and I’m expecting great things from them.”

Making government more open

Tennison explains that ‘opening up’ technology is crucial if the government is to succeed in its plans to reform technology and break away from a reliance on a small pool of suppliers.

She says, “What I think is really interesting here is how open source, open standards and open data all come together to support the same agenda. Government doesn’t want to be locked into massive IT contracts with a small set of suppliers. “Open" helps because when you have open data being published then you have a level playing field about what people know. When you have open standards being used then you have a level playing field about what tools can be used. And when you have open source established you have a level playing field about really just getting started with using that data and using those standards.

“We’re not going to see the effect [of the reforms] until the big contracts start coming to an end, but they are laying the ground work. We can’t think that it’s job done. If you look at Dun& Bradstreet identifiers and the ACORN classification, you get those being built into IT systems and that forces a reliance on one supplier. These things have been built into the heart of some of the IT systems and processes and metrics and everything that is being used within government in order to make decisions. You’re completely locked in.

“And the other thing that you have to be extremely careful about within government is ‘open-washing’ that sometimes goes on. So, for example, you will get an API that is described as an ‘open API’ where in fact it doesn’t mean that the data is open data or using an open standard. The underlying standard that is used is open but the actual language, the vocabulary itself, is developed by one company and only one company uses it.

“So there are lots of things to be careful of. And of course it’s completely understandable that it’s in a company’s interest to try and keep people locked into using them and they’re going to keep on trying to get people locked into using them, because that’s what companies do. The government initiatives around open source, open standards and open data are so important because it’s only those that push back. It’s only those that enable you to stop that kind of thing from happening.”

Moving beyond the public sector

Regarding the ODI’s future, Tennison says, “A really important role that ODI will have over the next few years is about moving the open data community outside the government data space. Because although government holds a lot of data, so do companies. And so do third sector organisations, like charities and not-for-profits.

“In just the same way you have websites for all kinds of organisations, we should be thinking about how we have open data for all kinds of organisations and how publishing open data can benefit every kind of organisation. So the most heartening thing to me about the way that it looks like the ODI will work in future is our partnerships with companies who are not just thinking about consuming open data but actually publishing open data themselves.

“And they’re doing that for a number of reasons: for transparency, for innovation, in order to communicate better with their partners and peers, and for regulation reasons. There’s a whole range of reasons why companies should open data, and to me that’s the untapped space.”

She adds, “In some ways some companies are able to move much quicker than government. I think it’s going to be very interesting to see if we can actually accelerate the amount of open data that’s available by moving into corporate open data.”

However, she says, “Pre-GDS, government transactional services had been very much behind the private sector which has pushed ahead. In the open data arena it’s actually government that’s ahead. Government has really led the way and pushed innovation.”

This article originally appeared on

What data can we best make use of - and where are the trade-offs? Photograph: Getty Images.

Charlotte Jee is a Reporter at Government Computing


Getty Images.
Show Hide image

Clive Lewis interview: I don't want to be seen as a future Labour leader

The shadow business secretary on his career prospects, working with the SNP and Ukip, and why he didn't punch a wall. 

“Lewis for leader!” Labour MP Gareth Thomas mischievously interjects minutes after my interview with Clive Lewis begins. The shadow business secretary has only been in parliament for 18 months but is already the bookmakers’ favourite to succeed Jeremy Corbyn. His self-assuredness, media performances and left-wing stances (he backed Corbyn in 2015 and again this year) have led many to identify him as Labour’s coming man.

On 19 September, I met Lewis - crop-haired, slim and wearing his trademark tweed jacket - in Westminster's Portcullis House. He conceded that he was flattered by the attention (“It’s lovely to hear”) but was wary of the mantle bestowed on him. “This place has lots of ex-would-be leaders, it’s littered with them. I don’t want to be one of those ex-would-be leaders,” the Norwich South MP told me. “I don’t want a big fat target on my head. I don’t want to cause the resentment of my colleagues by being some upstart that’s been here 18 months and then thinks they can be leader ... I’ve never asked for that. All I want to do is do my job and do it to the best of my ability.”

But he did not rule out standing in the future: “I think that anyone who comes into this place wants to do what’s best for the party and what’s best for the country - in any way that they can.”

Lewis, who is 45, was appointed to his current position in Labour’s recent reshuffle having previously held the defence brief. His time in that role was marked by a feud over Trident. Minutes before he delivered his party conference speech, the former soldier was informed that a line committing Labour to the project’s renewal had been removed by Corbyn’s office. Such was Lewis’s annoyance that he was said to have punched a wall after leaving the stage.

“I punched no walls,” he told me a month on from the speech. “Some people said to me ‘why don’t you just play along with it?’ Well, first of all it’s not true. And secondly, I am not prepared to allow myself to be associated with violent actions because it’s all too easy as a black man to be stereotyped as violent and angry - and I’m not. I’m not a violent person. Yes, it’s a bit of fun now, but very quickly certain elements of the media can begin to build up an image, a perception, a frame ... There’s a world of difference between violently punching a wall and being annoyed.”

Lewis said that he was “happy with” the speech he gave and that “you’re always going to have negotiation on lines”. The problem, he added, was “the timing”. But though the intervention frustrated Lewis, it improved his standing among Labour MPs who hailed him as the pragmatic face of Corbynism. His subsequent move to business was regarded by some as a punishment. “Do I think there was an ulterior motive? I’ll never know,” Lewis told me. “I’m confident that that the reason I was moved, what I was told, is that they wanted me to be able to take on a big portfolio”.

Nia Griffith, his successor as shadow defence secretary, has since announced that the party will support Trident renewal in its manifesto despite its leader’s unilateralism. “Jeremy Corbyn deserves credit for that,” Lewis said. “I think everyone understands that Jeremy’s position hasn’t changed. Jeremy still believes in unilateral disarmament, that is his modus operandi, that’s how he rolls and that’s one of the reasons why he is leader of the Labour Party ... But he’s also a democrat and he’s also a pragmatist, despite what people say.”

Lewis, himself a long-standing opponent of Trident, added: “You need a Labour government to ensure that we can put those nuclear missiles on the table and to begin to get rid of them on a global scale.”

He also affirmed his support for Nato, an institution which at times Corbyn has suggested should be disbanded. “The values that underpin Nato are social democratic values: liberty, democracy, freedom of expression. Let’s not forget, it was Clement Attlee and the New Deal Democrats that initiated and set up Nato. It’s about being in it to win it, it’s about winning the arguments inside Nato and making sure that it’s a force for good. Some people would say that’s impossible. I say you’ve got to be in it to be able to make those changes.”


Clive Anthony Lewis was born on 11 September 1971 and grew up on a council estate in Northampton. It was his Afro-Caribbean father, a factory worker and trade union official, who drew him to politics. “My dad always used to say “The Labour Party has fought for us, it’s really important that you understand that. What you have, the opportunities that working people and black people have, is down to the fact that people fought before you and continue to fight.”

After becoming the first in his family to attend university (reading economics at Bradford) he was elected student union president and vice president of the NUS. Lewis then spent a decade as a BBC TV news reporter and also became an army reservist, serving a tour of duty of Afghanistan in 2009. He was inspired to enlist by his grandfather. “He fought in Normandy in the Second World War and I used to go back over with him and see the camaraderie with the old paras ... Whatever people’s views of the armed forces, that’s one thing that no one can take away, they generate such friendships, such a bond of union”.

Lewis told me that his time in the military complemented, rather than contradicted, his politics. “I think many of the virtues and values of the army are very similar to the virtues and values of socialism, of the Labour Party. It’s about looking out for each other, it’s about working as a team, it’s about understanding. The worst insult I remember in the army is ‘jack bastard’. What that said was that you basically put yourself before the team, you’ve been selfish”.

He added: “People have to remember that the armed forces do as democratically elected governments tell them to do. They don’t arbitrarily go into countries and kick off. These are decisions that are made by our politicians.”

After returning from service in Helmand province, he suffered from depression. “I met guys who had lost friends, seen horrible things and they had ghost eyes, dead eyes, it’s the only way I can describe it. People that I saw had far more reason to have depression or worse. Part of my negative feedback loop was the fact that I felt increasingly guilty about being depressed because I didn’t feel that I had the right to be depressed because I knew people who’d seen far worse ...  I’m now told that is quite common but that doesn’t make it any easier.”

Lewis added: “It makes you realise that when the armed forces go abroad, when they do serve on our behalf, what they do, what they go through, that’s not something that anyone can take away from them.”

In May 2015, he was one of a raft of left-wing MPs (Richard Burgon, Rebecca Long-Bailey, Kate Osamor, Cat Smith) to enter parliament and back Corbyn’s leadership bid. As shadow business secretary, he believes that Brexit and Theresa May’s economic interventionism offer political openings for Labour. “I feel debate is moving onto natural Labour territory. But not the Labour territory of the 1970s, not picking winners territory. It’s moving to a territory that many on the left have long argued for, about having a muscular, brave, entrepreneurial state which can work in partnership with business”.

He added: “We can say we’re the party of business. But not business as usual ...  I think there are lots of people now, and businesses, who will be aghast at the shambles, the seeming direction we seem to be going in.

“The British people have spoken, they said they wanted to take back control, we have to respect that. But they didn’t vote to trash the economy, they didn’t vote for their jobs to disintegrate, they didn’t vote to see their businesses decimated, they didn’t vote to see a run on the pound, they didn’t vote for high levels of inflation.”

On the day we met, an Ipsos MORI poll put the Tories 18 points ahead of Labour (a subsequent YouGov survey has them 16 ahead). “I’m not too spooked by the polls at the moment,” Lewis told me when I mentioned the apocalyptic figures (he has a potentially vulnerable majority of 7,654). “Nobody wants to be where we are but I’m quite clear that once we get up a head of steam we’ll begin to see that narrow. I definitely don’t have any doubts about that, it will begin to narrow.”

Lewis is a long-standing advocate of proportional representation and of a “progressive alliance”. He told me that Labour, the Liberal Democrats and the Green Party should have fielded a single pro-European candidate in the recent Witney by-election (which the Conservatives won with a reduced majority) and that he was open to working with the SNP.

“There are lots of people, including the Scottish Labour Party, who are aghast that you can say that. I think it has to be put out there. I want to see a revival of Scottish Labour but we also have to be realistic about where they are, the time scale and timeframe of them coming back.

“I’m not talking them down, I’m simply saying that we want to see a Labour government in Westminster and that means asking some hard questions about how we’re going to achieve that, especially if the boundary changes come in ... If that means working with the SNP then we have to look at that.”

Even more strikingly, he suggested that Labour had to “think about talking to parties like Ukip to try and get over that finishing line.”

Lewis explained: “If Ukip survive as a political force these coming weeks and months they’re obviously pro-PR as well. I despise much of what Ukip stand for, it’s anathema to me, but I also understand that it could be the difference between changing our electoral system or not ... These are things that some people find deeply offensive but I’ve not come into politics to duck the tough issues." 

He praised Corbyn for “having won” the argument over austerity, for his “dignified” apology over the Iraq war and for putting Labour in surplus (owing to its near-tripled membership of 550,000).

“History will show that Jeremy Corbyn was someone who came in at a time when politics was tired, people were losing faith in it, especially people who come from the progressive side of politics.

“Whatever people think of Jeremy’s style, whatever they think of his leadership, whatever they think of him personally, you can’t take that away from him. He’s revived politics in a way that we haven’t seen in this country for a long time. I know he’s got his doubters and detractors but I think ultimately he’s made our party in many ways stronger than it was a year ago.”

I asked Lewis whether he expected Corbyn to lead Labour into the next general election. “Yes, I do. And I think it depends when that general election is. If it’s next year then most certainly.

“If it’s 2020? That’s a question for Jeremy. I think, as I understand it, he is going to but I don’t know the inside of his mind, I don’t know what he’s thinking. I haven’t heard anything to suggest that he has anything other than the intention to lead us into a general election and to become prime minister.”

Of his own prospects, he remained equanimous. “Always be wary of Greeks bearing gifts. It’s lovely to hear but I know my own fallibilities and weaknesses.

“I haven’t come from a background where I’ve had it imbued in me from an early age that I’m destined to lead or to rule. I don’t have that arrogant self-belief, the sense of entitlement that it’s coming my way or should do. I can’t believe I’m in the House of Commons and I can’t believe that I’m shadow business secretary. I still pinch myself. That’s enough for me at the moment, it really is. That’s the honest truth.”

George Eaton is political editor of the New Statesman.