Opening up government

Jeni Tennison, technical director of the Open Data Institute, discusses progress on releasing data, how it feeds into wider Whitehall technology reform and why companies should follow the government's lead.

It’s three years since the launch of data.gov.uk and we now have access to datasets covering topics as diverse as HS2, broadband and GP prescriptions. The UK has won many plaudits as a result and is often touted as a world leader in the field of open data.

Time then, perhaps, to give some credit to the people behind these achievements, one of whom is Jeni Tennison.

Tennison has a number of roles: primarily as technical director of the Open Data Institute, but she also sits on the Cabinet Office’s Open Data User Group (ODUG) and the Government Digital Service’s Open Standards Board, to name but a couple.

Tennison agrees that major progress has been made. She says:

We’re getting to a stage now in the UK where, happily, a lot of data that can easily be released has already been published.

However, she is far from complacent. Tennison says, “The challenges are around data that is still being made available for money, where a business case has to be made to demonstrate that there is wider economic benefit against the short-term cost of losing that revenue. There are also challenges around the formats in which information is made available and the regularity and guarantees around that data.”

She explains that a key part of ODUG’s work is representing user requirements for data into government and helping to prioritise datasets for release.

Tennison adds, “It’s just trying to unpick those requirements and represent them into government. Obviously we’ve also had campaigns that ODUG has run around open address information, and ongoing discussions with Ordnance Survey over its licensing terms for example.

“So basically ODUG is a campaigning body that’s close to government and that tries to represent the need for data, spanning a broad range of different interests including big businesses, small businesses and civil society.”

Privacy trade-offs

Regarding concerns raised about open data, particularly around security and privacy, Tennison explains, “I think that you'll find that within the open data community there is a very strong recognition of the need to protect privacy but it’s more complicated than baldly saying 'open data is not personal data'. There are grey areas where personal information needs to be anonymised or aggregated in order to provide the benefits of openness without intruding on people's lives.

“And we're doing that very slowly, very gradually and are very much aware of the risks. The worst thing that could possibly happen for the open data community would be for some personal data to be released under the open data agenda because it would set it back. So it's not in anyone's interest. It's something we feel very strongly about in fact.”

She adds, “It's not 'everything should be out there', it's much more 'what data can we best make use of', and where are the trade-offs?”

As an example of these concerns in practice, Tennison points to a recent HM Revenue & Customs consultation about the VAT register, which contains information on businesses.

However, she explains:

A large proportion of businesses in the VAT register, especially if they aren't in Companies House already, will be named individuals as sole traders. So there is personal information within the register, certainly personally identifiable information.

On the other hand there's a big transparency and open data story about being able to get VAT registration information because it helps you to tie up all of the spending that local government does and central government does with particular companies. So there are lots of benefits economically, and in terms of service delivery and transparency, of making that information available. But then you have this personal data. So you have to balance these up.

And the argument that we made in response to that was that there was a subset of fields that didn't reveal too much but provided enough information to be useful for the 80% of things that didn't need all of this information to be made available. And so, just opening that up would be sufficient to address most of the requirements. It doesn't address all of them but in this case that was the right place to draw the line.

Tennison suggests that organisations follow ICO (Information Commissioner’s Office) advice, which says “it’s open by default, but if you have personal information in your data then you should conduct a Privacy Impact Assessment: you consult with any stakeholders that would be affected by revealing this information, and you do a considered analysis of what the impact will be of releasing that data.

“And when you've done that considered analysis then you may find there are certain bits that shouldn't be released, other bits that can be released, and it allows you to make those trade-offs in a risk-assessment-like process.”

Expanding the audience

Tennison says that one of her worries is about trying to explain what open data means for the general public. She says, “People often don’t know they want open data until they need it, for example about their local schools and hospitals.”

“Certainly one of the things that we concentrate really strongly on at the ODI is trying to expand the set of people that think about and understand what open data is and what it means beyond the existing open data community and the geeks. And the kind of technique that we're using at the moment is lots of storiesto try to bring home how open data enables new things to happen.”

Tennison is also keen to expand an open data ethos throughout Whitehall. She regularly visits the Government Digital Service (GDS) for meetings of the Open Standards Board, but recently did a presentation during one of their monthly meetings.

She says, “The pitch that I made to GDS was that open data should be very much at the heart of what they are doing, and they get that. It’s been very hard for them because they’ve had to focus on getting GOV.UK running, getting the government departments in, and now getting the public sector agencies in too.

“They are starting now, really excitingly, to look more at how they make data available as open data and to build that into their thinking. GDS should be one of the greatest beneficiaries of open data from the public sector because they should be able to build any open data that is available into their own services and the transactions that they are supporting.

“And so I’d really like GDS to be making a stronger stand around asking for the data that they need to be made available as open data by departments. To be asking for that themselves. They’re beginning to do some really great things. They are starting to build in micro APIs to the GOV.UK back end, so if you’re building an application that depends on the VAT rate, you can have the API call that goes and gets whatever the current VAT rate is and uses that.

“That’s always been what they intended to do at GDS: making it wholesale, enabling other people to build in services and data that they’re making available. So it’s really good to see them starting to move in that direction and I’m expecting great things from them.”

Making government more open

Tennison explains that ‘opening up’ technology is crucial if the government is to succeed in its plans to reform technology and break away from a reliance on a small pool of suppliers.

She says, “What I think is really interesting here is how open source, open standards and open data all come together to support the same agenda. Government doesn’t want to be locked into massive IT contracts with a small set of suppliers. “Open" helps because when you have open data being published then you have a level playing field about what people know. When you have open standards being used then you have a level playing field about what tools can be used. And when you have open source established you have a level playing field about really just getting started with using that data and using those standards.

“We’re not going to see the effect [of the reforms] until the big contracts start coming to an end, but they are laying the ground work. We can’t think that it’s job done. If you look at Dun& Bradstreet identifiers and the ACORN classification, you get those being built into IT systems and that forces a reliance on one supplier. These things have been built into the heart of some of the IT systems and processes and metrics and everything that is being used within government in order to make decisions. You’re completely locked in.

“And the other thing that you have to be extremely careful about within government is ‘open-washing’ that sometimes goes on. So, for example, you will get an API that is described as an ‘open API’ where in fact it doesn’t mean that the data is open data or using an open standard. The underlying standard that is used is open but the actual language, the vocabulary itself, is developed by one company and only one company uses it.

“So there are lots of things to be careful of. And of course it’s completely understandable that it’s in a company’s interest to try and keep people locked into using them and they’re going to keep on trying to get people locked into using them, because that’s what companies do. The government initiatives around open source, open standards and open data are so important because it’s only those that push back. It’s only those that enable you to stop that kind of thing from happening.”

Moving beyond the public sector

Regarding the ODI’s future, Tennison says, “A really important role that ODI will have over the next few years is about moving the open data community outside the government data space. Because although government holds a lot of data, so do companies. And so do third sector organisations, like charities and not-for-profits.

“In just the same way you have websites for all kinds of organisations, we should be thinking about how we have open data for all kinds of organisations and how publishing open data can benefit every kind of organisation. So the most heartening thing to me about the way that it looks like the ODI will work in future is our partnerships with companies who are not just thinking about consuming open data but actually publishing open data themselves.

“And they’re doing that for a number of reasons: for transparency, for innovation, in order to communicate better with their partners and peers, and for regulation reasons. There’s a whole range of reasons why companies should open data, and to me that’s the untapped space.”

She adds, “In some ways some companies are able to move much quicker than government. I think it’s going to be very interesting to see if we can actually accelerate the amount of open data that’s available by moving into corporate open data.”

However, she says, “Pre-GDS, government transactional services had been very much behind the private sector which has pushed ahead. In the open data arena it’s actually government that’s ahead. Government has really led the way and pushed innovation.”

This article originally appeared on governmentcomputing.com

What data can we best make use of - and where are the trade-offs? Photograph: Getty Images.

Charlotte Jee is a Reporter at Government Computing

 

Getty
Show Hide image

Love a good box set? Then you should watch the Snooker World Championships

The game relies on a steady arm, which relies on a steady nerve. The result is a slow creeping tension needs time and space to be properly enjoyed and endured. 

People are lazy and people are impatient. This has always been so – just ask Moses or his rock – but as illustrated by kindly old Yahweh, in those days they could not simply answer those impulses and stroll on.

Nowadays, that is no longer so. Twitter, YouTube and listicles reflect a desire for complex and involved issues, expansive and nuanced sports – what we might term quality – to be condensed into easily digestible morsels for effort-free enjoyment.

There is, though, one notable exception to this trend: the box set. Pursuing a novelistic, literary sensibility, it credits its audience with the power of sentience and tells riveting stories slowly, unfolding things in whichever manner that it is best for them to unfold.

In the first episode of the first series of The Sopranos, we hear Tony demean his wife Carmela's irritation with him via the phrase “always with the drama”; in the seventh episode of the first series we see his mother do likewise to his father; and in the 21st and final episode of the sixth and final series, his son uses it on Carmela. It is precisely this richness and this care that makes The Sopranos not only the finest TV show ever made, but the finest artefact that contemporary society has to offer. It forces us to think, try and feel.

We have two principal methods of consuming art of this ilk - weekly episode, or week-long binge. The former allows for anticipation and contemplation, worthy pursuits both, but of an entirely different order to the immersion and obsession offered by the latter. Who, when watching the Wire, didn’t find themselves agreeing that trudat, it's time to reup the dishwasher salt, but we’ve run out, ain’t no thing. Losing yourself in another world is rare, likewise excitement at where your mind is going next.

In a sporting context, this can only be achieved via World Championship snooker. Because snooker is a simple, repetitive game, it is absorbing very quickly, its run of play faithfully reflected by the score.

But the Worlds are special. The first round is played over ten frames – as many as the final in the next most prestigious competition – and rather than the usual week, it lasts for 17 magical days, from morning until night. This bestows upon us the opportunity to, figuratively at least, put away our lives and concentrate. Of course, work and family still exist, but only in the context of the snooker and without anything like the same intensity. There is no joy on earth like watching the BBC’s shot of the championship compilation to discover that not only did you see most of them live, but that you have successfully predicted the shortlist.

It is true that people competing at anything provides compelling drama, emotion, pathos and bathos - the Olympics proves this every four years. But there is something uniquely nourishing about longform snooker, which is why it has sustained for decades without significant alteration.

The game relies on a steady arm, which relies on a steady nerve. The result is a slow creeping tension needs time and space to be properly enjoyed and endured. Most frequently, snooker is grouped with darts as a non-athletic sport, instead testing fine motor skills and the ability to calculate angles, velocity and forthcoming shots. However, its tempo and depth is more similar to Test cricket – except snooker trusts so much in its magnificence that it refuses to compromise the values which underpin it.

Alfred Hitchcock once explained that if two people are talking and a bomb explodes without warning, it constitutes surprise; but if two people are talking and all the while a ticking bomb is visible under the table, it constitutes suspense. “In these conditions,” he said, “The same innocuous conversation becomes fascinating because the public is participating in the scene. The audience is longing to warn the characters on the screen: ‘You shouldn't be talking about such trivial matters. There is a bomb beneath you and it is about to explode!’”

Such is snooker. In more or less every break, there will at some point be at least one difficult shot, loss of position or bad contact – and there will always be pressure. Add to that the broken flow of things – time spent waiting for the balls to stop, time spent prowling around the table, time spent sizing up the table, time spent cleaning the white, time spent waiting for a turn – and the ability for things to go wrong is constantly in contemplation.

All the more so in Sheffield’s Crucible Theatre. This venue, in its 40th year of hosting the competition, is elemental to its success. Place is crucial to storytelling, and even the word “Crucible” – whether “a ceramic or metal container in which metals or other substances may be melted or subjected to very high temperatures,” “a situation of severe trial”, or Arthur Miller’s searing play – conjures images of destruction, injustice and nakedness. And the actual Crucible is perhaps the most atmospheric arena in sport - intimate, quiet, and home to a legendarily knowledgeable audience, able to calculate when a player has secured a frame simply by listening to commentary through an earpiece and applauding as soon as the information is communicated to them.

To temper the stress, snooker is also something incredibly comforting. This is partly rooted in its scheduling. Working day and late-night sport is illicit and conspiratorial, while its presence in revision season has entire cohorts committing to “just one more quick frame”, and “just one more quick spliff”. But most powerfully of all, world championship snooker triggers memory and nostalgia, a rare example of something that hasn’t changed, as captivating now as it was in childhood.

This wistfulness is complemented by sensory pleasure of the lushest order. The colours of both baize and balls are the brightest, most engaging iterations imaginable, while the click of cue on ball, the clunk of ball on ball and the clack of ball on pocket is deep and musical; omnipresent and predictable, they combine for a soundtrack that one might play to a baby in the womb, instead of whale music or Megadeth.

Repeating rhythms are also set by the commentators, former players of many years standing. As is natural with extended coverage of repetitive-action games, there are numerous phrases that recur:

“We all love these tactical frames, but the players are so good nowadays that one mistake and your opponent’s in, so here he is, looking to win the frame at one visit ... and it’s there, right in the heart of the pocket for frame and match! But where’s the cue ball going! it really is amazing what can happen in the game of snooker, especially when we’re down to this one-table situation.”

But as omniscient narrators, the same men also provide actual insight, alerting us to options and eventualities of which we would otherwise be ignorant. Snooker is a simple game but geometry and physics are complicated, so an expert eye is required to explain them intelligibly; it is done with a winning combination of levity and sincerity.

The only essential way in which snooker is different is the standard of play. The first round of this year’s draw featured eight past winners, only two of whom have made it to the last four, and there were three second-round games that were plausible finals.

And just as literary fiction is as much about character as plot, so too is snooker. Nothing makes you feel you know someone like studying them over years at moments of elation and desolation, pressure and release, punctuated by TV confessions of guilty pleasures, such as foot massages, and bucket list contents, such as naked bungee jumping.

It is probably true that there are not as many “characters” in the game as once there were, but there are just as many characters, all of whom are part of that tradition. And because players play throughout their adult life, able to establish their personalities, in unforgiving close-up, over a number of years, they need not be bombastic to tell compelling stories, growing and undergoing change in the same way as Dorothea Brooke or Paulie Gualtieri.

Of no one is this more evident that Ding Junhui, runner-up last year and current semi-finalist this; though he is only 30, we have been watching him almost half his life. In 2007, he reached the final of the Masters tournament, in which he faced Ronnie O’Sullivan, the most naturally talented player ever to pick up a cue – TMNTPETPUAC for short. The crowd were, to be charitable, being boisterous, and to be honest, being pricks, and at the same time, O’Sullivan was playing monumentally well. So at the mid-session interval, Ding left the arena in tears and O’Sullivan took his arm in consolation; then when Ding beat O’Sullivan in this year’s quarter-final, he rested his head on O’Sullivan’s shoulder and exchanged words of encouragement for words of respect. It was beautiful, it was particular, and it was snooker.

Currently, Ding trails Mark Selby, the “Jester from Leicester” – a lucky escape, considering other rhyming nouns - in their best of 33 encounter. Given a champion poised to move from defending to dominant, the likelihood is that Ding will remain the best player never to win the game’s biggest prize for another year.

Meanwhile, the other semi-final pits Barry Hawkins, a finalist in 2013, against John Higgins, an undisputed great and three-time champion. Higgins looks likely to progress, and though whoever wins through will be an outsider, both are eminently capable of taking the title. Which is to say that, this weekend, Planet Earth has no entertainment more thrilling, challenging and enriching than events at the Crucible Theatre, Sheffield.

0800 7318496