Jack Straw's secret is now thrillingly out in the open. Against all expectations, the new draft freedom of information bill is the most radical departure this government has yet undertaken. Straw and his advisers have rejected the old and apparently bold white paper he took over from David Clark as being too 20th century for the 21st century we are about to enter.
Straw's bill, introducing what the Home Secretary calls "open-book government", will give everyone in reach of a PC, a digital TV or a telephone direct interactive access to policy information and their own personal files. Central government, local authorities, all public bodies and other significant institutions in modern Britain will have to adopt the Home Secretary's vision.
So anyone trying to piece together official policies and data will be able to do it for themselves, without relying on official gatekeepers in government or media interpretations of the news. Everyone dealing with a government department, local council or other public authority - over, say, their income tax, an immigration decision, a pension or welfare benefit - will be able to discover how their own case is being handled. Using a personal identification number, they will be able to find what stage their application has reached and who is handling it, without having to phone officials or visit government offices to find out what is happening.
Straw's bill includes funding to provide PC access points in all major public libraries, town halls and neighbourhood offices, social security offices, citizen's advice bureaux and other advice agencies.
In our dreams. Unfortunately, Straw adheres firmly to mid- century beliefs about the role of the state. The man (or woman) in Whitehall may no longer know best, but much of what he or she knows is best kept secret, and especially so if they are in the Home Office, the security forces or the police. So freedom of information legislation will probably be a far more conservative version of Clark's white paper plans, and will make little or no reference to the potential for electronic transparency and accountability.
But we should not assume that ministers can hold back the tide indefinitely. Government departments are constantly compared with private corporations in how they handle inquiries, complaints and access to information. If you send a parcel overseas via DHL, you get a code number that allows you to access the company's databases and track its progress, and if the parcel is going the wrong way, a simple e-mail to DHL ensures that it is redirected correctly and you are compensated. If these service standards become commonplace in the private sector, government can hardly stand aloof.
People who pay tax, who want to track a relative's application to enter Britain, or who apply (electronically) for a state benefit will increasingly expect to receive an immediate code reference - and to be able to dial up and track the progress of their "transaction". Claimants need never again be told that their giro is "in the post". It won't matter if a tax inspector is in or out of the office, you should still be able to find out exactly what is going on.
What we are talking about here is "open-book government". And it would reveal much more information about how government operates, how complex it is and how arcane many processes are. It would thus surely lead to better government.
This is the potential that should be driving reform. But what are the obstacles? First, the official mindset regards new information technology at best as a faster and cheaper means of doing what has always been done. When government systems were first computerised they were not "re-engineered" to make them simple and efficient - they were just shifted over as they were. The Conservative government's passion for contracting out government IT systems made things worse - civil servants no longer fully understand their own departments' systems, let alone know how to restructure them. Rather than put in new investment or system renewals, the government simply passed the mess on to the major IT companies.
Second, Whitehall by and large still regards the web as a small-scale change, a new opportunity for PR, a convenient place to store ministers' biographies, current departmental press releases and odd bits of public information, updated once or twice every three years. Despite pressure from Alastair Campbell's strategic communications unit for a unified "front end" for British government as a whole, government on the web remains an eclectic mess of different departments' and agencies' approaches. Whitehall cannot even get its act together to give the full information on the quangocrats who run the quango state, as recent Democratic Audit studies have shown. Yet that was one of Labour's pre-election pledges.
The existing government websites (and more so, local authority sites) are virtually all stand-alone, discrete operations - insulated from any connection to the actual working databases of departments and agencies. Whitehall officials react with horror to the idea that citizens might want to get beyond a public relations annex and gain access to core systems themselves. But that is what "open-book" systems are going to mean more and more in the private sector. So-called "web-enabled systems" are being used by corporations to create what they term "zero-touch" processes, which allow customers to access files electronically, place orders, pay and receive goods automatically - without a single employee so much as touching a keyboard. All this requires that web access goes straight into the company's main working systems. Sophisticated security systems protect companies' confidential information, rather than crude physical separation, as in Whitehall. The potential costs of freedom of information already trouble civil servants, but open-book government promises a self-funding, or even possibly a cost-saving, future for Whitehall.
This is the nub of it. Standard-issue freedom of information would make the quangos, as well as government at all levels, more transparent. But citizens would still have no access to meetings, to inspect minutes and verbatim transcripts of meetings, to know what meetings take place with interest groups, and so on. Add open-book government to the formula and suddenly the whole process moves far faster. The policies and decisions of the expert committees that rule on the safety of food and drugs, and control genetic experiments in the environment, hazardous substances, nuclear activities and waste, would at once be made open to peer-group review and public debate. The secret dealings that government departments and other authorities, including local councils, have with major national and local interest groups could be weighed against wider concepts of public consultation and interest.
Open-book government could also form a significant part of a new era of "electronic democracy". It need not simply improve access and transparency to government, but it could also help give new life and meaning to representative democracy as we know it. The same electronic advances could make public consultation and participation wider, easier and more diverse; and provide new media opportunities which could both focus and diversify the information people receive and obtain for themselves, as the old media fragment into more and more apolitical and specialised forms - sports channels, gardening channels, fashion channels, golf channels and so on.
A wonderful example of the potential here was the BBC's Election 97 website, which on election night itself recorded more than 1.5 million "hits". During the election, the website not only provided far more reliable basic information than any conventional mass media source, but it also allowed people to e-mail queries and get answers. As supposed experts in politics, we were stunned by the quality of the questions submitted, the insights they contained and the appetite for information and debate that they revealed - party policies, opinion polls, electoral trajectories and key issues were clarified and debated in depth. The site earned the BBC great credit. But not the aftermath. The site was briefly made permanent as "Politics 97", but has since been swept away in a Birtian re-modelling, cost-cutting and dumbing down of the BBC's entire web output.
Yet this and other innovations have shown the enormous potential for greater "discursive democracy". Government departments, local councils and other public bodies can make clear how they shape their policies and invite interested citizens and specialists to participate directly in determining them. Interactive question-and-answer sessions, policy forums, panels and discussion groups, planning consultations, chat-lines, even tabloid-style votes can all generate a great deal more information that policy-makers should consider. They could also give far more in-depth information more cheaply and conveniently, respond to people's questions and ideas and encourage the public to submit proposals for action.
Already the BSE inquiry has shown the way, transferring daily transcripts of its proceedings to the web within hours of witnesses having spoken. The potential for other public inquiries, parliamentary select committees and other central government bodies to do the same is immense. And at local level the possibilities for handling local planning decisions, plans for reorganising secondary schools or healthcare proposals are endless.
Electronic democracy opens up bigger questions about the relationship between politics and the media. Some commentators insist that the media perform an independent role, while critics allege that they are subject to a variety of pressures, including state dominance and political spin. But it is not too cynical to state that newspaper, TV and radio coverage is always mediated by forces and powers that are far from transparent; that the media reinforce powerful views and interests rather more often than they interrogate them; and that they seek to entertain rather than to inform.
Ten years from now, the media will be highly diversified, television viewers will be spoilt for choice, and the press will be a very special product indeed. But the new electronic media just might offer much more unmediated information - just as some television time will give raw access not only to major news dramas but also more specialised political and social happenings. Sky, for example, made a feature of unedited party political news conferences at the last election.
The government has a duty to kick-start the momentum for more diversified citizen access by introducing genuinely radical freedom of information laws and ending the era of closed-book government. The pressures for democratisation are running strongly in parallel with the desire for more efficient and effective government. Will new Labour seize the moment?
Patrick Dunleavy is professor of government at the London School of Economics; Stuart Weir is director of the Democratic Audit, University of Essex