Open data: beware the empire striking back, insiders warn

Two central figures of the UK government's initiative to open its data for re-use have warned of the risk of ministers being thwarted by reluctant or apathetic bureaucrats. "There is a real risk of the empire striking back," Andrew Stott, the government's first director of digital engagement and a former deputy chief information officer, said last week.

Stott said that, despite the government's promise to introduce a new "right to data", the campaign for open data is not over yet.

He told the Open Government Data Camp, an event in Warsaw hosted by the Open Knowledge Foundation, that the civil service has a wealth of tricks for diluting the coalition government's pledge to make data available. "We're seeing a move from Bureaucrat 1.0 - the straight 'No Minister' - to Bureaucrat 2.0. That's the one who says 'Yes Minister' but then quietly fails to execute the plan."

One trick, said Stott, is to change the definition of open data and say "we're free already". This the the claim made by Royal Mail for the Postcode Address File - however it is free only to people looking up fewer than 15 postcodes a day for personal purposes, he said.

Other tricks include requiring data users to register at websites on the grounds that "We need to keep users informed", scripting log-ins so that machines can't do it and requiring users to pass through a landing page "with a lot of guff about the department "on the grounds that "people need to understand the context".

Stott also warned that "corporate IT is starting to strike back" - with departments refusing to open data until they have a "proper system". One department had promised to make data available from a single integrated data system - planned to be running in 2017.

Another excuse, he said, was that "no one's really interested". He warned that transparency data tends to disappear from official websites. "You've got to watch that like a hawk."

Finally, he said, bureaucracies will argue that "killer apps" will use only a small proportion of the data available so "it's better to wait to be asked" rather than pro-actively releasing data. "We've got to track performance like hawks, challenge deficiencies," he said.

At the same meeting, Professor Nigel Shadbolt of Southampton University and a member of the UK Transparency Board said the campaign still had a long way to go. He singled out the Meteorological Office for failing to embrace the new Open Government Licence for re-use of data.. "You can't get primary weather data in the UK."

He urged anyone with an interest in the subject to respond to the Cabinet Office's two current consultations, on making open data real and the policies of the new Public Data Corporation. Both close this week.

"The value of data is in its use, not its sale," Shadbolt said.