Open data and all that

Posts Tagged ‘gov 2.0

Open Data: A threat or saviour for democracy?

with 2 comments

This is my presentation to the superb OKCON2011 conference in Berlin last week. It’s obviously openly licensed (CC-BY), so feel free to distribute widely. Comments also welcome.

Written by countculture

July 4, 2011 at 10:30 am

George Osborne’s open data moment: it’s the Treasury, hell yeah

with 2 comments

As a bit of an outsider, reading the government’s pronouncements on open data feels rather like reading official Kremlin statements during the Cold War. Sometimes it’s not what they’re saying, it’s who’s saying it that’s important.

And so it is, I think, with George Osborne’s speech yesterday morning at Google Zeitgeist, at which he stated, “Our ambition is to become the world leader in open data, and accelerate the accountability revolution that the internet age has unleashed“, and “The benefits are immense. Not just in terms of spotting waste and driving down costs, although that consequence of spending transparency is already being felt across the public sector. No, if anything, the social and economic benefits of open data are even greater.

This is strong, and good stuff, and that it comes from Osborne, who’s not previously taken a high profile position on open data and open government, leaving that variously to the Cabinet Office Minister, Francis Maude, Nick Clegg & even David Cameron himself.

It’s also intriguing that it comes in the apparent burying of the Public Data Corporation, which got just a holding statement in the budget, and no mention at all in Osborne’s speech.

But more than that it shows the Treasury taking a serious interest for the first time, and that’s both to be welcomed, and feared. Welcomed, because with open data you’re talking about sacrificing the narrow interests of small short-term fiefdoms (e.g. some of the Trading Funds in the Shareholder Executive) for the wider interest; you’re also talking about building the essential foundations for the 21st century. And both of these require muscle and money.

It also overseas a number of datasets which have hitherto been very much closed data, particularly the financial data overseen by the Financial Services Authority, the Bank of England and even perhaps some HMRC data, and I’ve started the ball rolling by scraping the FSA’s Register of Mutuals, which we’ve just imported into OpenCorporates, and tying these to the associated entries in the UK Register of Companies.

Feared, because the Treasury is not known for taking prisoners, still less working with the community. And the fear is that rather than leverage the potential that open data allows for a multitude of  small distributed projects (many of which will necessarily and desirably fail), rather than use the wealth of expertise the UK has built up in open data, they will go for big, highly centralised projects.

I have no doubt, the good intentions are there, but let’s hope they don’t do a Team America here (and this isn’t meant as a back-handed reference to Beth Noveck, who I have a huge amount of respect for, and who’s been recruited by Osborne), and destroy the very thing they’re trying to save.

Written by countculture

May 17, 2011 at 2:27 pm

What’s that coming over the hill, is it… the Public Data Corporation?

with 10 comments

A couple of days ago, there was a brief announcement from the UK Government of plans for a new Public Data Corporation, which would “bring together Government bodies and data into one organisation”.

A good thing, no? Well, up to a point, Lord Copper.

I tweeted after the announcement: “Is it just me, or does the tone of the Public Data Corp make any other #opendata types uneasy?” From the responses, I clearly wasn’t the only one, and in my discussions since then it’s clear there’s a lot of nervousness out there.

So, what is it, and should we be afraid? The answers are ‘Nobody knows’, and ‘Yes’.

To flesh that out a bit, none of the open data activists and developers that I’ve spoken to knows what it is, or what the real motivation is, and remember these are the people who did much to get us into a place where the UK government has declared that the public has a ‘Right To Data’ and that the excellent ‘Open Government Licence‘ should be the default licence.

In that context, the announcement of a ‘Public Data Corporation’ should be be treated with some wariness.

However, this wariness turns into suspicion, when you read the press release.

First the announcement is a joint one from the Cabinet Office minister Francis Maude (who seems to very much get the need for open public data in the changed world in which we live) and from Business Minister Edward Davey, who I know nothing about, but his department BIS (Dept of Business, Innovation & Skills) has very much not been pushing for open data, and in fact  has in the past refused to make data it oversees openly available.

(My sources tell me the proposal in fact originated from BIS, and thus could be seen as an attempt by the incumbents to co-opt the open data agenda, as a way of shutting it down, smothering it if you like.)

Second, despite the upbeat headline “Public Data Corporation to free up public data and drive innovation” (Shock horror: org states its aim is to innovate & be successful), the text contains a number of worrying statements:

  • By bringing valuable Government data together, governed by a consistent set of principles around data collection, maintenance, production and charging[my emphasis], the Government can share best practice, drive efficiencies and create innovative public services for citizens and businesses. The Public Data Corporation will also provide real value for the taxpayer.
    The idea of ‘value for the taxpayer’ is the same old stuff that got us into the unholy mess of trading funds, and the gordian knot of the Ordnance Survey licence wich is still being unpicked. This nearly always translates as value we can measure in £s, which in turn means what income we’ve got coming in (even if it’s from other public sector bodies).
  • “It will provide stability and certainty for businesses and entrepreneurs, attracting the investment these operations need to maintain their capabilities and drive growth in the economy” – quote from Edward Davey.
    If I were a cynic I’d say stability and certainty translates to stagnation and rent-seeking businesses, which may be music to civil servants’ ears but does nothing to help innovation. We’re in a rapidly changing world. Get over it.
  • “bringing valuable Government data together, governed by a consistent set of principles around data collection, maintenance, production and charging”.
    If this is the PDC’s mandate I think it could end up focused on the last of these, short-sighted though that would be.
  • It will also provide opportunities for private investment in the corporation.”
    Great. A conflicting priority, to delight the bureaucrats and muddy the focus. Keep it small, keep it simple, keep it agile.

Finally, there’s no mention of open data, no mention of the Open Government Licence, the Transparency Board and only one mention of transparency, and that’s in Francis Maude’s quote.

As Tom Steinberg (a member of the Transparency Board) wrote in a thread about the PDC on MySociety developer mailing list ”

If you’re a natural cynic, you’ll just say the government has already decided to flog everything off to the highest bidder. If you adopt that position, and give up without a fight, the people in Whitehall and the trading funds who want to do that will almost certainly win.

However, if you believe me when I say things are finely balanced, that either side could win, and enough well-organised external pressure could really make a difference over the next year, then you won’t just bitch, you’ll get stuck in.

He’s not wrong there. We’ve got perhaps 6 months to make this story turn out good for open data, and good for the wider community, and I suspect that means some messy battles along the way, forcing government to take the right path rather than slide into its bad old habits, perhaps with some key datasets, which should undoubtedly be public open data, but are currently under a restrictive licence.

I’ve got a couple in my sights. Watch this space.

Written by countculture

January 14, 2011 at 2:04 pm

Right to Government Data… unless it’s been outsourced

with 5 comments

The UK’s coalition government has followed up on its promises of a general Right To Public Data, with sweeping and fast-moving measures to open up whole sectors of public sector information, from the salaries of senior civil servants to spending by local councils.

For those who attended the pre-election Post-Bureaucratic Age (PBA) conference (which was heavily populated by Conservative thinkers), this is not such a surprise, and it’s also worth acknowledging the steps the previous government took – principally making the decision to open up Ordnance Survey data.

But, there’s one thing that nobody’s talking about, and significantly it’s a question that was asked (but ducked) at the final PBA panel: what about government data that happens to be held by the private sector, principally information that’s been outsourced?

Now lest anyone think this is nitpicking here, we’re not talking about the odd dataset here and there – whole areas of government work, both national and local, have been outsourced to the private sector, including the almost the complete frontline services for some councils.

The problem with this – as far as information goes – is that private sector companies aren’t covered by the same regulations as public sector bodies. They aren’t, for example, subject to FoI requests, even if they are doing the work that would have been done by the private sector (actually it’s much worse than this: even if companies are 90% owned by a public body they are not subject to FoI requests, nor, currently are Joint Ventures between public bodies).

The same problem looks likely to afflict the right to data, with no Right To Data held by private companies (or JVs) on behalf of public bodies.

[I should state here, that I’m no fan of outsourcing – buying services as a commodity, yes; outsourcing no – for more on this see my presentation Open Data & The Rewards of Failure. Too often it’s an unequal balance, with the contractor knowing way more than the purchaser, and often is carried out by companies just as bureaucratic as public bodies.]

A current example portal

Enough about the generalities. Let’s look at a specific case – one that was raised at the EPSI conference in Madrid last week, by Jose L. Marin from Euroalert. Turns out they’d been trying to get access to the information from the government tender portal, which is run on behalf of the Department for Business, Innovation and Skills (BIS) by BiP Solutions. In fact, according the the website, BiP Solutions are [merely] the service provider for the portal and are… responsible for its development, support and maintenance.

There’s clearly a case for making this information as widely available as possible, as you want the public bodies to have as good a choice as possible.

However, despite their best efforts, Euralert have not had any luck getting access to the data. They’d even enlisted the help of the good people from OPSI, who are tasked with facilitating access to public data. But even this didn’t work, as last week they received an email from OPSI, with this response from BIS.

As you are aware, both BIS and BiP have refused Mr. Marin’s request. There are both policy and commercial reasons why this decision has been made.

It is BIS’ view that the Re-use of Public Sector Information Regulations 2005 are not applicable in the circumstances. The information which Mr. Marin is seeking is not “held by a public sector body” as required by Regulation 4; BIS is a public sector body, but does not hold the information, BiP holds the information, but is not a public sector body. Furthermore, Regulation 7 makes it clear that decisions on re-use are a discretionary matter for the public sector body concerned. In the event that we considered that the Regulations did apply, we would nonetheless have still decided not to permit the re-use Mr. Marin requests for policy and commercial reasons.

In the alternative, if Mr. Marin considers that the information on Supply2Gov is held by the public sector bodies which placed the contract opportunities on the site, then the request for re-use in relation to each contract opportunity should be made to the public sector body concerned, not to BIS, which does not hold the information.

There’s something rather patronising and condescending about this response that I find quite appalling. In short, we don’t want you to have the data, and because it’s held by a private company there’s nothing you can do about it, and frankly you wouldn’t understand the reasons even if we could be bothered to tell you. In other words (and in the immortal words of the French soldier from Monty Python & The Holy Grail): ‘I fart in your general direction.

It seems clear to me that if the coalition is genuinely committed to open data, it will stamp out this sort of thing immediately – it’s a concern that the letter was sent post-election, and from a department headed by Liberal Democratic Vince Cable.

But whichever way it goes, make no mistake, if we don’t get the right to reuse public data that has been outsourced to the private sector, we will not only be missing out on the benefits of open data, but I suspect we will also see the most politically embarrassing data being outsourced just to hide it from public view.

Update: Just as I finished writing this I saw via Twitter that the Office of Government Commerce was being taken away from the BIS and moved to the Cabinet Office, home to many of the best people in the UK Government helping open up data. Given that the OGC is responsible for tendering/procurement it ‘s to be hoped that not only does go with it, it gets opened up as part of the Cabinet Office’s opendata work.

Written by countculture

June 15, 2010 at 3:54 pm

Open Data & The Rewards Of Failure

with 5 comments

What does Open Data have to do with failure? It change the incentives from big, slow, deniable failures to small, fast  failures from which you (and other people) can learn.

I’d been asked to give a ‘philosophical’ presentation about Open Data to the FutureEverything festival in Manchester as part of yesterday’s Challenges & Rewards of Open Data session.

Open Data & The Rewards Of Failure is the result. I’m not sure if it’s philosophical; I hope it’s entertaining and thought-provoking (many of the attendees told me it was); and more than that, I hope it’s another drop of water that starts to erode the existing processes and structures and allows new ones to flourish.

It’s embedded below (hopefully), and is licensed under a CC BY-SA licence, so feel free to forward, download, distribute, and comment.

Written by countculture

May 14, 2010 at 11:16 am

The GLA and open data: did he really say that?

with 5 comments

The launch on Friday of the Greater London Authority’s open data initiative (aka London Datastore) was a curious affair, and judging from some of the discussions in the pub after, I think that the strangeness – a joint teleconferenced event with CES Las Vegas – possibly overshadowed its significance and the boldness of the GLA’s action.

First off the technology let it down – if Skype wanted to give a demo of just how far short its video conferencing is from prime time they did a perfect job. Boris did a great impromptu stand-up routine, looking for the world like he was still up from the night before, but the people at CES in Las Vegas missed the performance and whose images and words occasionally stuttered in to life to interrupt the windows/skype error messages.

However in between the gags Boris came out with this nugget, “We will open up all the GLA’s data for free reuse”.

What does that mean, I wondered, all their data? All that’s easy to do? Does it include info from TransportForLondon (TfL), the Metropolitan Police? To be honest I sort of assumed it was Boris just paraphrasing. Nevertheless, I thought, it could be a good stick to enforce change later on.

However then it was Deputy Mayor Sir Simon Milton’s turn to give the more scripted, more plodding, more coherent version. This was the bit where we would find out what’s really going to happen. [What you need to realise that the GLA doesn’t actually have a lot of its own data – mostly it’s just some internal stuff, slices of central government data, and grouping of London council info. The good stuff is owned by those huge bodies, such as TfL and the Met, that it oversees.

So when Steve said: “I hope that our discussions with the GLA group will be fruitful and that in the short term we can encourage them to release that data which is not tied to commercial contracts and in the longer term encourage them when these contracts come up for renewal to apply different contractual principles that would allow for the release of all of their data into the public domain“, all I heard was yada yada yada.

The next bit, however, genuinely took me by surprise:

“I can confirm today, however, that as a result of our discussions around the Datastore, TfL are willing to make raw data available through the Datastore. Initially this will be data which is already available for re-use via the TfL website, including live feeds from traffic cameras, geo-coded information on the location of Tube, DLR and Overground stations, the data behind the Findaride service to locate licensed mini-cab and private hire operators and data on planned works affecting weekend Tube services.

“TfL will also be considering how best to make available detailed timetabling data for its services and welcomes examples of other data which could also be prioritised for inclusion in the Datastore such as the data on live departures and Tube incidents on TfL’s website”

So stunned was I in fact (and many others too) we that we didn’t ask any questions when he finished talking came to it , or for that matter congratulate Boris/Simon on the steps they were taking.

Yes, it’s nothing that hasn’t been done in Washington DC or San Francisco, and it isn’t as  big a deal as the Government’s open data announcement on December 7 (which got scandalously little press coverage, even in the broadsheets, yet may well turn out to be the most important act of this government).

However it is a huge step for local government in the UK and sets a benchmark for other local authorities to attain, and for the GLA to have achieved what it already has with Transport for London will only have come after a considerable trial of will, and one, significantly, that they won.

So, Simon & Boris, and all those who fought the battle with TfL, well done. Now let’s see some action with the other GLA bodies – the Met, London Development Agency, London Fire Brigade, he London Pensions Fund Authority in particular (I’m still trying to figure out its relationship to Visit London and the London Travel Watch).

Update: Video embedded below

Useful links:

Written by countculture

January 8, 2010 at 5:56 pm