countculture

Open data and all that

Posts Tagged ‘semantic web

Introducing the Open Election Project: tiny steps towards opening local elections

with 18 comments

Update: The Open Election Data project is now live at http://openelectiondata.org.

Here’s a fact that will surprise many: There’s no central or open record of the results of local elections in the UK.

Want to look back at how people voted in your local council elections over the past 10 years? Tough. Want to compare turnout between different areas, and different periods? Sorry, no can do. Want an easy way to see how close the election was last time, and how much your vote might make a difference? Forget it.

It surprised and faintly horrified me (perhaps I’m easily shocked). Go to the Electoral Commission’s website and you’ll see they quickly pass the buck… to the BBC, who just show records of seat numbers, not voting records.


In fact, there is an unofficial database of the election results — held by Plymouth University, and this is what they do (remember we’re in the year 2010 now):

We collect them and then enter them manually into our database. This process is begun in February where we assess which local authority wards are due up for election, continues during March and April when we collect electorates and candidate names and then following the elections in May (or June in some years) we contact the local authorities for their official results”

Not surprisingly, the database is commercial (I guess they have to pay for all that manual work somehow), though they do receive some support from the Electoral Commission, which means as far as democracy, open analysis, and public record goes, it might as well not exist.

There are, of course, records of local election results on local authority websites, but accessible/comparable/reusable they ain’t, nor are they easy to find, and they are in so many different formats that it makes getting the information out of them near impossible, except manually.

So in the spirit of scratching your own itch (I’d like to put the information on OpenlyLocal.com, and I’m sure lots of other bodies would too, from the BBC to national press), I came up with a grandiose title and a simple plan: The Open Election Data project, an umbrella project to help local authorities to publish their election results in an open, reusable common format.

I had the idea at the end of the first meeting of the Local Public Data Panel, of which I’m a member and which is tasked with finding ways of opening up local public data. I then did an impromptu session at the UK Gov Barcamp on January 23, and got a great response. Finally I had meetings and discussions with all sorts of people, from local govt types, local authority CMS suppliers, council webmasters, returning officers and standards organisations. Finally, it was discussed at the 2nd Local Public Data Panel meeting this week, and endorsed there.

So how does it work? Well, the basic idea is that instead of councils writing their election results web pages using some arbitrary HTML (or worse, using PDFs), they use HTML with machine-readable information added into it using something called RDFa, which is already used by many organisations for the this purpose (e.g. for government’s consultations).

This means that pretty much any competent Content Management System should be able to use the technique to expose the data, while still allowing them to style it as they wish. Here, for example, is what one of Lichfield District Council’s election results pages currently looks like:

And this is what it looks like after it’s had RDFa added (and a few more bits of information):

As you can see (apart from the extra info), there appears to be no change to the user. The difference is however, that if you point a machine capable of understanding RDFa at it, you can slurp up the results, and bingo, suddenly you’ve got an election results database for free, putting local elections on a par with national ones for the first time.

So where do things go from here?

I’m also presenting this at the localgovcamp tomorrow(March 4), and we hope to have some draft local authority election results pages in the weeks shortly afterwards (although the focus is on getting as many councils to implement this by the local elections on May 6, there’s nothing to stop them using it on existing pages, and indeed we’d encourage them to, so they can get a feel for and indeed expose those earlier results). I’m also discussing setting up a Community of Practice to help enable council webmasters discuss implementation.

Finally, many thanks to those who have helped me draw up the various RDFa stuff and/or helped with the underlying idea: especially Jeni Tennison, Paul Davidson from LeGSB, Stuart Harrison of Lichfield District Council, Tim Allen of the LGA, and many more.

Written by countculture

March 3, 2010 at 11:24 pm

David Eaves’ Three Laws of Open Government Data

with 2 comments

Mentioned David Eaves’ Three Laws of Open Government Data at yesterday’s excellent Talk About Local unconference, and had a few people asking me what they were and where to find them. So here they are (from http://eaves.ca/2009/09/30/three-law-of-open-government-data/):

The Three Laws of Open Government Data:

  1. If it can’t be spidered or indexed, it doesn’t exist
  2. If it isn’t available in open and machine readable format, it can’t engage
  3. If a legal framework doesn’t allow it to be repurposed, it doesn’t empower

There are also a few other useful links in the comments.

p.s. I don’t know David, but I really like the conciseness of these

Written by countculture

October 4, 2009 at 10:05 am

Open Data Feeds from Councils: brain dump

with one comment

This post is something of a brain dump about some possible common principles for open data for Local Authorities. It’s pretty much the text of a post I made to the tyoc google group, which is helping to organise a TheyWorkForYou-type website for Manchetser County Council.

It’s probably not the first post on the subject (link to other ones welcome), and certainly won’t be the last, but hopefully will provide some useful thoughts for those councils or groups working on exposing their data. (Maybe if there’s anyone else interested we can get this a bit more formalized.)

==========

My thoughts have been influenced by exposing the data from OpenlyLocal.com and also from consuming XML data from other authorities, but obviously these are only my initial ideas, and I’m using as an example OpenlyLocal urls and also those for Lichfield District Council, who’s got a great webmaster who kindly exposed all the council democratic data they could as XML.

  1. The api should expose the authority’s internal UIDs (as well as the id of the record in the application if it’s not the council exposing the data). The idea is to open up the data, not creating another walled garden. See http://openlylocal.com/members/3114.xml and http://www.lichfielddc.gov.uk/site/custom_scripts/meetings_committees…
  2. UIDs should be absolutely unique to the object being exposed and should not change if the name changes (so no strings for councillor IDs, as these can change if the councillors name changes, e.g. through marriage, titles or simply what they prefer to be known as).
  3. The api should use and expose common identifiers when possible to allow definitive identification, e.g. Wards should expose the ONS Snac Ids. E.g. http://openlylocal.com/wards/982.xml
  4. The api should given information about when the object was last updated. At the moment on OpenlyLocal, all objects have created_at and updated_at fields exposed. However, given OpenlyLocal.com and the ‘Your MCC’ projects are basically proxies we should possibly also expose a “last_checked_at” field, so the timeliness can be worked out.

Written by countculture

September 21, 2009 at 1:24 pm

Opening Up Local Government Information: APPSI Presentation

with one comment

Just got back from doing a presentation to the Advisory Panel on Public Sector Information

Though there’s a bit about OpenlyLocal.com (the site I run that opens up and makes accessible local government data), most of it is about the present and future of local government data, and the obstacles that need to be overcome.

The presentation seemed to go down reasonably well, despite Powerpoint messing up the formatting (it was created on Keynote), and I’m hoping we can start to get some traction on opening up local government data.

I’m embedding it and making it available under Creative Commons non-commercial share-alike licence. Comments welcome:

Written by countculture

September 17, 2009 at 4:39 pm

OpenlyLocal new features: RSS feeds, wards, calendars

leave a comment »

Time for a quick update on OpenlyLocal.com — the website that opens up local government in an accessible and consistent way (a sort of local government TheyWorkForYou). Since the last post announcing the site I’ve added quite a few new features, as well as a fair bit of tweaking to the back-end .

The main ones are:

  • Loads more councils (we have now opened up over 65 councils, with info on more than 4000 councillors and 2300 committees). If you council isn’t there give me a nudge and I’ll see what we can do
  • Keyword filtering of committee minutes and agendas. So if you want to find a council’s minutes and agendas featuring Tesco, for example you can now do that.

    Brighton Council minutes/agendas with keyword 'Tesco'

  • RSS feeds for minutes and agendas, including ones restricted to keywords. Just click on the RSS feed link at the bottom of the page.
  • Even more Calendar feeds. You can now subscribe to a calendar for a whole council, an individual councillor, or a particular council committee.
  • Added Wards to the information we’re extracting from websites.
  • Added RDFa markup, to allow integration with other datasets and become part of the semantic web.
  • Plus tons of backend improvements making it easier for me to add new councils, and loads of bugfixes

There’s still a long way to go, but OpenlyLocal does feel as if it’s moved a little beyond the proof-of-concept stage and is now genuinely useful for local democracy.

Written by countculture

September 1, 2009 at 11:53 am

Follow

Get every new post delivered to your Inbox.

Join 80 other followers