countculture

Open data and all that

Introducing the Open Election Project: tiny steps towards opening local elections

with 18 comments

Update: The Open Election Data project is now live at http://openelectiondata.org.

Here’s a fact that will surprise many: There’s no central or open record of the results of local elections in the UK.

Want to look back at how people voted in your local council elections over the past 10 years? Tough. Want to compare turnout between different areas, and different periods? Sorry, no can do. Want an easy way to see how close the election was last time, and how much your vote might make a difference? Forget it.

It surprised and faintly horrified me (perhaps I’m easily shocked). Go to the Electoral Commission’s website and you’ll see they quickly pass the buck… to the BBC, who just show records of seat numbers, not voting records.


In fact, there is an unofficial database of the election results — held by Plymouth University, and this is what they do (remember we’re in the year 2010 now):

We collect them and then enter them manually into our database. This process is begun in February where we assess which local authority wards are due up for election, continues during March and April when we collect electorates and candidate names and then following the elections in May (or June in some years) we contact the local authorities for their official results”

Not surprisingly, the database is commercial (I guess they have to pay for all that manual work somehow), though they do receive some support from the Electoral Commission, which means as far as democracy, open analysis, and public record goes, it might as well not exist.

There are, of course, records of local election results on local authority websites, but accessible/comparable/reusable they ain’t, nor are they easy to find, and they are in so many different formats that it makes getting the information out of them near impossible, except manually.

So in the spirit of scratching your own itch (I’d like to put the information on OpenlyLocal.com, and I’m sure lots of other bodies would too, from the BBC to national press), I came up with a grandiose title and a simple plan: The Open Election Data project, an umbrella project to help local authorities to publish their election results in an open, reusable common format.

I had the idea at the end of the first meeting of the Local Public Data Panel, of which I’m a member and which is tasked with finding ways of opening up local public data. I then did an impromptu session at the UK Gov Barcamp on January 23, and got a great response. Finally I had meetings and discussions with all sorts of people, from local govt types, local authority CMS suppliers, council webmasters, returning officers and standards organisations. Finally, it was discussed at the 2nd Local Public Data Panel meeting this week, and endorsed there.

So how does it work? Well, the basic idea is that instead of councils writing their election results web pages using some arbitrary HTML (or worse, using PDFs), they use HTML with machine-readable information added into it using something called RDFa, which is already used by many organisations for the this purpose (e.g. for government’s consultations).

This means that pretty much any competent Content Management System should be able to use the technique to expose the data, while still allowing them to style it as they wish. Here, for example, is what one of Lichfield District Council’s election results pages currently looks like:

And this is what it looks like after it’s had RDFa added (and a few more bits of information):

As you can see (apart from the extra info), there appears to be no change to the user. The difference is however, that if you point a machine capable of understanding RDFa at it, you can slurp up the results, and bingo, suddenly you’ve got an election results database for free, putting local elections on a par with national ones for the first time.

So where do things go from here?

I’m also presenting this at the localgovcamp tomorrow(March 4), and we hope to have some draft local authority election results pages in the weeks shortly afterwards (although the focus is on getting as many councils to implement this by the local elections on May 6, there’s nothing to stop them using it on existing pages, and indeed we’d encourage them to, so they can get a feel for and indeed expose those earlier results). I’m also discussing setting up a Community of Practice to help enable council webmasters discuss implementation.

Finally, many thanks to those who have helped me draw up the various RDFa stuff and/or helped with the underlying idea: especially Jeni Tennison, Paul Davidson from LeGSB, Stuart Harrison of Lichfield District Council, Tim Allen of the LGA, and many more.

Written by countculture

March 3, 2010 at 11:24 pm

18 Responses

Subscribe to comments with RSS.

  1. Great idea. Great work.
    Is there a crowdsourvcing option, for those of us with recalcitrant LAs?

    megov

    March 4, 2010 at 8:10 am

    • Not yet, but the linked data process would potentially make this possible, or indeed to link to other info about the elections/candidates

      countculture

      March 4, 2010 at 9:41 am

      • I imagine a possible post-election task we could make available to Democracy Club volunteers could be along those lines.

        Tim Green

        March 5, 2010 at 3:30 pm

      • Hi Chris,

        I’m working with a coalition of state and local technologists working within government to develop this process and data structure in the U.S. We also want to put a strong focus on candidate data and ways to do verifiable crowdsourcing of data. Some information on this can be found at http://ideas.topplabs.org/wiki/Who_is_my_government%3F

        We’re now in the process of ramping up this effort and I am expecting to get some support from the federal government in this endeavor. I will be meeting with them about this in the near future.

        Can we get in touch to discuss how these efforts can work together in more detail?

        It’s wonderful to see this moving forward. Keep up the excellent work.

        -Phil

        Philip Ashlock

        March 17, 2010 at 8:08 pm

      • Phil: Sure, no problem

        countculture

        March 17, 2010 at 8:10 pm

  2. This is a brilliant idea. Let me know if I can be any help.

    jamie

    March 5, 2010 at 6:30 am

  3. I stumbled across this looking for a list of elected positions and when due for re-election. Fairly static data only changes when boundaries change or authorities go unitary. And, it would give people advanced notice so they could prepare to challenge incumbants.

    I suppose this is a bit too radical.

    Roy Benford

    March 9, 2010 at 4:27 pm

  4. […] Some truly inspirational stuff about the impacts of linked and open public data on safety, disaster relief and equalities in service provision. There’s even a great example of polling data in Afghanistan. Looks like they have more open data than we do on local elections, though hopefully that will soon change (read a bit about the Open Elections project […]

  5. Have you tried just contacting the local authority concerned? Not all authorities have websites, but town clerks are usually pretty helpful if you want to know about election results.

    This site seems a bit like reinventing the wheel.

    David

    March 17, 2010 at 3:13 pm

    • Think you’re missing the point somehow. That’s rather like saying why should councils have websites, when you can wander down to the town hall…

      The point is to make the information as data, so you have access to it in a variety of ways, and can compare with with related data (other areas, elections etc).

      countculture

      March 17, 2010 at 3:19 pm

  6. […] I also posted recently about scraping election data from Lichfield Council (Screenscraping With Google Spreadsheets App Script and the =importHTML Formula), again prompted by work @countculture is doing around local government (Introducing the Open Election Project: tiny steps towards opening local elections). […]

  7. FWIW, Wikipedia also has a collection of past results; for example:

    http://en.wikipedia.org/wiki/Barking_(UK_Parliament_constituency)#Election_results

    Coverage is, admittedly, variable.

    pigsonthewing

    March 18, 2010 at 9:32 pm

    • The problem is these results are for parliamentary elections, not local authority ones.

      countculture

      March 18, 2010 at 11:30 pm

      • Of course: but the model (user-generated, wiki-based, templated, categorised) may still have lessons.

        pigsonthewing

        March 18, 2010 at 11:52 pm

  8. Excellent project and glad to see it taking off.

    One comment though about the Plymouth data: I’ve always found the team who look after it very helpful and willing to provide data from it when I’ve been in touch with specific queries. E.g. in reply to a question about what data they had on the gender of candidates, I got a time series for each party and even a graph thrown in too.

    That’s not to say that there isn’t a lot to be said for a public data set, but within the current circumstances I think the Plymouth team are pretty open.

    Mark Pack

    March 27, 2010 at 4:31 pm

  9. […] Taggert, another volunteer, has come to the rescue with his Open Election Project. He’s been promoting an electoral results RDFa – a neat, lightweight form of the semantic web […]

  10. Any idea why the opendata.org site appears to have gone belly up – just when we were about to look at it for inspiration for the forthcoming May election?

    Martin Waters

    February 10, 2011 at 9:19 am

    • It’s on a tiny shared server with OpenCharities, which is consuming most of the resources, due to its popularity. I need to move OpenCharities onto a bigger server, but been busy importing the local council spending data

      countculture

      February 10, 2011 at 10:33 am


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: