Open data and all that

Posts Tagged ‘parliament

George Osborne’s open data moment: it’s the Treasury, hell yeah

with 2 comments

As a bit of an outsider, reading the government’s pronouncements on open data feels rather like reading official Kremlin statements during the Cold War. Sometimes it’s not what they’re saying, it’s who’s saying it that’s important.

And so it is, I think, with George Osborne’s speech yesterday morning at Google Zeitgeist, at which he stated, “Our ambition is to become the world leader in open data, and accelerate the accountability revolution that the internet age has unleashed“, and “The benefits are immense. Not just in terms of spotting waste and driving down costs, although that consequence of spending transparency is already being felt across the public sector. No, if anything, the social and economic benefits of open data are even greater.

This is strong, and good stuff, and that it comes from Osborne, who’s not previously taken a high profile position on open data and open government, leaving that variously to the Cabinet Office Minister, Francis Maude, Nick Clegg & even David Cameron himself.

It’s also intriguing that it comes in the apparent burying of the Public Data Corporation, which got just a holding statement in the budget, and no mention at all in Osborne’s speech.

But more than that it shows the Treasury taking a serious interest for the first time, and that’s both to be welcomed, and feared. Welcomed, because with open data you’re talking about sacrificing the narrow interests of small short-term fiefdoms (e.g. some of the Trading Funds in the Shareholder Executive) for the wider interest; you’re also talking about building the essential foundations for the 21st century. And both of these require muscle and money.

It also overseas a number of datasets which have hitherto been very much closed data, particularly the financial data overseen by the Financial Services Authority, the Bank of England and even perhaps some HMRC data, and I’ve started the ball rolling by scraping the FSA’s Register of Mutuals, which we’ve just imported into OpenCorporates, and tying these to the associated entries in the UK Register of Companies.

Feared, because the Treasury is not known for taking prisoners, still less working with the community. And the fear is that rather than leverage the potential that open data allows for a multitude of  small distributed projects (many of which will necessarily and desirably fail), rather than use the wealth of expertise the UK has built up in open data, they will go for big, highly centralised projects.

I have no doubt, the good intentions are there, but let’s hope they don’t do a Team America here (and this isn’t meant as a back-handed reference to Beth Noveck, who I have a huge amount of respect for, and who’s been recruited by Osborne), and destroy the very thing they’re trying to save.

Written by countculture

May 17, 2011 at 2:27 pm

Introducing the Open Election Project: tiny steps towards opening local elections

with 18 comments

Update: The Open Election Data project is now live at

Here’s a fact that will surprise many: There’s no central or open record of the results of local elections in the UK.

Want to look back at how people voted in your local council elections over the past 10 years? Tough. Want to compare turnout between different areas, and different periods? Sorry, no can do. Want an easy way to see how close the election was last time, and how much your vote might make a difference? Forget it.

It surprised and faintly horrified me (perhaps I’m easily shocked). Go to the Electoral Commission’s website and you’ll see they quickly pass the buck… to the BBC, who just show records of seat numbers, not voting records.

In fact, there is an unofficial database of the election results — held by Plymouth University, and this is what they do (remember we’re in the year 2010 now):

We collect them and then enter them manually into our database. This process is begun in February where we assess which local authority wards are due up for election, continues during March and April when we collect electorates and candidate names and then following the elections in May (or June in some years) we contact the local authorities for their official results”

Not surprisingly, the database is commercial (I guess they have to pay for all that manual work somehow), though they do receive some support from the Electoral Commission, which means as far as democracy, open analysis, and public record goes, it might as well not exist.

There are, of course, records of local election results on local authority websites, but accessible/comparable/reusable they ain’t, nor are they easy to find, and they are in so many different formats that it makes getting the information out of them near impossible, except manually.

So in the spirit of scratching your own itch (I’d like to put the information on, and I’m sure lots of other bodies would too, from the BBC to national press), I came up with a grandiose title and a simple plan: The Open Election Data project, an umbrella project to help local authorities to publish their election results in an open, reusable common format.

I had the idea at the end of the first meeting of the Local Public Data Panel, of which I’m a member and which is tasked with finding ways of opening up local public data. I then did an impromptu session at the UK Gov Barcamp on January 23, and got a great response. Finally I had meetings and discussions with all sorts of people, from local govt types, local authority CMS suppliers, council webmasters, returning officers and standards organisations. Finally, it was discussed at the 2nd Local Public Data Panel meeting this week, and endorsed there.

So how does it work? Well, the basic idea is that instead of councils writing their election results web pages using some arbitrary HTML (or worse, using PDFs), they use HTML with machine-readable information added into it using something called RDFa, which is already used by many organisations for the this purpose (e.g. for government’s consultations).

This means that pretty much any competent Content Management System should be able to use the technique to expose the data, while still allowing them to style it as they wish. Here, for example, is what one of Lichfield District Council’s election results pages currently looks like:

And this is what it looks like after it’s had RDFa added (and a few more bits of information):

As you can see (apart from the extra info), there appears to be no change to the user. The difference is however, that if you point a machine capable of understanding RDFa at it, you can slurp up the results, and bingo, suddenly you’ve got an election results database for free, putting local elections on a par with national ones for the first time.

So where do things go from here?

I’m also presenting this at the localgovcamp tomorrow(March 4), and we hope to have some draft local authority election results pages in the weeks shortly afterwards (although the focus is on getting as many councils to implement this by the local elections on May 6, there’s nothing to stop them using it on existing pages, and indeed we’d encourage them to, so they can get a feel for and indeed expose those earlier results). I’m also discussing setting up a Community of Practice to help enable council webmasters discuss implementation.

Finally, many thanks to those who have helped me draw up the various RDFa stuff and/or helped with the underlying idea: especially Jeni Tennison, Paul Davidson from LeGSB, Stuart Harrison of Lichfield District Council, Tim Allen of the LGA, and many more.

Written by countculture

March 3, 2010 at 11:24 pm

MP attendance: end of term report

with 3 comments

[Note: Voting attendance is an imperfect proxy for actual attendance, as the figure may be depressed by silent abstentions (i.e. not voting in a division, rather than voting both ‘aye’ and ‘no’) and by just turning up to vote, but failing to attend the debate. However, until Parliament provides a better measure for attendance, or more transparency of MPs actions, this is the only one we have.]

It’s recess time again, and time for MPs’ end-of-term report. I’ll leave it to others to comment on how they’ve dealt with some of the genuinely momentous events since the summer recess. This post deals solely with their voting attendance record.

First off, let’s have an overall look at the overall figures for the period:

Oct-Dec 08 May 97-Jul 08
All MPs 70.2% 64.5%
74.8% 69.8%
67.8% 61.7%
72.2% 64.7%

The figures above are pretty self-explanatory. All parties have improved their attendance of votes, by 5 to 8 percentage points. Perhaps not surprising given the financial crisis.

Now let’s have a look at the main parties in detail, using the same histograms used before to show the distribution of the parties attendance figures. Interestingly (well, in a wonkish sort of way), the distributions are a bit more spread out than the long-term average. In part this is probably down to the shorter time period showing up variations that are hidden in longer period, but it’s interesting nevertheless to note that though all parties have improved their overall attendance figures, the number and proportion of Labour MPs who’ve voted in fewer than half the divisions has nearly tripled, from 11 MPs to 30 of them.


[Note: there’s no significance to the width of the columns — the recent ones are narrower so that both can be seen on the same graph]

Finally, let’s have a look at those outliers, first, the MPs who attended divisions less than than 50% of the time:

Attended fewer than 50% of divisions Oct–Dec 08

Name Party Constituency Attendance % votes attended/possible
David Cameron Con Witney 28.6% 24/84
Robert Walter Con North Dorset 34.5% 29/84
Michael Mates Con East Hampshire 35.7% 30/84
Tim Yeo Con South Suffolk 35.7% 30/84
Michael Howard Con Folkestone & Hythe 36.9% 31/84
Peter Bottomley Con Worthing West 40.5% 34/84
Liam Fox Con Woodspring 40.5% 34/84
Julian Lewis Con New Forest East 42.9% 36/84
Caroline Spelman Con Meriden 42.9% 36/84
David Tredinnick Con Bosworth 42.9% 36/84
David Wilshire Con Spelthorne 42.9% 36/84
David Mundell Con Dumfriesshire, Clydesdale & Tweeddale 42.9% 36/84
Malcolm Rifkind Con Kensington & Chelsea 45.2% 38/84
Michael Gove Con Surrey Heath 45.2% 38/84
William Hague Con Richmond (Yorks) 46.4% 39/84
Mark Lancaster Con North East Milton Keynes 48.8% 41/84
Mark Oaten LDem Winchester 23.8% 20/84
Daniel Rogerson LDem North Cornwall 47.6% 40/84
Nicholas Clegg LDem Sheffield, Hallam 47.6% 40/84
Margaret Hodge Lab Barking 0.0% 0/84
Jessica Morden Lab Newport East 0.0% 0/84
Gordon Brown Lab Kirkcaldy & Cowdenbeath 3.6% 3/84
Kali Mountford Lab Colne Valley 14.3% 12/84
David Miliband Lab South Shields 19.0% 16/84
Khalid Mahmood Lab Birmingham, Perry Barr 21.4% 18/84
Gordon Banks Lab Ochil & Perthshire South 27.4% 23/84
Alistair Darling Lab Edinburgh South West 29.8% 25/84
Adam Ingram Lab East Kilbride, Strathaven & Lesmahagow 29.8% 25/84
Glenda Jackson Lab Hampstead & Highgate 31.0% 26/84
Mike Wood Lab Batley & Spen 32.1% 27/84
Frank Cook Lab Stockton North 33.3% 28/84
Geraldine Smith Lab Morecambe & Lunesdale 36.9% 31/84
Rudi Vis Lab Finchley & Golders Green 39.3% 33/84
Claire Curtis-Thomas Lab Crosby 40.5% 34/84
Alan Milburn Lab Darlington 40.5% 34/84
Keith Vaz Lab Leicester East 40.5% 34/84
Tom Harris Lab Glasgow South 40.5% 34/84
Doug Henderson Lab Newcastle upon Tyne North 42.9% 36/84
Denis Murphy Lab Wansbeck 42.9% 36/84
Bill Etherington Lab Sunderland North 44.0% 37/84
Kate Hoey Lab Vauxhall 45.2% 38/84
Denis MacShane Lab Rotherham 45.2% 38/84
Stuart Bell Lab Middlesbrough 46.4% 39/84
Joe Benton Lab Bootle 46.4% 39/84
Roger Godsiff Lab Birmingham, Sparkbrook & Small Heath 46.4% 39/84
Bill Rammell Lab Harlow 46.4% 39/84
John Austin Lab Erith & Thamesmead 47.6% 40/84
Caroline Flint Lab Don Valley 47.6% 40/84
Robert Marshall-Andrews Lab Medway 48.8% 41/84

No surprise that GB‘s in this list, voting in only 3 divisions (see here for the divisions he did take time out to vote on). Ditto Alistair Darling and David Cameron.

However, the rest of the list is more interesting. Some of those on the front bench, for example (e.g. Jacqui Smith, Jack Straw), surprisingly don’t make the list, i.e. they voted in at least 50% of the divisions. Ditto some of the opposition spokespeople.

But what about the backbenchers who are on the list. Possibly there’s a good reason for Margaret Hodge and Jessica Morden for failing to attend a single division — illness perhaps (though there’s nothing on either of their websites to indicate such a factor)? And what about Kali Mountford (14.3%) and Khalid Mahmood (21.4%).

If I was in their constituency, I’d like to know, particularly since they took little part in debates, either. Similarly for the low-raters for the Conservatives — Michael Mates and Tim Yeo (6 directorships!) at 35.7% each.

Now the MPs who voted more than 90% of the time:

Attended more than 90% of divisions

Name Party Constituency Attendance % votes attended/possible
George Young Con North West Hampshire 91.7% 77/84
Willie Rennie LDem Dunfermline & Fife West 90.5% 76/84
Andrew Stunell LDem Hazel Grove 95.2% 80/84
John Hemming LDem Birmingham, Yardley 97.6% 82/84
Ian Cawsey Lab Brigg & Goole 90.5% 76/84
Paul Clark Lab Gillingham 90.5% 76/84
John Cummings Lab Easington 90.5% 76/84
John Heppell Lab Nottingham East 90.5% 76/84
Fraser Kemp Lab Houghton & Washington East 90.5% 76/84
Phyllis Starkey Lab Milton Keynes South West 90.5% 76/84
David Heyes Lab Ashton-under-Lyne 90.5% 76/84
Kevan Jones Lab North Durham 90.5% 76/84
Ian Lucas Lab Wrexham 90.5% 76/84
Siân James Lab Swansea East 90.5% 76/84
Barbara Keeley Lab Worsley 90.5% 76/84
Clive Betts Lab Sheffield, Attercliffe 91.7% 77/84
Angela Eagle Lab Wallasey 91.7% 77/84
Maria Eagle Lab Liverpool, Garston 91.7% 77/84
John Healey Lab Wentworth 91.7% 77/84
Keith Hill Lab Streatham 91.7% 77/84
Alun Michael Lab Cardiff South & Penarth 91.7% 77/84
David Taylor Lab North West Leicestershire 91.7% 77/84
Natascha Engel Lab North East Derbyshire 91.7% 77/84
Shahid Malik Lab Dewsbury 91.7% 77/84
Liz Blackman Lab Erewash 92.9% 78/84
Bob Blizzard Lab Waveney 92.9% 78/84
Jeff Ennis Lab Barnsley East & Mexborough 92.9% 78/84
Fiona Mactaggart Lab Slough 92.9% 78/84
Andrew Miller Lab Ellesmere Port & Neston 92.9% 78/84
Kerry McCarthy Lab Bristol East 92.9% 78/84
Mary Creagh Lab Wakefield 92.9% 78/84
Lyn Brown Lab West Ham 92.9% 78/84
Kevin Barron Lab Rother Valley 94.0% 79/84
Janet Dean Lab Burton 94.0% 79/84
Jim Fitzpatrick Lab Poplar & Canning Town 94.0% 79/84
Mike Hall Lab Weaver Vale 94.0% 79/84
David Kidney Lab Stafford 94.0% 79/84
Shona McIsaac Lab Cleethorpes 94.0% 79/84
Judy Mallaber Lab Amber Valley 94.0% 79/84
James Plaskitt Lab Warwick & Leamington 94.0% 79/84
Nick Raynsford Lab Greenwich & Woolwich 94.0% 79/84
Angela Smith Lab Basildon 94.0% 79/84
Ann McKechin Lab Glasgow North 94.0% 79/84
Clive Efford Lab Eltham 95.2% 80/84
David Hanson Lab Delyn 95.2% 80/84
Dan Norris Lab Wansdyke 95.2% 80/84
Bill Olner Lab Nuneaton 95.2% 80/84
Stephen McCabe Lab Birmingham, Hall Green 96.4% 81/84
John Spellar Lab Warley 96.4% 81/84
Dave Watts Lab St Helens North 96.4% 81/84
Helen Goodman Lab Bishop Auckland 96.4% 81/84
Andrew Gwynne Lab Denton & Reddish 96.4% 81/84
Neil Gerrard Lab Walthamstow 97.6% 82/84
Brian Jenkins Lab Tamworth 97.6% 82/84
Helen Jones Lab Warrington North 97.6% 82/84
Thomas McAvoy Lab Rutherglen & Hamilton West 97.6% 82/84
Chris Bryant Lab Rhondda 97.6% 82/84
Chris Mole Lab Ipswich 97.6% 82/84
Diana Johnson Lab Kingston upon Hull North 97.6% 82/84
Tony Cunningham Lab Workington 98.8% 83/84
Dennis Skinner Lab Bolsover 100.0% 84/84

Some amazing figures in there. In fact, given his 100% record you wonder if the Beast of Bolsover has got a home to go to.


Notes on calculations

  • The above calculations were derived from the voting record freely available from the Public Whip project, and cover the period from Oct 2008 to Dec 2008. The data can be downloaded in the form of a MySQL database, and this was used together with custom MySQL queries to generate the figures.

Written by countculture

December 31, 2008 at 7:02 pm

How often do MPs turn up for work (Part 4): the ministerial effect

leave a comment »

[Note: Voting attendance is an imperfect proxy for actual attendance, as the figure may be depressed by silent abstentions (i.e. not voting in a division, rather than voting both ‘aye’ and ‘no’) and by just turning up to vote, but failing to attend the debate. However, until Parliament provides a better measure for attendance, or more transparency of MPs actions, this is the only one we have.]

A frequent arguments for low attendance of voting divisions by MPs is that the figure is depressed by ministers (and shadow spokespersons), whose other responsibilities prevent them from attending as many votes (as they’d like to), thus bringing down the overall average.

Seems reasonable, so let’s have a look at just how much of an influence this ‘ministerial effect’ has on the overall figures. First, let’s look at the average voting attendance for ministers and non-ministers (calculation details below):

Attendance rates May 97 – July 08
All MPs 65.1%
Non-Ministers 64.4%
Ministers 67.2%

Er, wait a minute, so the average voting attendance rate for ministers is higher than non-ministers? That’s not what we expected. However, basic averages (i.e. the mean) can hide a multitude of sins, so let’s have a look at the distribution of those attendance figures.

As you can see, while the peak of the ministerial attendance is around the 65% mark (less than that for the non-ministerial one), there were far more divisions in which 90%+ of ministers voted than there were for which 90%+ of non-ministers voted.

This makes sense, in a way, as ministers are far more likely than backbenchers to turn up en masse for votes their party sees as important. It’s this that largely accounts for the figures we saw in the table above. However, what the graph also shows is that when you take the ministers out of the equation, attendance definitely does not shoot up. There is, in short, no ‘ministerial effect’ to account for the low attendance of MPs.

[It’s worth mentioning that the ministerial office records are slightly incomplete — the record of Parliamentary Private Secretaries is missing during some periods — so I’ve run the figures for ministers both including and excluding PPSs. As you can see, it doesn’t make a lot of difference.]

The party lines

Having looked at the big picture, it’s time to look at the ministerial vs non-ministerial attendance by party, specifically the three main parties in Parliament.

As you can see, the relationship between ministerial and non-ministerial attendance is noticeably different for each of the parties. Labour ministers do indeed have noticeably lower attendance rates than their backbenchers, though not as much as I’d expected and not enough to alter the distribution massively.

However, for the Tories and LibDems, the surprising thing — for me, at least — was the attendance rates for their spokespersons are actually noticeably better than their backbenchers, raising rather than lowering the overall figures. What, I wonder, is the reason for this?

Finally, a couple of quick graphs to wrap this post up. One shows, perhaps not surprisingly, that Labour ministerial attendance rates are less than for the shadow spokespersons — presumably the time commitment for a governmental position is greater than that for the equivalent shadow position.

The other shows the distribution of backbenchers attendance figures, by party. I’ll leave that one without making any further comment.


Notes on calculations

  • The Ministerial/non-ministerial attendance rates were calculated by looking at every Commons division between May 1997 and July 2008, and working out the number of ministers/non-ministers who could have voted in that division, and the number who actually did vote. The average attendance figures in the table were calculated by dividing the aggregate number of votes by the aggregate number of possible votes.
    To calculate the distribution of attendance rates I calculated the ministerial/non-ministerial attendance rate for each division, and plotted these on a graph to show how those attendance rates are distributed (as usual, I’ve made the underlying figures are available as a spreadsheet here and here if you want to examine them further).
  • Ministers are those holding any sort of ministerial office as per the PublicWhip database, including whips, but excluding select committee members (although it wouldn’t be hard to run the figures to include select committee members). The Parliamentary Private Secretaries record at the Public Whip is incomplete for several periods, and unfortunately (and ridiculously) there is no historical record of ministers available from Parliament’s own website.
  • The above calculations were derived from the voting record freely available from the Public Whip project, and cover the period from May 1997 to July 22, 2008 (when the house rose for the summer recess). The data can be downloaded in the form of a MySQL database, and this was used together with custom MySQL queries to generate the figures.
  • The graphs are visual representations of the density of the distribution, and were plotted using R using the kernel densityplot function.

Written by countculture

November 3, 2008 at 5:17 pm

How often do MPs turn up for work (Part 3): attendance week by week

leave a comment »

GEEK CONTENT ALERT: This post is possibly only for those really into the details of attendance figures, as the data is somewhat dense. See also notes at bottom of page for what ‘attendance’ actually means.

Last time I looked at how evenly distributed of MPs attendance was, looking at how the averages were made up by party, and at some of those individual MPs who had particularly low attendance rates.

This time I’m going to investigate the attendance over time — specifically, how it varies week by week. I’ve done this by working out the average attendance for each week, and graphing them over time (see below, click for full-size image).

As you can see, there’s a lot of information on the graph, but there are probably a few key things to point out:

  1. Note the red weeks — these are the weeks where the average voting attendance is less than 50%. Sometimes this is due to the opposition abstaining en masse, for example.
    Other times the reason is less clear — the 16% attendance figure in early 2006, for example. Though there was plenty of business that week, there appears to be only one division — on the report stage of the Climate Change and Sustainable Energy Bill on March 10, in which only 105 (including 4 tellers) out of 646 turned up.
    Perhaps the attendance during the rest of the week was much higher. Unfortunately Parliament gives us no way of telling. It’s not even possible to look at the video of the event to see how full the chamber as unlike the CSpan Archive of the US Congress, video coverage of Parliament is only available for 28 days after it happens.
  2. The periods when the Commons is in recess are in blue, and cover a multitude of things, from general election campaigns, holidays, party conferences (in 2003 and 2004 there were distinct blocks of time off for these, separate from the summer break — now there’s just a single very long break), and time during which, in the words of the official Parliament website, “Members can carry out their other duties”.
    The recess periods tend to be slightly longer than shown, as they typically start sometime during the preceding week.
  3. To me, a lot of the low attendance weeks appeared to be close or adjacent to recess periods. Is this a coincidence, or is there a possibility that MPs have a tendency to, well, bunk off early? Well, sorry to disappoint you, but the figures don’t support it. The average attendance for weeks adjacent to recess periods is 65.3%, for all other weeks 64.7% (see spreadsheet here).

I then wondered what the main parties attendance looked like, so below I’ve overlaid those figures on the same graph. It’s a very dense graph, but quite interesting for the geek at heart (again, click on the graph to see it full size).

There’s loads of info there to delve into, especially when there are strong divergences between the parties, or strong convergencies, for that matter. There are also some potentially suspicious drops, though like the apparent drops around recesses, care should be taken as the figures can have multiple readings.

For example, there’s a drop in attendance in week 20 of 2007 (May 14-18). Attendance this week was only 45.6% (see spreadsheet here), but a closer look at the voting divisions that week, shows that for much of the week voting attendance was relatively high:

Monday 15/05/07
Armed Conflict — Parliamentary Approval, 539 votes cast

Tuesday 16/05/07
Points of Order — Housing, 544 votes
Orders of the Day — Clause 2 — Meaning of “relevant duty of care”, 531 votes

Wednesday 17/05/07
Orders of the Day — New Clause 29 — Reduction of regulatory burdens, 400 votes
Orders of the Day — Clause 81 — “Local improvement targets”: interpretation, 403 votes
Orders of the Day — Clause 179 — Abolition of functions of Patients’ Forums, 376 votes

Thursday 17/05/07
Orders of the Day — Clause 182 — Duty to consult users of health services, 277 votes

However, the figures were brought right back down due to a series of very poorly attended votes on the Friday (May 18).
Motion to sit in private, 56 votes
Freedom of Information (Amendment) Bill — Question Proposed, 144 votes
Freedom of Information (Amendment) Bill — Constituent letters, 132 votes
Freedom of Information (Amendment) Bill — MP’s correspondence, 137 votes
Freedom of Information (Amendment) Bill — Third Reading, 125 votes
Freedom of Information (Amendment) Bill — Third Reading — Closure, 143 votes

The suspicious (and sporting) at heart may recall that the England vs West Indies test match was on at Lords from May 17-21, and speculate that a number of MPs decided to enjoy the second day’s play.

However, another, perhaps more likely, reason for the low attendance is what was being voted upon — a Bill that would have “taken away the public’s right to know about the internal workings of Parliament, or see any correspondence between MPs and any government department or local authority“. Like many such votes on transparency, MPs seemed curiously reluctant to be seen to taking a position against openness, yet apparently have no interest in supporting it either. (The Motion to sit in private, by the way, was a procedural tactic to prevent the bill being filibustered).

Of course, it’s also perfectly possible that both explanations are true 🙂


Next time I’ll compare ministerial voting records vs back-benchers

Notes on the calculations

The above calculations were derived from the voting record freely available from the Public Whip project, and cover the period from May 1997 to July 22, 2008 (when the house rose for the summer recess). The data can be downloaded in the form of a MySQL database, and this was used together with custom MySQL queries to generate the weekly attendance of MPs, grouped by party (available here as a spreadsheet).

The attendance are calculated by whether an MP voted in a division. As noted in my first post on the subject, this is an imperfect proxy for actual attendance, as the figure may be depressed by silent abstentions (i.e. not voting in a division, rather than voting both ‘aye’ and ‘no’) and by just turning up to vote, but failing to attend the debate. However, until Parliament provides a better measure for attendance, this is the only way of calculating it.

Written by countculture

October 14, 2008 at 5:55 pm

How often do MPs turn up for work (Part 2): the good, the average, and the downright lazy

with 6 comments

Last time I briefly looked at MPs average attendance figures, and also the attendance figures for MPs for the three main parties. This time I’m going to examine what makes up those averages — are MPs much of a muchness (the average attendance, remember, is about 64%), or are there wide variations, with the figure being inflated (or reduced) by high (or low) attending MPs.

It’s worth starting by looking at the figures visually. (For the moment, I’m concentrating on the three main parties, and specifically current MPs. This means that the figures we’re looking at are higher than the overall average, as the three main parties’ attendance records are better than the minority parties, and current MPs’ figures are better than for those who’ve left parliament — I’ll perhaps look at why that is another time.)

The graphs below show the proportion of MPs in each attendance band. So the first band represents MPs who vote between 0% and 5% of the time, the next between 5% and 10% and so on.

Chart showing distribution of attendance of current MPs, by main parties

As you would expect given the figures on average attendance, the Labour MPs are more skewed towards the higher attendance figures than are the other parties, with the highest proportion attending between 80% and 85% (compared with Liberals 75-80% and Conservatives 65-70%). As before, that raises the question: why are Conservative MPs attendance rates so much worse than the other two main parties?

The part-timers

Most interesting, perhaps, are those MPs at the bottom end of the scale, who can’t even manage to turn up 50% of the time, and they are shown in the list below.

Attendance Party Constituency Attended/possible
Gordon Brown 14.7% Lab Kirkcaldy & Cowdenbeath 492/3352
Michael Mates 35.5% Con East Hampshire 1189/3352
David Blunkett 36.7% Lab Sheffield, Brightside 1229/3352
Jack Straw 40.7% Lab Blackburn 1363/3352
Charles Kennedy 42.1% LDem Ross, Skye & Lochaber 1412/3352
John Prescott 42.4% Lab Kingston upon Hull East 1420/3352
Ruth Kelly 43.9% Lab Bolton West 1470/3352
Francis Maude 44.6% Con Horsham 1495/3352
David Mundell 44.8% Con Dumfriesshire, Clydesdale & Tweeddale 373/833
Michael Howard 45.0% Con Folkestone & Hythe 1507/3352
Stephen Dorrell 45.1% Con Charnwood 1512/3352
Adam Ingram 45.8% Lab East Kilbride, Strathaven & Lesmahagow 1535/3352
William Hague 46.0% Con Richmond (Yorks) 1543/3352
Michael Ancram 46.2% Con Devizes 1547/3352
Nicholas Clegg 47.4% LDem Sheffield, Hallam 395/833
Frank Cook 47.6% Lab Stockton North 1597/3352
Kenneth Clarke 47.9% Con Rushcliffe 1605/3352
Claire Curtis-Thomas 47.9% Lab Crosby 1605/3352
Richard Caborn 48.2% Lab Sheffield Central 1615/3352
Liam Fox 48.2% Con Woodspring 1617/3352
Robert Walter 48.5% Con North Dorset 1627/3352
Tessa Jowell 48.7% Lab Dulwich & West Norwood 1631/3352
Nicholas Soames 48.9% Con Mid Sussex 1638/3352
Mark Oaten 49.0% LDem Winchester 1635/3337
Patrick Cormack 49.6% Con South Staffordshire 1654/3333
Malcolm Rifkind 49.7% Con Kensington & Chelsea 414/833
Bill Etherington 49.97% Lab Sunderland North 1675/3352

As you can see, some of the 27 MPs appearing on the list are ministers or shadow ministers, including of course Gordon Brown (Tony Blair only managed 8.3% attendance).

The argument of course is that ministers should concentrate on running their department rather than attending every vote; the counter argument is that if it’s important enough to legislate about, it’s important enough for all MPs to discuss and vote on.

In many cases, however, there seems on the face of it little justification for an MP to vote in fewer than 50% of the divisions, and justify it is surely what they should do.

Perhaps they work tirelessly on behalf of the constituents who voted them in. Perhaps they do sterling work investigating and challenging the government’s claims. Perhaps there is some other legitimate reason why they attend so rarely.

Whatever the reason, it is surely fair to expect MPs to explain to their constituents and the citizens for whom they work why attend so rarely.

For Example

Let’s look, for example, at the MPs at the better end of this list, those that almost managed to turn up 50% of the time.

There’s Bill Etherington, Labour MP for Sunderland North (with an attendance rate of just under 50%). He’s apparently leaving Parliament at the next election due to a boundary change, and looks as though he’s counting the days, having spoken in only one debate in the past year, according to They Work For You.

Now in his sixties, according to his Wikipedia profile he started work age 14 in the shipyards, and eventually became a union official, before becoming an MP 16 years ago. You might guess that he’s had enough and is now ready to retire. And indeed, he told the Sunderland Echo, “I shall not be fighting the next election. I’ll be 68 or 69 by then and I’ve done my bit.”  Fair enough, but does that make it OK to be only a part-timer for the rest of his time as MP, and does it really explain why his attendance record has been consistently so poor?

Or what about the MP next to him in the list, Conservative MP for Kensington & Chelsea Malcolm Rifkind, former Foreign Secretary (and other ministerial positions) in the last Conservative government, before losing his Edinburgh seat in 1997. Now he’s been re-elected to Michael Portillo’s old seat is he showing the new boys a thing or two about commitment and hard work? Not if his attendance record is anything to go by.

Again, perhaps there’s a good reason. Being Shadow Secretary of State for Work and Pensions? But that lasted only 7 months. Of course, the suspicion is that perhaps there isn’t enough time, given other outside interests (a directorship, a consultancy, membership of an advisory board, various writing and speaking engagements).

Then there’s another Conservative, Patrick Cormack, MP for South Staffordshire, and currently member of the Liaison Committee and chair of the Northern Ireland Affairs Committee. Having been an MP since 1970, he’s often referred to as a ‘Tory grandee’, is the Chairman and Life President of The House Magazine, and has a number of other outside interests. Yet, since 1997 he’s only turned up to vote 49.6% of the time.

Finally there’s Mark Oaten, Liberal Democrat MP for Winchester, who since 1997 has managed 49% attendance, and since May 2005 just 33.6%. He was Shadow Secretary of State for Home Affairs for the first six months, but 33.6%? Compare that with the Liberal Democrat Shadow Chancellor (and briefly Acting Leader) Vince Cable MP who in the same period achieved 69.5% attendance.

So What?

These four MPs, taken from the better end of the part-timers list, are possibly not representative of the list as a whole. It’s possible even that all 27 MPs have a good reason for their poor attendance record, as it implies a lack of trust.

Possibly. On the other hand, MPs have been time and again been extremely resistant to operating transparently, with many being dragged kicking and screaming to reveal information about outside interests, family members employed as staff, expenses payments, and so on.  So much so that it’s perhaps not surprising that people reading these figures suspect the worst.

I’ll perhaps return to this list in a future post, but in the meantime if any of these MPs represent you, why not drop them a line via Write To Them and ask them to justify their poor attendance record.


Next time I’ll have a look at how attendance rates change week-by-week.

Notes on the calculations

The above calculations were derived from the voting record freely available from the Public Whip project, and cover the period from May 1997 to July 22, 2008 (when the house rose for the summer recess). The data can be downloaded in the form of a MySQL database, and this was used together with custom MySQL queries to generate the attendance of all current MPs (available here as a spreadsheet).

The attendance are calculated by whether an MP voted in a division. As noted in my first post on the subject, this is an imperfect proxy for actual attendance, as the figure may be depressed by silent abstentions (i.e. not voting in a division, rather than voting both ‘aye’ and ‘no’) and by just turning up to vote, but failing to attend the debate. However, until Parliament provides a better measure for attendance, this is the only way of calculating it.

Written by countculture

September 27, 2008 at 10:41 am

Posted in parliament

Tagged with , , ,

How often do MPs turn up for work (Part 1)?

with 3 comments

With MPs are still on their summer break (barring recalls for bank failures), it seems a good time to examine their performance. Consider it an end-of-term report.

Ironically, given how keen the government has been on public-sector targets, there are few ways we can measure how good a job MPs do, and even the most basic — how often do they turn up to work? — has be be worked out through a roundabout way. There’s no attendance register, still less any way of knowing how often they attend constituency surgeries, or other parliamentary work outside of the Commons.

All we have is the fairly rough measure of whether (and how) they voted in any given division. It’s crude and given to over and underestimating (abstentions appear as a no-show, whereas turning up only to vote is the parliamentary equivalent to going to work to get your card stamped), but it’s the best measurement we’ve got, until Parliament itself decides to improve transparency by giving us something else.

Thanks here should go to the people at the Public Whip project who collate this data by parsing Hansard into a form you can then analyse. The figures, however, only go back to 1997, so it’s not possible to compare the figures between a Conservative and Labour government, for example.

The basics

So, how often does the average MP turn up? About 64% of the time. Or to look upon it another way, 36% of the time (over a third) the average MP doesn’t bother to turn up to vote.

That figure is the average for votes since 1997, but it’s been consistently between 59% and 71% (see chart below). As the table shows, the attendance rate for the current crop of MPs is a little better (the main total includes MPs that have since left the house), and that for the House of Lords predictably low.

Attendance rates 1997 – Jul 2008
All MPs 64.5%
All Current MPs 68.3%
Lords 29.9%
Labour MPs 69.8%
Conservative MPs 61.7%
LibDem MPs 64.7%

However, as anyone with rudimentary knowledge of statistics or a healthy degree of scepticism will know, a raw average will hide a multitude of sins, so it’s worth delving into that overall figure a little closer.

The Political Parties

When you break down the attendance figures by party, one thing becomes pretty clear, Labour MPs vote a lot more often than either Conservative or Liberal Democrat ones do. The broad trend (i.e. moving average) shows some improvement over the past few years, but still remains below 70%.

Why is this? Why are the Labour figures so much higher? Maybe it’s something to do with being in power (remember the figures only go back to 1997), or perhaps Labour MPs really do care more, or perhaps they are keen to be seen to be supporting their party’s bills. Or perhaps there are more Labour backbenchers with not much else to do.

Whatever the answer, the Conservative’s low attendance figure does seem at odds with David Cameron’s statement that “MPs are meant to go to parliament; that’s what their job’s about.”

The minority parties have almost universally poor attendance records (with Sinn Fein it was party policy not to attend), and if we exclude those the average (i.e. for the three main parties) rises a little to 66.9%.

Why is the figure so low?

Personally, I’m not sure that’s a great figure. Is it really OK that for over a third of the time MPs had something better to do?

Perhaps if they were ministers, they were busy running their department, which is understandable.

Perhaps they were dealing with urgent constituency business — certainly possible, though I’m not sure it would account for such a high figure.

Perhaps, they attended the debate, but decided to abstain —  though given MPs can vote both ‘No’ and ‘Aye’ at the same time, effectively an active abstention — it would seem unlikely to be the case, or at least be a significant factor.

Perhaps, also, MPs are human like the rest of us, and given a job to do (much of which is mundane) have a tendency to bunk off when there’s no supervision.


Next time I’ll have a look at how evenly spread attendance rates are, in particular whether the average is unduly affected by high- and low-attending MPs.

Notes on the calculations

The above calculations were derived from the voting record freely available from the Public Whip project. The data can be downloaded in the form of a MySQL database, and this was used together with custom MySQL queries to generate a spreadsheet (available here) containing Average MPs voting percentage for 1997- July 2008, and for individual years within that range.

The political parties’ percentages were they obtained by taking the average (specifically the arithmetic mean) of the MPs attendance percentages, so the above figures for the parties are the the average of their MPs average attendance rates.

The attendance figure for the House of Lords is only for divisions to early June 2008.

Photo at top is Parliamentary Copyright.

Written by countculture

September 17, 2008 at 10:34 am

Posted in parliament

Tagged with ,