Open data and all that

Archive for the ‘politics’ Category

A simple demand: let us record council meetings

with 16 comments

A couple of months ago we had the ridiculous situation of a local council hauling up one of their councillors in front of a displinary hearing for posting videos of the council meeting on YouTube.

The video originated from the council’s own webcasts, and the complaint by Councillor Kemble was that in posting these videos on YouTube, another councillor, Jason Kitcat

(i) had failed to treat his fellow councillors with respect, by posting the clips without the prior knowledge or express permission of Councillor Theobald or Councillor Mears; and
(ii) had abused council facilities by infringing the copyright in the webcast images

and in doing so had breached the Members Code of Conduct.

Astonishingly, the standards committee found against Kitcat and ruled he should be suspended for up to six months if he does not write an apology to Cllr Theobald and submit to re-training on the roles and responsibilities of being a councillor, and it is only the fact that he is appealing to the First-Tier Tribunal (which apparently the council has decided to fight using hire outside counsel) that has allowed him to continue.

It’s worth reading the investigator’s report (PDF, of course) in full for a fairly good example of just how petty and ridiculous these issues become, particularly when the investigator writes things such as:

I consider that Cllr Kitcat did use the council’s IT facilities improperly for political purposes. Most of the clips are about communal bins, a politically contentious issue at the time. The clips are about Cllr Kitcat holding the administration politically to account for the way the bins were introduced, and were intended to highlight what the he believed were the administration’s deficiencies in that regard, based on feedback from certain residents.
Most tellingly, clip no. 5 shows the Cabinet Member responsible for communal bins in an unflattering and politically unfavourable light, and it is hard to avoid the conclusion that this highly abridged clip was selected and posted for political gain.

The using IT facilities, refers, by the way, not to using the council’s own computers to upload or edit the videos (it seems agreed by all that he used his own computer for this), but the fact that the webcasts were made and published on the web using the council’s equipment (or at least those of its supplier, Public-i). Presumably it he’d taken an extract from the minutes of a meeting published on the council’s website that would also have been using the council’s IT resources.

However, let’s step back a bit. This, ultimately, is not about councillors not understanding the web, failing to get get new technology and the ways it can open up debate. This is not even about the somewhat restrictive webcasting system which apparently only has the past six month’s meetings and is somewhat unpleasant to use (particularly if you use a Mac, or Linux — see a debate of the issues here).

This is about councillors failing to understand democracy, about the ability to taking the same material and making up your own mind, and critically trying to persuade others of that view.

In fact the investigator’s statement above, taking “a politically contentious issue at the time… holding the administration politically to account for the way the bins were introduced… to highlight what the he believed were the administration’s deficiencies in that regard” is surely a pretty good benchmark for a democracy.

So here’s simple suggestion for those drawing up the local government legislation at the moment, no let’s make that a demand, since that’s what it should be in a democracy (not a subservient request to your ‘betters’):

Give the public the right to record any council meeting using any device using Flip cams, tape recorders, frankly any darned thing they like as long as it doesn’t disrupt the meeting.

Not only would this open up council meetings and their obscure committees to wider scrutiny, it would also be a boost to hyperlocal sites that are beginning to take the place of the local media.

And if councils want to go to the expense of webcasting their meetings, then require them to make the webcasts available to download under an open licence. That way people can share them, convert them into open formats that don’t require proprietary software, subtititle them, and yes, even post them on YouTube.

I can already hear local politicians saying it will reduce the quality of political discourse, that people may use it in ways they don’t like and can’t control.

Does this seem familiar? It should. It’s the same arguments being given against publishing raw data. The public won’t understand. There may be different interpretations. How will people use it?

Well, folks that’s the point of a democracy. And that’s the point of a data democracy. We can use it in any way we damn well please. The public record is not there to make incumbent councillors or senior staff memebers look good. It’s there to allow the to be held to account. And to allow people to make up their own minds. Stop that, and you’re stopping democracy.

Links: For more posts relating to this case, see also Jason Kitcat’s own blog postsBrighton Argus post, and posts form Mark Pack at Liberal Democrat voice, Jim Killock,  Conservative Home, and even a tweet from Local Government minister Grant Shapps.


Written by countculture

September 27, 2010 at 12:46 pm

Right to Government Data… unless it’s been outsourced

with 5 comments

The UK’s coalition government has followed up on its promises of a general Right To Public Data, with sweeping and fast-moving measures to open up whole sectors of public sector information, from the salaries of senior civil servants to spending by local councils.

For those who attended the pre-election Post-Bureaucratic Age (PBA) conference (which was heavily populated by Conservative thinkers), this is not such a surprise, and it’s also worth acknowledging the steps the previous government took – principally making the decision to open up Ordnance Survey data.

But, there’s one thing that nobody’s talking about, and significantly it’s a question that was asked (but ducked) at the final PBA panel: what about government data that happens to be held by the private sector, principally information that’s been outsourced?

Now lest anyone think this is nitpicking here, we’re not talking about the odd dataset here and there – whole areas of government work, both national and local, have been outsourced to the private sector, including the almost the complete frontline services for some councils.

The problem with this – as far as information goes – is that private sector companies aren’t covered by the same regulations as public sector bodies. They aren’t, for example, subject to FoI requests, even if they are doing the work that would have been done by the private sector (actually it’s much worse than this: even if companies are 90% owned by a public body they are not subject to FoI requests, nor, currently are Joint Ventures between public bodies).

The same problem looks likely to afflict the right to data, with no Right To Data held by private companies (or JVs) on behalf of public bodies.

[I should state here, that I’m no fan of outsourcing – buying services as a commodity, yes; outsourcing no – for more on this see my presentation Open Data & The Rewards of Failure. Too often it’s an unequal balance, with the contractor knowing way more than the purchaser, and often is carried out by companies just as bureaucratic as public bodies.]

A current example portal

Enough about the generalities. Let’s look at a specific case – one that was raised at the EPSI conference in Madrid last week, by Jose L. Marin from Euroalert. Turns out they’d been trying to get access to the information from the government tender portal, which is run on behalf of the Department for Business, Innovation and Skills (BIS) by BiP Solutions. In fact, according the the website, BiP Solutions are [merely] the service provider for the portal and are… responsible for its development, support and maintenance.

There’s clearly a case for making this information as widely available as possible, as you want the public bodies to have as good a choice as possible.

However, despite their best efforts, Euralert have not had any luck getting access to the data. They’d even enlisted the help of the good people from OPSI, who are tasked with facilitating access to public data. But even this didn’t work, as last week they received an email from OPSI, with this response from BIS.

As you are aware, both BIS and BiP have refused Mr. Marin’s request. There are both policy and commercial reasons why this decision has been made.

It is BIS’ view that the Re-use of Public Sector Information Regulations 2005 are not applicable in the circumstances. The information which Mr. Marin is seeking is not “held by a public sector body” as required by Regulation 4; BIS is a public sector body, but does not hold the information, BiP holds the information, but is not a public sector body. Furthermore, Regulation 7 makes it clear that decisions on re-use are a discretionary matter for the public sector body concerned. In the event that we considered that the Regulations did apply, we would nonetheless have still decided not to permit the re-use Mr. Marin requests for policy and commercial reasons.

In the alternative, if Mr. Marin considers that the information on Supply2Gov is held by the public sector bodies which placed the contract opportunities on the site, then the request for re-use in relation to each contract opportunity should be made to the public sector body concerned, not to BIS, which does not hold the information.

There’s something rather patronising and condescending about this response that I find quite appalling. In short, we don’t want you to have the data, and because it’s held by a private company there’s nothing you can do about it, and frankly you wouldn’t understand the reasons even if we could be bothered to tell you. In other words (and in the immortal words of the French soldier from Monty Python & The Holy Grail): ‘I fart in your general direction.

It seems clear to me that if the coalition is genuinely committed to open data, it will stamp out this sort of thing immediately – it’s a concern that the letter was sent post-election, and from a department headed by Liberal Democratic Vince Cable.

But whichever way it goes, make no mistake, if we don’t get the right to reuse public data that has been outsourced to the private sector, we will not only be missing out on the benefits of open data, but I suspect we will also see the most politically embarrassing data being outsourced just to hide it from public view.

Update: Just as I finished writing this I saw via Twitter that the Office of Government Commerce was being taken away from the BIS and moved to the Cabinet Office, home to many of the best people in the UK Government helping open up data. Given that the OGC is responsible for tendering/procurement it ‘s to be hoped that not only does go with it, it gets opened up as part of the Cabinet Office’s opendata work.

Written by countculture

June 15, 2010 at 3:54 pm

Open data meme suggestion: Enabler or blocker?

with 2 comments

Are you a blocker or enabler?

Earlier today I gave a presentation at the Open Knowledge Conference on open local data, OpenlyLocal and the Open Election Data project. It was a slight update of the talk I gave to the Manchester Social Media Cafe earlier in the month, and one of the key additions was a simple idea I added on the final page, which was about where we should go from here.

I’d been using the idea in conversation for the past months ago (and I’m sure I didn’t invent it), but it seemed to resonate with the audience, and so I thought it’s worth repeating as a short blog post, and it’s this:

When dealing with government, with organizations, with public officials, with outsourcing companies we need to develop the meme:

Are you an enabler or a blocker?

It’s a blunt and somewhat unsophisticated weapon, but in the past few months of doing the Open Election Data project, it seems to have been far more effective that any other I’ve tried — better than appealing to the public good, better than engaging on an intellectual level, better than asking for it nicely, better even than talking about potential savings.

Maybe it’s because, as someone suggested to me after the first meeting of the UK government’s Local Public Data Panel on which I sit, civil servants and other public officials only do things because there’s a benefit to them (or a downside if they don’t). [I’m not sure they’re any different than most people working in the private sector in this respect, by the way.] I don’t know, and I don’t really care. What I do care about is getting things done, and this seems to be working for me.

So, I offer it out there, not as an original idea (I’m sure it isn’t), but as a suggestion of both engaging with public bodies, and as a method of dealing with problems.

When you come across people or organisations given them the option: do you want to be an enabler or a blocker. If you’re an enabler, great, let’s see how we can make this work; if you’re a blocker, fine also — now we know we’ll just go around you and get on with it anyway.

Written by countculture

April 24, 2010 at 2:23 pm

MP attendance: end of term report

with 3 comments

[Note: Voting attendance is an imperfect proxy for actual attendance, as the figure may be depressed by silent abstentions (i.e. not voting in a division, rather than voting both ‘aye’ and ‘no’) and by just turning up to vote, but failing to attend the debate. However, until Parliament provides a better measure for attendance, or more transparency of MPs actions, this is the only one we have.]

It’s recess time again, and time for MPs’ end-of-term report. I’ll leave it to others to comment on how they’ve dealt with some of the genuinely momentous events since the summer recess. This post deals solely with their voting attendance record.

First off, let’s have an overall look at the overall figures for the period:

Oct-Dec 08 May 97-Jul 08
All MPs 70.2% 64.5%
74.8% 69.8%
67.8% 61.7%
72.2% 64.7%

The figures above are pretty self-explanatory. All parties have improved their attendance of votes, by 5 to 8 percentage points. Perhaps not surprising given the financial crisis.

Now let’s have a look at the main parties in detail, using the same histograms used before to show the distribution of the parties attendance figures. Interestingly (well, in a wonkish sort of way), the distributions are a bit more spread out than the long-term average. In part this is probably down to the shorter time period showing up variations that are hidden in longer period, but it’s interesting nevertheless to note that though all parties have improved their overall attendance figures, the number and proportion of Labour MPs who’ve voted in fewer than half the divisions has nearly tripled, from 11 MPs to 30 of them.


[Note: there’s no significance to the width of the columns — the recent ones are narrower so that both can be seen on the same graph]

Finally, let’s have a look at those outliers, first, the MPs who attended divisions less than than 50% of the time:

Attended fewer than 50% of divisions Oct–Dec 08

Name Party Constituency Attendance % votes attended/possible
David Cameron Con Witney 28.6% 24/84
Robert Walter Con North Dorset 34.5% 29/84
Michael Mates Con East Hampshire 35.7% 30/84
Tim Yeo Con South Suffolk 35.7% 30/84
Michael Howard Con Folkestone & Hythe 36.9% 31/84
Peter Bottomley Con Worthing West 40.5% 34/84
Liam Fox Con Woodspring 40.5% 34/84
Julian Lewis Con New Forest East 42.9% 36/84
Caroline Spelman Con Meriden 42.9% 36/84
David Tredinnick Con Bosworth 42.9% 36/84
David Wilshire Con Spelthorne 42.9% 36/84
David Mundell Con Dumfriesshire, Clydesdale & Tweeddale 42.9% 36/84
Malcolm Rifkind Con Kensington & Chelsea 45.2% 38/84
Michael Gove Con Surrey Heath 45.2% 38/84
William Hague Con Richmond (Yorks) 46.4% 39/84
Mark Lancaster Con North East Milton Keynes 48.8% 41/84
Mark Oaten LDem Winchester 23.8% 20/84
Daniel Rogerson LDem North Cornwall 47.6% 40/84
Nicholas Clegg LDem Sheffield, Hallam 47.6% 40/84
Margaret Hodge Lab Barking 0.0% 0/84
Jessica Morden Lab Newport East 0.0% 0/84
Gordon Brown Lab Kirkcaldy & Cowdenbeath 3.6% 3/84
Kali Mountford Lab Colne Valley 14.3% 12/84
David Miliband Lab South Shields 19.0% 16/84
Khalid Mahmood Lab Birmingham, Perry Barr 21.4% 18/84
Gordon Banks Lab Ochil & Perthshire South 27.4% 23/84
Alistair Darling Lab Edinburgh South West 29.8% 25/84
Adam Ingram Lab East Kilbride, Strathaven & Lesmahagow 29.8% 25/84
Glenda Jackson Lab Hampstead & Highgate 31.0% 26/84
Mike Wood Lab Batley & Spen 32.1% 27/84
Frank Cook Lab Stockton North 33.3% 28/84
Geraldine Smith Lab Morecambe & Lunesdale 36.9% 31/84
Rudi Vis Lab Finchley & Golders Green 39.3% 33/84
Claire Curtis-Thomas Lab Crosby 40.5% 34/84
Alan Milburn Lab Darlington 40.5% 34/84
Keith Vaz Lab Leicester East 40.5% 34/84
Tom Harris Lab Glasgow South 40.5% 34/84
Doug Henderson Lab Newcastle upon Tyne North 42.9% 36/84
Denis Murphy Lab Wansbeck 42.9% 36/84
Bill Etherington Lab Sunderland North 44.0% 37/84
Kate Hoey Lab Vauxhall 45.2% 38/84
Denis MacShane Lab Rotherham 45.2% 38/84
Stuart Bell Lab Middlesbrough 46.4% 39/84
Joe Benton Lab Bootle 46.4% 39/84
Roger Godsiff Lab Birmingham, Sparkbrook & Small Heath 46.4% 39/84
Bill Rammell Lab Harlow 46.4% 39/84
John Austin Lab Erith & Thamesmead 47.6% 40/84
Caroline Flint Lab Don Valley 47.6% 40/84
Robert Marshall-Andrews Lab Medway 48.8% 41/84

No surprise that GB‘s in this list, voting in only 3 divisions (see here for the divisions he did take time out to vote on). Ditto Alistair Darling and David Cameron.

However, the rest of the list is more interesting. Some of those on the front bench, for example (e.g. Jacqui Smith, Jack Straw), surprisingly don’t make the list, i.e. they voted in at least 50% of the divisions. Ditto some of the opposition spokespeople.

But what about the backbenchers who are on the list. Possibly there’s a good reason for Margaret Hodge and Jessica Morden for failing to attend a single division — illness perhaps (though there’s nothing on either of their websites to indicate such a factor)? And what about Kali Mountford (14.3%) and Khalid Mahmood (21.4%).

If I was in their constituency, I’d like to know, particularly since they took little part in debates, either. Similarly for the low-raters for the Conservatives — Michael Mates and Tim Yeo (6 directorships!) at 35.7% each.

Now the MPs who voted more than 90% of the time:

Attended more than 90% of divisions

Name Party Constituency Attendance % votes attended/possible
George Young Con North West Hampshire 91.7% 77/84
Willie Rennie LDem Dunfermline & Fife West 90.5% 76/84
Andrew Stunell LDem Hazel Grove 95.2% 80/84
John Hemming LDem Birmingham, Yardley 97.6% 82/84
Ian Cawsey Lab Brigg & Goole 90.5% 76/84
Paul Clark Lab Gillingham 90.5% 76/84
John Cummings Lab Easington 90.5% 76/84
John Heppell Lab Nottingham East 90.5% 76/84
Fraser Kemp Lab Houghton & Washington East 90.5% 76/84
Phyllis Starkey Lab Milton Keynes South West 90.5% 76/84
David Heyes Lab Ashton-under-Lyne 90.5% 76/84
Kevan Jones Lab North Durham 90.5% 76/84
Ian Lucas Lab Wrexham 90.5% 76/84
Siân James Lab Swansea East 90.5% 76/84
Barbara Keeley Lab Worsley 90.5% 76/84
Clive Betts Lab Sheffield, Attercliffe 91.7% 77/84
Angela Eagle Lab Wallasey 91.7% 77/84
Maria Eagle Lab Liverpool, Garston 91.7% 77/84
John Healey Lab Wentworth 91.7% 77/84
Keith Hill Lab Streatham 91.7% 77/84
Alun Michael Lab Cardiff South & Penarth 91.7% 77/84
David Taylor Lab North West Leicestershire 91.7% 77/84
Natascha Engel Lab North East Derbyshire 91.7% 77/84
Shahid Malik Lab Dewsbury 91.7% 77/84
Liz Blackman Lab Erewash 92.9% 78/84
Bob Blizzard Lab Waveney 92.9% 78/84
Jeff Ennis Lab Barnsley East & Mexborough 92.9% 78/84
Fiona Mactaggart Lab Slough 92.9% 78/84
Andrew Miller Lab Ellesmere Port & Neston 92.9% 78/84
Kerry McCarthy Lab Bristol East 92.9% 78/84
Mary Creagh Lab Wakefield 92.9% 78/84
Lyn Brown Lab West Ham 92.9% 78/84
Kevin Barron Lab Rother Valley 94.0% 79/84
Janet Dean Lab Burton 94.0% 79/84
Jim Fitzpatrick Lab Poplar & Canning Town 94.0% 79/84
Mike Hall Lab Weaver Vale 94.0% 79/84
David Kidney Lab Stafford 94.0% 79/84
Shona McIsaac Lab Cleethorpes 94.0% 79/84
Judy Mallaber Lab Amber Valley 94.0% 79/84
James Plaskitt Lab Warwick & Leamington 94.0% 79/84
Nick Raynsford Lab Greenwich & Woolwich 94.0% 79/84
Angela Smith Lab Basildon 94.0% 79/84
Ann McKechin Lab Glasgow North 94.0% 79/84
Clive Efford Lab Eltham 95.2% 80/84
David Hanson Lab Delyn 95.2% 80/84
Dan Norris Lab Wansdyke 95.2% 80/84
Bill Olner Lab Nuneaton 95.2% 80/84
Stephen McCabe Lab Birmingham, Hall Green 96.4% 81/84
John Spellar Lab Warley 96.4% 81/84
Dave Watts Lab St Helens North 96.4% 81/84
Helen Goodman Lab Bishop Auckland 96.4% 81/84
Andrew Gwynne Lab Denton & Reddish 96.4% 81/84
Neil Gerrard Lab Walthamstow 97.6% 82/84
Brian Jenkins Lab Tamworth 97.6% 82/84
Helen Jones Lab Warrington North 97.6% 82/84
Thomas McAvoy Lab Rutherglen & Hamilton West 97.6% 82/84
Chris Bryant Lab Rhondda 97.6% 82/84
Chris Mole Lab Ipswich 97.6% 82/84
Diana Johnson Lab Kingston upon Hull North 97.6% 82/84
Tony Cunningham Lab Workington 98.8% 83/84
Dennis Skinner Lab Bolsover 100.0% 84/84

Some amazing figures in there. In fact, given his 100% record you wonder if the Beast of Bolsover has got a home to go to.


Notes on calculations

  • The above calculations were derived from the voting record freely available from the Public Whip project, and cover the period from Oct 2008 to Dec 2008. The data can be downloaded in the form of a MySQL database, and this was used together with custom MySQL queries to generate the figures.

Written by countculture

December 31, 2008 at 7:02 pm

How often do MPs turn up for work (Part 4): the ministerial effect

leave a comment »

[Note: Voting attendance is an imperfect proxy for actual attendance, as the figure may be depressed by silent abstentions (i.e. not voting in a division, rather than voting both ‘aye’ and ‘no’) and by just turning up to vote, but failing to attend the debate. However, until Parliament provides a better measure for attendance, or more transparency of MPs actions, this is the only one we have.]

A frequent arguments for low attendance of voting divisions by MPs is that the figure is depressed by ministers (and shadow spokespersons), whose other responsibilities prevent them from attending as many votes (as they’d like to), thus bringing down the overall average.

Seems reasonable, so let’s have a look at just how much of an influence this ‘ministerial effect’ has on the overall figures. First, let’s look at the average voting attendance for ministers and non-ministers (calculation details below):

Attendance rates May 97 – July 08
All MPs 65.1%
Non-Ministers 64.4%
Ministers 67.2%

Er, wait a minute, so the average voting attendance rate for ministers is higher than non-ministers? That’s not what we expected. However, basic averages (i.e. the mean) can hide a multitude of sins, so let’s have a look at the distribution of those attendance figures.

As you can see, while the peak of the ministerial attendance is around the 65% mark (less than that for the non-ministerial one), there were far more divisions in which 90%+ of ministers voted than there were for which 90%+ of non-ministers voted.

This makes sense, in a way, as ministers are far more likely than backbenchers to turn up en masse for votes their party sees as important. It’s this that largely accounts for the figures we saw in the table above. However, what the graph also shows is that when you take the ministers out of the equation, attendance definitely does not shoot up. There is, in short, no ‘ministerial effect’ to account for the low attendance of MPs.

[It’s worth mentioning that the ministerial office records are slightly incomplete — the record of Parliamentary Private Secretaries is missing during some periods — so I’ve run the figures for ministers both including and excluding PPSs. As you can see, it doesn’t make a lot of difference.]

The party lines

Having looked at the big picture, it’s time to look at the ministerial vs non-ministerial attendance by party, specifically the three main parties in Parliament.

As you can see, the relationship between ministerial and non-ministerial attendance is noticeably different for each of the parties. Labour ministers do indeed have noticeably lower attendance rates than their backbenchers, though not as much as I’d expected and not enough to alter the distribution massively.

However, for the Tories and LibDems, the surprising thing — for me, at least — was the attendance rates for their spokespersons are actually noticeably better than their backbenchers, raising rather than lowering the overall figures. What, I wonder, is the reason for this?

Finally, a couple of quick graphs to wrap this post up. One shows, perhaps not surprisingly, that Labour ministerial attendance rates are less than for the shadow spokespersons — presumably the time commitment for a governmental position is greater than that for the equivalent shadow position.

The other shows the distribution of backbenchers attendance figures, by party. I’ll leave that one without making any further comment.


Notes on calculations

  • The Ministerial/non-ministerial attendance rates were calculated by looking at every Commons division between May 1997 and July 2008, and working out the number of ministers/non-ministers who could have voted in that division, and the number who actually did vote. The average attendance figures in the table were calculated by dividing the aggregate number of votes by the aggregate number of possible votes.
    To calculate the distribution of attendance rates I calculated the ministerial/non-ministerial attendance rate for each division, and plotted these on a graph to show how those attendance rates are distributed (as usual, I’ve made the underlying figures are available as a spreadsheet here and here if you want to examine them further).
  • Ministers are those holding any sort of ministerial office as per the PublicWhip database, including whips, but excluding select committee members (although it wouldn’t be hard to run the figures to include select committee members). The Parliamentary Private Secretaries record at the Public Whip is incomplete for several periods, and unfortunately (and ridiculously) there is no historical record of ministers available from Parliament’s own website.
  • The above calculations were derived from the voting record freely available from the Public Whip project, and cover the period from May 1997 to July 22, 2008 (when the house rose for the summer recess). The data can be downloaded in the form of a MySQL database, and this was used together with custom MySQL queries to generate the figures.
  • The graphs are visual representations of the density of the distribution, and were plotted using R using the kernel densityplot function.

Written by countculture

November 3, 2008 at 5:17 pm