Posts Tagged ‘techie’

Wisdom of Crowds Mark II

May 21, 2010

After twitVoteUK comes soccervote. You may know the FIFA World Cup kicks off in South Africa in June, and so we’ve put together a World Cup prediction site at www.soccervote.org. The backend is pretty much the same as twitVoteUK, but Chris has put a lot of work into the frontend to offer a global view, refine the form and add lots of language support, given we want a global submitting audience.

There is a subtle difference behind this effort. Last time it was about obtaining people’s opinions in order to get a poll to try to reflect how the nation might vote. Those people were an attempt to see if that sample set would predict the outcome of the election. They were representing the people who would go to the polls earlier this month and place their votes for real. This time the people who add their predictions on our site are not going to be playing in South Africa, they have no influence on the outcome of the World Cup. Instead this is a wisdom of crowds experiment, ie can the majority view of people who submit an opinion on our site accurately predict the winners of the World Cup? Indeed, will the second, third and fourth listed teams make the semi-finals and so on? Once the tournament begins and teams get knocked out we’ll use the rules in our event processing system to prevent folks from voting for invalid teams, and if volumes of votes increase, then we’ll modify rules to highlight particularly interesting countries of origin in terms of frequency of voting or frequency of change of leader.

What would be good, though, was if we could include Scotland in the list of contenders. Unfortunately we missed out on qualification again. 😦

Advertisements

TwitVoteUK: the aftermath

May 16, 2010

It’s over a week since the election, and how did TwitVoteUK end up? On a pure results level it was completely wide of the real result. Overall we had around two-thirds of votes for the Liberal Democrats, against 23% in the election. Labour and the Conservatives came second and third, but with 10% and about 7% respectively. Lots of the seats that we had votes recorded for only had one vote, meaning that the likes of the Pirate Party UK would have won seats according to our poll. The overall percentages were similar to other online polls, such as CountMyVote, so I’m not disappointed with the final stats. We did reflect the mood of Twitter, which anyone would have seen by following either the #ge2010 or #ukelection tags. We received just over 1500 votes from unique Twitter users, with many people voting more than once, and taking advantage of our facilities to change your vote without having duplicates. In my opinion, to have produced good numbers for each constituency we would have needed at least 100 votes per seat, or 65000 overall, which was far too difficult without some relentless advertising and press coverage. We did get a piece in the Isle of Wight County Press and the Ventnor Blog, both of which managed to drive the Isle of Wight seat to the top of our leaderboard for number of votes.

We received some positive comments in technical blogs, such as Andy Piper and James Governor. I was very pleased to get respect bloggers and social media people giving positive feedback on what we’d done on the site. Also, Ben Marsh, whose uksnow website was the inspiration, tweeted some very positive comments:

If I was going to build a #uksnow inspired election map app, I’d probably end up with something like @twitvoteuk. Nice work chaps. #ge2010

In terms of the result, okay, it was not where near the election result, but it did show that you could gather seat-by-seat data and display that on a map. The gap between each vote and it appearing on the web was no more than two minutes, and it would be fairly straightforward to reduce the window quite considerably. The technology we were using allowed us to restrict each Twitter user to holding only one vote, and to only voting within intervals we set, normally 24 hours. It was quick to change that window, for example during the leaders’ debates we cut it to 5 minutes, so that people could be influenced by what they were watching.

Going forward, we’ve got something on the go that will have more world wide appeal, and some improvements to the backend in order to make it more robust and increase the response time between the vote and the map being updated.

Finally, thanks to Chris Phillips and Ben Hardill, with whom I’ve been working on this with. There is no way I could have done this on my own because I have insufficient knowledge about web technologies to have scoured Twitter for the votes or to do the website, and Chris is really pushing the rest of us on the next rev.

UK Election Polling on Twitter

April 24, 2010

Ok, this post is way off beam compared to what’s gone before, so if you are neither interested in a) Twitter; b) the UK General Election, then don’t read any further.

Over the past couple of weeks I’ve been spending lots of time polishing off an idea I had a couple of months ago. It was inspired by the site that Ben Marsh put together at http://uksnow.benmarsh.co.uk/, using a hashtag on twitter to gather the observations of folks looking out their window at the snowfall when the UK was hit by a couple of snowstorms in January/February of this year. You could look at his map of the UK and at the rainfall radar charts on the Met Office website and see a direct correlation between the two, verifying the accuracy of the crowd-sourced data on the uksnow website.

I thought that a similar idea could be applied to people’s opinion on who they would vote for at the upcoming general election. The main problems were:

  • Ben used the first part of people’s postcode to put a symbol on a map, but quick research showed this wouldn’t be possible for mapping to constituencies, which I wanted to do. Postal districts and constituency boundaries do not match. Therefore the full postcode would be needed and would people stick that in a tweet?
  • Ben had a simple score out of ten for folks to measure how heavy the snow was. This, in my system, would be the party they’d vote for, but there are many of those, and the variety of spellings and abbreviations people might use could lead to missed votes.

Ideally I’d like to have just parsed tweets to establish sentiment, but I quickly dismissed this as being far too difficult, although I note there are a couple of sites experimenting with this approach, eg tweetminster.

So the journey began when I spoke to the renowned Twitter expert at work, IBM Distinguish Engineer and fellow Isle of Wight resident, Andy Stanford-Clark. He created the twittering ferries and twittering house. He reckoned it was a worthwhile idea, and put me in touch with a couple of other folks who had done web and twitter based applications, Chris Phillips and Ben Hardill. After discussing my idea with them, they were sufficiently interested to join and help me to put it together, and I especially needed their help because, hey, I’m not particularly technically proficient, and know nothing about websites beyond very basic html.

We didn’t do a great deal for a few weeks, mostly due to any of us being away from the office, other than putting together an architecture picture. Fast forward through the three of us finally pulling stuff together, and Chris registering our domain and pulling in one of his friends to design the website, and twitvoteuk was born. The how to page on the site describes how the internals work and so does a blog posting from another Andy from work (Piper this time), so I won’t cover it here.

How will this play out? I don’t know, but the most important is to get as many opinions recorded in our system as possible. Two interesting things have happened. The first was last Monday, shortly after we’d activated a form on the website so tweeps didn’t have to post their full postcode to the web, and some LibDem mailing list must have published our website and encouraged folks to use it, and for a while that afternoon Chris and I were chatting on instant messanging, asking each other what was wrong with our system because every single vote coming in was for the LibDems. Fortunately all was good with the system, it was only a party mobilising its supporters. Ever since, they’ve stayed ahead.

The other incident was on the evening of the same day when I noticed that someone had posted an opinion for the Pirate Party UK in a constituency where the BBC didn’t show they had a candidate. I tweeted about this from our @twitvoteuk id, and quickly got a response back from one of their supporters saying we didn’t state whether we were polling for parties people would vote for, or who they would like to vote for. Ever since, there has been the occasional tweet encouraging Pirate Party supporters to grab available constituencies and they remain in second place in our poll.

A final note – while I have political opinions, I do not support any particular party and this endeavour has no political aim. It simply offers a technical solution to the problem of polling a population of people and visualising that data in a geographical way.

Twitter

January 15, 2010

I’ve been using Twitter for a several months now, but never mentioned it here. If you use Twitter and would like to follow me, I’m @kieronb1.

September Time

September 13, 2009

We’re well into September, and that means Hannah is back to school and the nights are drawing in. Even Ella has remarked on that, pointing out her window at bedtime and saying ‘it’s dark’. How do you explain the seasons and the rotation of the Earth around the Sun to a two year old?

While the return to school doesn’t mean much to Ella (other than less time with Mummy, which is, of course, important), it does mean that Dancing Tots has restarted and she can get her weekly fix of dancing randomness lessons. Apparently she really enjoyed it this week. She does do lot of dancing whenever she hears music, which I guess must be a good thing.

What isn’t improving is her ability to share. No doubt I’ve mentioned this before. Yesterday she pushed another kid at the play-park who tried to share some climbing equipment with her. Usually she isn’t that aggressive, but it is something we have to constantly watch out for and coach her to let other children near whatever she is playing with, whether it’s large or small.

As I said above, Hannah is back to school, and already complaining that she might be losing her voice. I think that’s normal at the start of the school year. She did get a big promotion at the start of term, which recognises her as a very good teacher. She is very pleased with that, and Ella and I are very proud. The start of term also means that choir is back. Check the Facebook group for friends of the choir or on YouTube for stuff on Medina High Community Choir. The early reherseals have been tough going, as Hannah has introduced some trickier new material, but I’m sure we’ll work our way through it. Some of the new material has been arranged by Hannah, which isn’t a new thing, but now is digitally enhanced, after I finally cracked how to use Cakewalk Music Creator, that I bought for Hannah several months ago. Having worked out how to use it, I realised it lacked some features for printing music (ties, repeats as examples), so downloaded a rival product, Finale PrintMusic, which once we got the right version, does almost all Hannah needs it to. Cakewalk might be good for other things, but not for printing out music. As part of this success, I managed to get Hannah’s Clavinova to talk to her laptop (by simply finding the right instructions and following them). This means Hannah can play the music on the Clavinova, and the application on the laptop will write up the music notation. Cool when it works.

Obviously I’m working away too, having been very focussed on a process milestone at work. That’s nearly done, but IBM being IBM, I can say no more.

New Phone on the Horizon

July 1, 2009

My phone is (I think) 3 years old. It’s a Windows Smartphone with touchscreen, and does most of what a modern smartphone does. It is a bit long in the tooth, though and quite blocky, and Windows Smartphone 2003 is not the best mobile OS. Plus I’d like to merge my MP3 player (a Creative Zen; also aging: I bought it for Hannah and I’s holiday to Mauritius in 2006) and therefore reduce my small device charging requirement by 1 and have one less device to carry around.

I’d had a look at the HTC Touch 3G on Orange, as that’s the network I’m on now, and the phone I have now is a rebadged HTC, so I’m used to the manufacturer. It’s another Windows Smartphone and that would mean being able to easily access stuff on my Windows desktop (files, music etc), install Yahoo Go for email and contacts, and get all the normal things off the internet that you can for Windows. It’s nothing special as a smartphone, though. I don’t want an iPhone, as I don’t have an iPod (most of my music is in WMA format), don’t have an Apple Mac and don’t use iTunes. I had looked at the HTC Magic, one of the Google Android phones, but that would mean switching network to Vodafone, and Hannah would still be on Orange, meaning higher charges for her calling me, and so had dismissed it. But… two of the guys in my team have just got one (with rebate from the IBM Rewards scheme for employees) and both rave about it and the apps it comes with. The major advantage of getting a new phone seems to be location awareness, ie the phone knowing where it is and being able to link you to local services etc.

I am tempted by the Magic, we’ll see what transpires.