Info Clarity

What does one trillion dollars LOOK like?

2009-03-25T12:57:00.003Z

It's amazing how difficult it is to make sense of large numbers. I think they really do need to be compared to something to make sense. I loved an approach that the New York Times took a couple of years back.

The following is another great example from an email forward that I just got.

*****

All this talk about "stimulus packages" and "bailouts"...

A billion dollars...

A hundred billion dollars...

Eight hundred billion dollars...

One TRILLION dollars...

What does that look like? I mean, these various numbers are tossed around like so many doggie treats, so I thought I'd take Google Sketchup out for a test drive and try to get a sense of what exactly a trillion dollars looks like.

We'll start with a $100 dollar bill. Currently the largest U.S. denomination in general circulation. Most everyone has seen them, slighty fewer have owned them. Guaranteed to make friends wherever they go.

A packet of one hundred $100 bills is less than 1/2" thick and contains $10,000. Fits in your pocket easily and is more than enough for week or two of shamefully decadent fun.

Believe it or not, this next little pile is $1 million dollars (100 packets of $10,000). You could stuff that into a grocery bag and walk around with it.

While a measly $1 million looked a little unimpressive, $100 million is a little more respectable. It fits neatly on a standard pallet...

And $1 BILLION dollars... now we're really getting somewhere...

Next we'll look at ONE TRILLION dollars. This is that number we've been hearing about so much. What is a trillion dollars? Well, it's a million million. It's a thousand billion. It's a one followed by 12 zeros.

You ready for this?

It's pretty surprising.

Go ahead...

Scroll down...

Ladies and gentlemen... I give you $1 trillion dollars...

(Look at the little guy in the corner at the left. And notice those pallets are double stacked.)

So the next time you hear someone toss around the phrase "trillion dollars"... that's what they're talking about.

I only read the Beano

2009-01-25T20:23:00.004Z

Amongst many fascinating stories in Malcolm Gladwell's Outliers, chapter eight has one that brings back a strange memory for me. And one that should remind us to always question whether the data we are shown really supports the conclusions we draw from it.

Consider the following table of scores achieved by kids in Baltimore public schools across 1st-5th grade. (The test referred to is the California Achievement Test, but that's not important to the example.)

What conclusions might we draw from this? I think we might reasonably start to think that Baltimore public schools were failing low income pupils. They start off with only a slight disadvantage from their moneyed peers (32 poits), but end school significantly under-performing them (73 points).

You get a very different story if you look just at what happens during the time pupils were in school. Karl Alexander tested pupils at the start and at the end of every school year, enabling him to measure how many points they gained while actually in school. Here are the results:

Now it seems that, if anything, schools are of more benefit to poorer kids. Across the five grades, during the school years, they gaines 189 points, while the wealthy kids gained only 184 points. The difference between the first table and the second lies in how many points the pupils gained or lost during the long summer holidays:

And if we know this, we know that the story isn't about education at all. Its about what happens in the school holidays. Poor kids on average neither gain nor loose points over holidays. But richer kids consistently gain. So by the end of the school period, they outperform their peers.

So the analytical lesson here is to always be careful that the data you are using really supports the conclusions you are drawing from it. Ask yourself: If you cut the data differently, might it tell a completely different story? If so, give it a shot.

And what of the Beano you might ask? I have an odd memory in primary school of my class gathering around the teacher after the long summer break one year. He asked us to each pick one book we'd read over the summer and tell the class about it. One by one the pupils in my class told their classmates about one of the books they had enjoyed. When the teacher pointed to me, I had to ask whether a comic counted, since that was all I'd read over the summer.

1.4 million New Yorkers in DC for Inauguration!

2009-01-20T08:07:00.002Z

... or so it might seem if you take polling results seriously.

In a recent poll, 9% of New Yorkers said they were planning to head to DC for the event. There are about 16 million adults in the New York area, suggesting 1.4 million people planned to make the trip. The lesson here is that there are some things you shouldn't use polling for!

From pollster.com: One problem with a question like this one may be that it lends itself to social desirability bias. As we know, citizens tend to over-report the extent to which they will (or did) vote in elections. In a similar way, some respondents may be proclaiming that they will attend the inauguration when they don't have any real intention of going. They may do so because they hear of so many others who are attending and they feel as though it is something they should be doing as well.

This reminds me of another article pointing out something else you shouldn't use polls for: asking people whether they were at historic events.

From Political Animal: I remember reading, years ago when I lived in Miami, that a significant percentage of the population of South Florida believes they were in attendance for the famous Dolphins-Charges playoff game in 1982. That's impossible, of course, since the capacity of the Orange Bowl was only about 75,000, and the population of Miami-Dade is in the millions, but locals remembered the game so fondly, they'd fooled themselves into thinking they actually saw the game in person. It's similar to the phenomenon of the number of people claiming to have been on hand for Woodstock in 1969 -- more people believe it than could have possibly shown up.

You also shouldn't use polls to ask people whether they did things that turned out to be a very, very bad idea:

Again from Political Animal: [in a recent poll] only 33 percent of respondents admit to having voted for the guy twice, while 52 percent said they'd never voted for him at all. If that were actually true, of course, Bush would never have had the chance to run the country so firmly into the ground that people are now pretending they never liked him

... so what does that leave us that polling IS good for?

Sometimes its not cool to gather data

2009-01-08T12:14:00.003Z

There are some occasions where you need to keep your data gathering quiet!

From the excellent XKCD.

I also love this one. I wish I'd have thought of it!!

Excel 2007 is good for something, after all

2008-12-08T14:52:00.002Z

Ok, so there are actually a few cool features in there. Its still annoying to get used to, though!

One feature I've found pretty handy recently is the conditional formatting enhancements. They really let you visualise data quickly and easilly. Great stuff.

There are a wide range of new options that are just a button-press away:

Here's an example of what I've been using it for:

Much better than looking at a table of numbers, I'm sure you'll agree!

Interest rates racing down

2008-12-05T17:05:00.000Z

There was a nice little chart in the Wall Street Journal yesterday showing clearly how central banks are racing interest rates towards zero.

(From WSJ via Paul Kedrosky)

Florence Nightingale's eye for info clarity

2008-12-05T07:49:00.003Z

I just got around to reading Science News' article on how Forence Nightingale was a pioneer in using novel and innovative charts to present data. Apparently she went to great lengths to do so in order to convince Queen Victoria of the need for social change. She was worried that without the clear presentation of data in charts, Queen Victoria's eyes would glaze over as she scanned statistics and tables of data.

Her most famous chart is an enhancement to what we would call a pie chart. It shows the number of deaths each month by their cause.

Each month is a twelfth of a circle. Months with more deaths are longer, meaning the area of each month shows the number of deaths. You can see that during the first part of the war, the blue wedges (disease) aremuch bigger than the red ones (wounds) or black ones (other causes). After March 1855, when the Sanitary Commission arrived, the blue wedges start becoming dramatically smaller.

From Science News:

The conventional way of presenting this information would have been a bar graph, which William Playfair had created a few decades earlier. Nightingale may have preferred the coxcomb graphic to the bar graph because it places the same month in different years in the same position on the circle, allowing for easy comparison across seasons. It also makes for an arresting image. She said her coxcomb graph was designed “to affect thro’ the Eyes what we fail to convey to the public through their word-proof ears.”

Some argue that a bar graph would have made her point more dramatically, though. One of the peculiarities of Nightingale’s circular presentation is that the deaths are proportional to the area, not the radius. Since the area of a circle is pr2, the area is proportional to the square of the radius rather than to the radius itself. This difference tends to de-emphasize the contrast between the small areas and the large ones. (In an early version of this diagram, Nightingale didn’t catch this distinction and drew the graphic incorrectly, with the radii proportional to the deaths. She quickly corrected her mistake.)

It seems to have worked. The 830 page report she wrote lead to massive changes in hospitals and by the end of the century, Army mortality was lower than civilian mortality.

Spotting fraud in numbers

2008-11-30T19:57:00.006Z

I remember reading about Benford's law years ago with fascination and thought I'd share it. Such a fun use of maths in the real world. Here's one application:

Dr. Theodore P. Hill asks his mathematics students at the Georgia Institute of Technology to go home and either flip a coin 200 times and record the results, or merely pretend to flip a coin and fake 200 results. The following day he runs his eye over the homework data, and to the students' amazement, he easily fingers nearly all those who faked their tosses.

Smart, eh? Its all because people don't know enough about how numbers really work and so can't fake data convincingly. The first thing that people get wrong when faking data is assuming that each number 0-10 has an equal chance of being used. They do not. In the real world, numbers are much more likely to start with a '1' than any other digit.

From "The First-Digit Phenomenon" by T. P. Hill, American Scientist, July-August 1998

Benford's law can be used to predict the frequency of numbers. As you can see fro mthe above, it matches closely to real-world data sets. It predicts that '1' is the most likely first digit, then '2' less so and so on and so on.

When you see the analysis of fraudulent data sets, it really comes to life:

From "The First-Digit Phenomenon" by T. P. Hill, American Scientist, July-August 1998

From the same article:

Benford's law can be used to test for fraudulent or random-guess data in income tax returns and other financial reports. Here the first significant digits of true tax data taken by Mark Nigrini from the lines of 169,662 IRS model files follow Benford's law closely. Fraudulent data taken from a 1995 King’s County, New York, District Attorney's Office study of cash disbursement and payroll in business do not follow Benford's law. Likewise, data taken from the author's study of 743 freshmen's responses to a request to write down a six-digit number at random do not follow the law. Although these are very specific examples, in general, fraudulent or concocted data appear to have far fewer numbers starting with 1 and many more starting with 6 than do true data.

Back in 2005 I worked with a team of organisers who would report the number of doors that each of their teams had knocked on each day. I thought it would be fun to see how that data compared to Benford's law. Overall, you see that it doesn't look like people were being honest:

Buit its not all bad news. It looks like some were more honest than others:

Why keep data?

2008-11-22T15:02:00.002Z

Not the best case study in the world, but a fun one:

I was bored at work seven years ago today. I can tell that because I was just looking through old emails and seven years ago today I was emailing Seb with an 'audit of recent text messages'. Here is who I was text messaging seven years ago:

What fun to have that data and be able to look back - not at who I remember being in touch with - but who I was ACTUALLY in touch with. There are some names on there I had completely forgotten about! One good reason to gather data and keep it laying around, I guess. I'll probably dig it out again in another seven years and then I'll really appreciate it!

(If I hadn't swapped phones so often I'd have kept all text messages since then and I could do some trend analysis. But technology has sadly not made this possible, but I'm starting again now with my iPhone and this helpful site)

(As shown in a previous post, there is a neat tool to do this kind of analysis on your Gmail, but I'm not enough of a nerd to be able to get it to work. Rubbish.)

Common BNP names

2008-11-19T17:29:00.004Z

Since someone leaked the British National Party's membership list to the internet, I thought I'd take a look.

Here are some common first names:

And some common last names:

No surprises there then!

I couldn't find a Singh or Patel. Odd.

Misleading conclusions

2008-11-19T10:31:00.004Z

I'm currently working on advertising strategy and someone just forwarded me a document arguing that when a major brand 'goes dark' on advertising for a long period, it significantly damages the brand. But I drew the opposite conclusion from the document!

The article is by an organisation with -to put it politely- a vested interest in keeping the advertising dollars flowing and so there are a few misleading things in there. But I thought one misleading conclusion was particularly fun. Here it is:

Note that they chose to not visually display the 'no change' result, because it is by far the biggest segment and does not help their case! As a result their tag line is plain misleading: The most likely thing to happen when a brand goes dark is actually that there is no impact on the measures they show! That should be the headline message: "brand measures are unlikely to change when you go dark."

And the chance of going dark and having no impact or of an impact being favourable is 72%-76%! That is pretty good odds, I'd say. I'm willing to bet that we've done smarter analysis than the competition and so if we were to do that we'd get even better odds ...

How to avoid supermarket waste

2008-11-13T12:09:00.000Z

I saw another article today on how supermarkets waste too much fresh food. I thought I'd post a quick chart showing an easy way to reduce waste: buy more stuff!

Slow selling products lead to waste. They always will. So look for products that don't sell very well and buy more of them!

The following chart shows what waste might be expected for fresh food, at different sales rates. Slow selling fresh food (2 singles per week in the chart) might be expected to usually waste more than 150% of what you sell. So for every two you sell, you waste more than three! (And some products waste way more than this, if they are very short life, or if they must be ordered in large cases from suppliers for example.)

You'll see that this decreases massively as sales increase slightly. Go from sales of two to four singles per week and your expected waste falls below 100% of sales. Increase the weekly sales to eight and you're back at a reasonable level: expected waste of about 20% of sales.

So if you are worried about the ammount of food that supermarkets waste, simply look out for the slow selling products and put a couple in your basket. a couple of extra sales will make all the difference to supermarket waste.

When you are lost, any map will do

2008-11-08T12:56:00.006Z

I was playing around with a really cool and easy to use analysis tool for Excel, Analyse It. And I thought I'd show how easy it is by running an old data set through it. So here is some quick analysis of an old political campaign I worked on. Seriously, this took about 30 minutes total.

We won the race overwhelmingly. In fact we won every precinct in the city. One thing the tool allows you to do is compare groups of data. Here is our vote share by city ward, with each blob representing a precinct:

This type of chart can be created with about three button presses. It shows we won big in our home ward (yay!). It also shows the distribution of precincts. They were pretty bunched together, but it might be interesting to look at the outliers. Why were they so different?

We did an ok job of predicting where turnout was going to come from. I can tell this by running a regression of our predicted turnout by precinct and the actual turnout by precinct. The tool does this very simply, with no specialist knowledge required:

This graph also takes about three button presses to generate, and comes with a neat set of stats telling us that we did an ok (not great) job of predicting turnout by precinct. But then we should have done, since historical turnout by precinct was widely available, and the best predictor of future behaviour is past behaviour.

Now here is the interesting bit: we were crap at predicting how people would actually vote. I should stress that this was a Democrat versus Democrat election, so it is a notoriously difficult thing to call. But either way, we didn't do a good job! Here is our categorisation of precincts in to how we thought they would vote in advance versus how they actually voted:

You have to say that's pretty bad. In the precincts we called for our opponent, we actually did better than those that we thought were too close to call. Its good to know that we did better in the precincts we thought would support our candidate. But in those precincts there is a really wide range of support. They could probably have been better broken out in to different categories.

We did try to break them out further, but without much luck. Here is a more detailed look at what vote we thought we would get versus what we actually got:

We did a good job of predicting the very best precincts, but beyond that group, all of the other five groups we decided on behaved very similarly.

We also broke the precincts out that we thought our opponent would do well in. That also didn't turn out to be a good predictor of the actual vote:

Pretty depressing really!

Data and targeting has played a huge part in many political campaigns. That is particularly clear from '08 and organisations like Catalist have played a huge part in this. In politics I had the pleasure of working with a fantastic group of people who would honestly evaluate their own work, but ...

... usually after elections finish, candidates' political campaigns close their doors and everyone goes off to the next campaign, tired and often ready to forget about the campaign. There is no will to evaluate what worked and no money to fund evaluation. And besides, the people who did the analysis often don't want to evaluate for fear that their analysis that they were paid handsomely for wasn't actually helpful after all. This is a tragedy.

Even if predictions aren't that helpful, having lots of data on a campaign makes people believe. Everyone has confidence in a well presented graph and they feel that those extra phone calls and those extra door knocks are being well targeted. In reality, I often believe that when you are lost, any old map will do.

However if there were more evaluation after the effect, perhaps campaign analysis would improve quicker and it would more often actually help the campaign to win, rather than just boosting morale.

Knowing where you were

2008-09-05T11:15:00.005+01:00

I'm loving my iPhone. Today I'm loving it mainly because it adds another dimension to the data that I can gather: where things take place.

For example, I use InstaMapper to track where I am in the world. Here is my journey home from work:

And here is a *live* map of the most recent time I ran the program:

&amp;amp;amp;amp;amp;amp;lt;br&amp;amp;amp;amp;amp;amp;gt;

Another great feature of the GPS is that it automatically geocodes photos. For example, I took regular photos here to track a recent trip to LA. Here a section from the drive to Heathrow through London:

[I can't seem to embed the actual map, which allows you to click on a dot to see the photo. But it is live here]

Such fun to have that data and be able to look back on where you were at a particular time, and what you saw while you were there. I love it.

Analytics? You can't touch this!

2008-08-20T23:58:00.001+01:00

Even MC Hammer thinks its cool.

(Courtesy of Juice Analytics)

The difficulty and danger of tipping

2008-08-01T11:10:00.001+01:00

This blog is about how you can analyse information in the world to bring clarity and make better decisions by doing so. This post is about some thoughts on how difficult that sometimes can be!

We all like to be nice and to reward people who are nice to us. So why is working out the right tip so difficult?

Ideally you should be able to use tips to signal both good and bad service. I like the standard and well understood assumption in New York that 15-20% tip is expected on restaurant meals. I tip 0% or 5% to signal truly terrible service. 15-20% to signal service that was about what I expected and 25% or so to signal that they exceeded my expectations. I have a nice system that I'm happy signals what I want it to ...

... however, even in this simple case, I very much doubt that most waiters take my tip and really know what was meant by the signal! For example:

When I tip 5%, am I bring deliberately unkind? Do I even know that 15-20% is expected? Am I just being 'cheap'?
When I tip 25% am I signalling good service or am I just being flash with my cash? Did I think that was a normal tip?

So even in places we are familiar with, I believe that signalling what you want to signal with a tip is usually difficult. When considering how much to tip, I think you need to balance the following pieces of information at the very least.

There is a lot of information there that you need to make a good decision!!!

First, you need to base your tip on the tip that the person is expecting from you. This varies by industry (restaurant workers yes, shop workers no. Why?), type of establishment (don't tip in a McDonalds restaurant, do tip in somewhere with white table cloths. Why? Do tip in hotels, not in B&Bs. Why?), location (by city and by country. Why?) amongst other factors.
If they are expecting 15% then you need to go to 20% or 25% to show appreciation. Is it ok to go below what they are expecting to signal bad service?
Is the tip already included in the bill? We have all made the mistake of adding a tip on only to realise on the way home that it was already in the bill and we ended up tipping twice!
Why tip as a percent of the bill? Does a waiter in a high class restaurant need four or five times the dollar value of tips that a waiter in a more downmarket place gets?
What wage is the person paid? Can they live on their salary, or do they genuinely need the tips to make a decent wage? If so, the tip is not really a tip. Its really part of the bill. If its really part of the bill, then the service would have to be pretty bad to justify you lowering the tip from their expectation. (You might argue that doing so would be akin to challenging the price of the actual goods / service you received.)

When away from places and industries that we know, it is very hard to find out what is expected as a tip. And as such, it is very hard to deliberately signal and to avoid accidentally signalling! Ask any two people and they will give you different answers. Some examples:

London taxis are, perhaps, the most expensive in the world, but offer perhaps the best trained drivers in the world. So what I consider reasonable for a tip probably doesn't mean much to the driver! I have lived in London for eight years and I'm not quite sure what the cab drivers actually expect by way of a tip!
Do you tip maids at hotels? I spend a lot of time in hotels with work and I can't claim back such tips so I don't leave them. Am I the only person in the hotel to not tip? I honestly don't know. One hotel I stay in a lot has a more local feel. I know the staff and so I feel bad not tipping. But I keep forgetting, so every now and then I leave a substantial tip on the bed with a thank you note. Hopefully this makes up for the times when I don't tip? [Someone recently told me that they tip maids in hotels as an incentive not to steal their stuff or clean the toilet with their toothbrush. That is another concept not included in my diagram: tipping as 'protection money' to avoid being treated badly!]
Do you tip the guy who takes your bag to your room in a chain hotel? How much? I have no idea. I try to carry my own bags to avoid the uncomfortable moment when they pause to see if I will tip and when I fumble through my wad of local currency I got from the airport ATM on the way and desperately try to calculate the exchange rate. Sometimes they insist on showing you to your room and its rude to not accept! Do you tip then?

In less developed parts of the world, it is even more difficult to know what and who to tip. And our inclination to be generous can easily be dangerous.

In places you are not familiar with, I think the following is the minimum required to calculate an appropriate tip:

For a start you are probably guided only by your guide book's view of what is 'expected'. Ask any two locals and you will get a different answer!

There is the dangerous inclination to be more generous in local terms than you otherwise might be. "Why not just tip five dollars to the boy that helps carry your bags? Its nothing to me!" It is dangerous because you may well be giving that boy the equivalent of a week's wage for a local factory worker. In situations like this you create huge incentives for people to get close to tourists to get tips, rather than to learn skills and get more traditional jobs. Just think what this does to the local economy!

But in such a situation you face the choice of looking stingy to the boy who is holding his hand out and to the people who are watching you decide what to give him, or being seen to be a big spending generous person. So its tough to do the right thing and give a tip that is reasonable in local terms rather than one that is reasonable in your terms.

This also creates a whole industry of people whose very purpose is to do some small act and then guilt foreigners in to giving them a tip for it. In Marrakesh you will see the boys on the street who are seemingly stood around to help tourists lost in the maze of Souks. Show even the smallest glint in your eye of being lost and one will pounce. Even if you tell him you don't need help and don't tell him where you want to go, he will walk in front of you to your destination and when you get there, he will very vocally demand a tip of a few dollars for taking you there. They are a nuisance created by the lure of what we see as small tips. Who can refuse a few dollars of tip to a kid who has his hand out and is beginning to be an embarrassment to you by insisting that you owe him a tip for showing you the way?

What do you think? Am I over-analysing? Is there a simpler way to ensure you tip the right amount without all the worry, above?

Visualising the wedding data

2008-07-20T21:49:00.000+01:00

Another quick post on the wedding. This time to show one of the benefits of keeping your data in one neat spreadsheet: it allows you to easily play with it!

Here is a simple treemap visualisation of the data in IBM's ManyEyes that you can play with:

Here is a breakdown of the guests by their relationship with us:

And here is a quick look at where people travelled from:

Nerdy, but fun!

I just got married ... thanks Google!!!

2008-06-28T13:40:00.000+01:00

Thanks to the the fact that Blogger allows you to write a post and have it posted at some future date and time, this should magically appear on the interweb just as I walk down the aisle with my beautiful fiancee.

I wanted to write a little post about all the wonderful ways that Google had helped make the planning for this event possible. By helping us to share information with each other and with our guests, the whole things was a very smooth process.

First, there is Google Docs. We stored EVERYTHING to do with the wedding on here. Draft documents, to do lists and most importantly the list of invites, RSVPs and such like. That way my fiancee, myself, our parents and others who needed to know could always have the latest information to hand and could make changes as and when they needed.

This also allowed us to have one master list of guests. One version of the truth for everything from who was coming, how names were spelt, where people were staying, addresses and on and on. This saved an amazing amount of time!

Google Page Creator was the next great thing we used. We could share the news with friends and family and update them with information about the wedding as it became available.

Google Webmaster Tools helped us to keep the site private.

Google Analytics helped us to see that people were actually using the site.

On our website, we embedded personalised Google Maps for a couple of reasons. Partly to show guests where the wedding was and where the key things were in the area: hotels, bars, sights.

... and partly because we wanted guests who were travelling from the UK to the wedding to get a feel for New York and Boston so they could easily turn their trip into a holiday. We have a shared Google Map that either of us can add things to with a couple of clicks of the mouse and they instantly feed to the site and are available for guests. We gave a couple of friends access to the map, so they could add places, which automatically appeared on our site. Just remembered a great restaurant to eat at? No problem, it's there in a couple of clicks!

Here is our New York map.

View Larger Map

One other fun Google tool we used on the site was the forms you can now add to Google Docs. We had a page where we gathered guests' ideas for songs that mean something to them. The results were gathered automatically in to a neat Google Spreadsheet and we picked the best ones and added them to the playlist for the evening.

Gmail helped because of the large amount of free storage and how easily you can search it. Can't remember what Aunty Valerie wanted to eat for the rehearsal dinner? No problems! Did Dylan ever send me his address? No problems! A quick search will answer the question for us. (She wanted lobster and he didn't send his address!)

Google Calendar helped Bergin and I to coordinate our schedules. She can see my calendar and I can see hers. So when it came to picking a wedding date, there was no need to compare diaries, they were both right there in front of both of us!

Oh, and we'll of course be sharing photos from our excellent photographer and from our guests on Picassa.

So thanks Google. Your mission to make information accessible and to help to share information helped set us up for our special day!

Hidden in words

2008-06-14T12:29:00.005+01:00

Quick post to show off a cool web application I found the other day, Wordle. You feed it text and it gives you a beautiful word cloud, helping you to see common words.

I tried it with an export of the last 999 status updates my friends had posted on Facebook. Here is the result:

(If you go to the Wardle version, you can play with fonts, layout etc. Its great fun!)

Fun, eh? I like that its mostly about fun things: 'thinking' and 'wondering' and 'weekend'.

Moving information around the web

2008-06-12T23:14:00.000+01:00

One thing that's great about the electronic world is that you can have all sorts of information sources linked together. As soon as you update the master, the other places automatically get the update. There are a million more useful applications of this concept, but a fun application I played with this weekend was my Facebook Status.

For no reason other that 'because I could' I decided to try to get my Facebook status synchronised with my Twitter account and my Google Chat status. Why have three different status messages (at least two of which must therefore be 'out of date') when you can have just one?

Through a lot of Googling Here's what I came up with:

This blog isn't about tech stuff, so I won't go in to how to do it here (except to say this pipe and this blog will give you what you need).

While I was doing this, it gave me another idea for a way to use similar technology. I used FindMe (from the guys at Electric Pocket) on my BlackBerry, which made it recognise when I'm in certain locations and posts them to my Facebook profile. I thought this was kinda fun. 'David is at work', 'David is at home', that kind of thing.

I thought it would be fun to have that information automatically store in my calendar. That way I could look back and see what time I got in to work, what time I got home, etc. Here's what I came up with, using the excellent similar LocaTags BlackBerry application and a Yahoo! Pipe:
It got me interested because its free information. To get it I don't need to keep a diary. I don't need to keep timesheets in the office. I don't need to do anything other than one off effort of setting up this arrangement (which was fun!).

And one of my mottos is that when there is free data you can gather, never turn it down. You just don't know when it might be useful ...

The importance of a good quality notebook

2008-06-03T18:14:00.003+01:00

I was excited to get a new notebook yesterday. The brand new Moleskine Volant Pocket in fact. Easy to slip in your pocket, but really well built and with fantastic paper quality. I in fact spent much of the evening in the pub writing down thoughts and ideas to celebrate finding it.
Then today I was wondering what it was that I loved about it? Then a chart came in to my head:

I can group all my previous notebooks in to three types.

Bad quality notebooks don't work for me. I tried the cheap spiral ones, but I scrawl anything and everything down. I end up with a mess that is ugly to read and contains things so mundane that even I would never really want to look back at it anyway. Junk!

I have also tried great quality books: old-skool hard-cover Moleskins or books from Smythson. But I feel like I can't write something in there unless it is uber-important. Only the highest quality thoughts. Since I don't sit arouond coming up with little gems of wisdom to capture in my notebook, they so rarely get used that they aren't that helpful.

And that's exactly why I love my new book. Not flash, not cheap, and I feel its going to contain just the right mix of good quality and good quantity of notes :)

Towards some clarity on airline loyalty

2008-05-27T20:11:00.006+01:00

Air Miles seem like a great deal. But I'm not sure they really are. The problem is that you have to do a bit of research and some maths to really work out whether they are a good deal or not. Not an example of clear information!

With a corporate discount, flying Upper Class London to LA with Virgin gets me 7,836 "Flying Co Miles" for £3,869 (£0.49 per Air Mile). You can't actually get a flight for the miles without paying taxes on that flight. And on some routes, taxes are a large part of the fare, so you don't actually get close to getting a free flight when you spend them - just a discount.

Here are the routes I tried:

I was looking for a trip to Havana, where my miles that cost £0.49 each were worth just £0.0088 each. Just under a penny. To save £440 on my flight, I was spending 50,000 miles, which cost my company £24,687. That's 1.8% of their money back. Not a great deal, considering the ticket cost!

If I spent the miles on the first New York flight I found, the miles that cost £0.49 each would be worth only £0.0054 each (just over half a penny). To save £195 on the fare I looked at, I would spend 36,000 miles which cost my company £17,775 to earn. That's 1.1% of their money back.

Sometimes its good to BRAG

2008-05-27T00:54:00.005+01:00

(follow on to my earlier post on root causes)

Ok. So you have a thing. A thing that is good or bad or indifferent, but that you want to keep an eye on. You want to keep it good, or explain why it is bad, for example. But its complicated -there are a lot of other things that make it the way it is. So you have also put together a root cause tree. So you know all the things that are contributing to making it the way it is. What next?

BRAG it. That is Blue, Red, Amber, Green it. Add a splash of colour and, as clear as day, show what is great (blue), good (green), poor (amber) or down right crappy (red).

This has massive advantages over the other method that springs to mind: scoring. Tempting as it is - as an analyst - to add numbers and decimal places, sometimes they really aren't needed. Sometimes all you need is a really, really simple message. Sometimes BRAG will do. Why confuse things with the complexity of numbers? Why make someone read a number and ask them self whether than number is good or bad? Does adding numbers add value or just add complexity?

Here's my root cause tree for how happy I am, with BRAG added.

The lesson there for me is that I'm doing pretty well (green overall!) despite neglecting some things I really care about. Must get on that bike (red), do something to make a difference (red), save some cash (red) and spend a little more time with friends (amber)!

I also looked back a few years to a time when I wasn't happy at work to see if my 'happy at work' root cause tree worked. Sure enough. Despite having a green or two in there, there were a lot of ambers and reds. It pretty much sums up that feeling I had each morning on the way to work.

But when I'm happy at work, does it work? I tried BRAGging the same diagram as I would have done after a few months at my current job. It works! It pretty much explains why I was jumping out of bed each morning and racing to the office. My job had the whole package!

So for me, root cause seems to do a pretty good job. Now the challenge is to use root causes and BRAGs to make better decisions, rather than to use up time analysing past ones. After all, if we analysts can't use information to make better decisions, we are just a bunch of people who sit in dark rooms having fun with our spreadsheets.

Making sense of big numbers

2008-05-26T14:17:00.004+01:00

As the New York Times says so clearly: "The human mind isn’t very well equipped to make sense of big numbers. We don’t deal with a trillion of anything in our daily lives, and so when we come across such a big number, it is hard to distinguish it from any other big number. Millions, billions, a trillion — they all start to sound the same."

So when trying to communicate the annual cost of the Iraq war, the New York Times article compared it to things we could understand, like healthcare, schooling and cancer research. After all: if you don't spend the money on war, you can certainly spend it somewhere else: "The way to come to grips with $1.2 trillion is to forget about the number itself and think instead about what you could buy with the money. When you do that, a trillion stops sounding anything like millions or billions."

This is just great. Suddenly the big numbers mean something. The only other place I deal with big numbers is at work though. So perhaps next time I have to communicate the benefits of a project at work, I'll compare it to the things that money could otherwise be spent on: like shorter lines at the checkout (more checkout staff), cheaper veg and such like.

I remember seeing this when The Times first published it in early 2007 and loving it. Glad I finally got around to posting it on the blog!

Understanding the root causes

2008-05-26T12:54:00.013+01:00

When you are not happy about something there are probably a dozen or more potential causes. Its usually easy to jump to an idea which of these to blame. But do we really know it is the main cause? And have we even thought about the other causes to see how much each of them is to blame?

I had this problem in a big way at work. A project wasn't working. Something big. All sorts of systems and processes and people weren't quite playing together nicely and the collective outcome was bad, bad, bad. Each team involved had an idea of what the problem was. Some blamed a system. Some blamed processes that weren't being followed. But the fascinating thing was than nobody involved could come up with a complete list of what all the root causes could be, and how much each was contributing to the poor performance. So we developed a guide (obscured because its confidential - sorry!):

This shows the three main measures of 'performance': What are the things we can measure to know whether performance is 'good' or 'bad'? It then lists the seven things that can cause performance to be bad. It has to be one of these and all should be investigated to understand poor performance. Finally it branches out in to a tree: If one of the seven things looks like its a cause, it shows you what to check next.

This was used to great effect recently by one of our projects. Things were looking bad. The overall results were terrible. I visited the country in question and they had used the principles from the root cause tree to really understand it. They could tell me straight away that although overall performance was poor, 60% was due to XXX, 20% due to YYY, 10% was ZZZ and the remainder was a bunch of smaller issues that weren't worth investigating. Amazing insight. We were then able to confidently and quickly focus our efforts on the real root cause.

Its not just boring work projects that would benefit from explanation in this way. I thought about what it is that makes me happy and came up with this:

... which immediately made me realise that I had got the balance wrong in daily life!!

I also thought about what makes me happy at work, and came up with this:
A little simpler than the life version, and a lot simpler than the one I did for work. But it gives me something to measure my job against and to judge other jobs by.

Finally I was looking back with frustration at political campaigns I have been involved in. The bits I was interested in is the gathering of data and the use of data to help increase the campaign's effectiveness. I sketched out a diagram with the overall thing I sought to achieve in campaigns at the top (Improved effectiveness and clear reporting on progress) and all of the things that were needed to achieve this. No campaign I have worked on had these elements. But I have also never seen a diagram with all of the elements on it before. So its not really a surprise that it was never achieved if nobody had ever clearly set out what needs to be achieved!

So all in all I have fallen in love with drawing root cause trees as a way to understand complex problems and thought I'd write a little blog post to share it. (Even though there are no numbers or charts involved.)

I guess the moral of this story is, if there is a complex problem out there with lots of things contributing to it, try listing them all and joining them with little arrows. It just may help to bring some clarity.