I have been doing a bit of work with data visualization recently, using Tableau. It got me thinking about the way we use data to produce information, and how that is changing.
One of my early career challenges was to analyse what effect promotions had on overall product sales. In Unilever at that time the standard practice was to run product promotions with the supermarkets every few months. The idea was to gain more prominent shelf space, and so increase sales. The promotion had to offer something extra (money off, extra free, two for one) and manufacturing had to be geared up to support the extra volume. The annual financial plan had to be modelled on the anticipated peaks and troughs of volume. In fact you could say that the whole operation was geared around these promotions.
But the question was: did we actually increase overall profitability; or did we displace volume from one cycle to another? My job was to look at the evidence to see what we could conclude about the effectiveness of promotion on profit.
The trouble is, I had no tools. I could get data about production, physical sales to the supermarket and market share by getting reports from the "mainframe", but I had no tools to analyse them. I had to draw graphs by hand. I plotted sales volume against market share and drew these up on paper and on acetates (remember those?). The results were presented to the Board, and I was asked to go and discuss them. I could make only the vaguest conclusions: promotions did not seem to increase market share in any sustained way; sales volume seemed to fall after a promotion by as much as it had increased; average price sold and profitability went down as much as sales volume went up.
At that time there were no computers on desks. Now the purpose of the desk is to hold the computer. Today I would be able to draw nice graphs, with bubbles expanding and floating upwards. But would it make any difference? No, because there was no useful data to make the correlation between the promotion and the effect on consumer behaviour. The real difference between then and now is not the computer. It is the data.
One of my pet peeves is the phrase "the pace of change is increasing". No, it is not. The pace of change is a constant. If it were increasing, it would either have to change direction and start slowing down at some point, or it would have to increase ad infinitum, which would be an absurdity. The phrase is a rhetorical device to encourage action. But you have to consider that if your call to action is a logical absurdity then there is something wrong.
OK, so what is changing, because something is? It is the availability of data about the world and our actions in it. The steadily lower cost of technology is making more and more data available, and giving us better tools to turn the data into usable information. We have more information, so we can act with more knowledge. We can use the data to gain a new insight into the behaviour of the world. It may be what we guessed intuitively, without data, or it may be new. So instead of "the pace of change is increasing" we have "the availability of data and information and knowledge is constantly increasing". We can respond in two ways:
- Collect more data. This is what the Internet of Things is about.
- Use the data more effectively. This is what Data Visualization is about.