In my previous post on analytics and retrofitting measurability, I talked about the first two phases of getting to grips with metrics and analytics in an existing system. I divvied it up so because otherwise it would have been too long to read.
So by now, you’ve gotten to grips with churning out the best you can do with the data available. What’s next?
Retro-fitting measurability (continued):
PHASE 3 : Only answer the answerable questions (but make sure people know what the answerable questions are)
OK. you’ve done the gruntwork. Time to ‘chin high, puff chest’, as a certain Ron Smith* would say.
Formalise the business questions that the business folks are ALLOWED to ask. You should be able to. They’ve asked you enough times. You should have been learning and listening.
Now you get to come up with what is effectively an SLA for your fledgling metrics department (of one?). A decree: “Hereof, all ye who ask for metrics shall come to know only these things, for upon these other things we are unable to deliberate further until the system has been changed”. That sort of thing. There’s no point bashing yourself over the head to extrapolate, from shreds of data, insights that simply wouldn’t stand up to scrutiny. Business owners always want good news on fancy charts, but if there ain’t the data then there ain’t the data. Only commit to answering the answerable questions, and make it a (business) priority to launch the project which puts in place the framework for finding out other stuff.
This means that you require an official signoff on your list of answerable (allowed) analytics questions. Wait… tell the pointy-haired business folks what they can and cannot have? Oh, woe! FEAR AND TREPIDATION!! Wow… is it really that bad? OK… well in lieu of a proper signoff, at least do yourself a favour and de-scope any impossible inquiries with
make_polite_string("our system does not f**** DO THIS.");
It is crucial that you learn to do this, otherwise you will spend time chasing red-herrings that don’t go anywhere. That’s what phase 1 is for. By now you should know what the red herrings are, and you should avoid them like the plague. So that when you get a request like “Hey Jude… what’s our cost per average revenue cycle pricepoint per 1000 for doodad sales from the foo ads?”, open a spreadsheet, then type “incomputable” in cell A1. Go down to cell A2 and type: “But here’s a very similar metric, and besides it’s more useful.” Enter the useful data. Done. Send it off. You can do it!
PHASE 4 : Introduce infrastructural change. Piecemeal.
Now that you can make known what is knowable from your system, its time to tackle what is not yet knowable. You need to get in on the development or optimisation cycles for the system in question. How you do this will vary from organisation to organisation. But at some point, you need to be able to put through small requests for enhancements to infrastructure that:
- records what types of events (eg service delivered, goods received, “sales conversions” like purchases, joining or registering, telling a friend about the system, sharing or consuming some piece of media, bringing more people into an organisation… any action or event of import or value to the system)
- records when these events occur (date and time information)
- records where sales are made and at which point in the (system-dependent) life-cycle of a new interactor with the system
- records which trajectories, or sequences of events (pre-defined as being important), are taken by entrants / interactors with the system.
By introducing change incrementally, your solution stands a better chance of being a) adaptive and b) small – you want to capture the least bit of data that allows you to answer the big questions.
And you don’t need to build the world’s next biggest data-warehousing op. Why? Because the business/organisational decisions that matter rest on surprisingly few (but potent) questions.
Keep that in mind, and you won’t go wrong.
*Roots Manuva’s front man Ron Smith’s rap-esque poetry will get you through this. Go listen to him. He has more problems than you do.