Warning: count(): Parameter must be an array or an object that implements Countable in /home/patmurra/public_html/penfield.one/wp-content/themes/specular/includes/view/blog/loop-index.php on line 49

Building a Data Foundation for Analytics

We work with many companies that are just getting started on the pathway to analytics.  Their data is usually better than they think it is, though it frequently is far from usable in the state we find it.  Just this week a client said that they had the data we wanted, but that it would take 150 man-hours to get it together for us.  A little skeptical, we probed deeper to find that this client had between 12 and 19 individual data files per week, each representing a category of products.  We needed 156 weeks worth of data, which meant we were sent over 2,100 individual files.  We built a quick macro to extract the relevant data out of the files (which made us heroes in the eyes of the client), but it was still an unpleasant task.

So, if you’re looking ahead to building an analytical capability and wondering what you can do today to ensure success tomorrow, here are a few rules of thumb:

1) Keep it centrally
This doesn’t necessarily mean a large database or warehouse investment- often it can be as simple as a shared drive.  Too often we find a computer gets wiped, or someone quits, and historical data is lost.  Don’t worry about making it clean necessarily, but make sure it exists and isn’t tied to one individual.

2) Extract it from your agencies
Your agency usually had loads of data about what you’ve executed, when you did it, and where it ran.  But agencies are not partners for life.  Make sure that if the agency relationship goes South, you’ve got your data somewhere where it won’t be held hostage.

3) MarComms, or Marketing Calendars are great, but…
We’ve been sent plenty of colorful charts with merged cells and rainbows of colors that are supposed to contain all the marketing and media data that we could ever ask for.   But we ask for a lot, and rarely does a MarComm show the media weight, with geographic breakouts (if applicable), by week, with creative details.  So we use them for guidance and for validation, but we still will ask for actualized executions.

4) Buying competitive data is rarely a bad idea
Few of our clients regret buying competitive data, whether syndicated data on their marketing activity, web-scraped price and discount data, or market share data.


  • Jan 29 2011
Is anyone doing hosted data warehouses for marketing analytics? For example, why not build a web application where clients can store their data in the cloud? Building analytics and reporting tools on top of this might be a good way create a scale offering.
    • Jan 30 2011
    There are a few services that will collect, integrate, and validate marketing analytics data and host it for you on an ongoing basis. The most notable is MMA's MarketView product. But any such service, MMA's included, will always require a lot of human effort to both collect and validate the data, and the diagnostic models themselves are not able to be automated. There's still too much interpretation, balancing of insights vs. statistical indicators, and other activities for machines to do.