In the previous checklist, I talked about setting up for success by planning key aspects before any data is mapped, any vendors are evaluated. The next step is to consider data readiness.
During the implementation of my first analytic application, it was only near the end of the project that I realized that we weren’t showing data in a single, common currency. We had euros mixed with rubles combined with the currency we were supposed to be using — U.S. dollars. The lesson? Take data preparation seriously.
Here’s my checklist for data readiness:
Do you know what kind of data you’ll be using for your analytics?
Unfortunately, data comes in all different flavors. Some is highly structured with a format that’s easily understood (like an order record) while other data is a bit messier (like support call notes). Data may even take the form of audio or video information. The type of data you need for your analytics needs to be considered early so that you can decide how to approach the situation.
Have you defined a plan for data cleansing/preparation?
A first step is to have a plan for data cleansing and preparation. Don’t assume that data, even if coming from enterprise systems under management of IT, is ready for analytic use. You will almost always need to perform some level of prep. And, you’ll need a plan.
Have you performed “buy vs. build” analysis for data preparation needs?
These days there are many options for making data ready for analytic use. Some teams choose the manual route of preparing data while others make use of the tools on the market. Some analytic platforms include robust data preparation capabilities while others require the use of a dedicated data management system. Look over the options and determine which is best for your situation, taking care to remember that this isn’t a “one and done” situation. It’s a long-term proposition.
Have your evaluated data preparation vendors/platforms?
The tools on the market aren’t all the same. While some are designed for data preparation experts, others are a bit more business-user friendly. Make sure the vendors you’re considering match the users that will have to perform the data operations. Also, when performing a data preparation vendor evaluation, be sure to consider the following:
- Have you evaluated the cost of the data preparation solution under high/medium/low user growth scenarios?
- Have you evaluated the initial (year 1) and longer-term (years 2-5) cost of buying a data preparation solution?
- Have you selected a vendor (if not building in-house)?
- Does the vendor make professional services available?
- Will you be using vendor professional services for implementation?
- Will the vendor train your team or will you need to rely on services in the future?
- Have you evaluated the support model offered by the data preparation vendor?
Have you determined where you’ll store your data?
Although there are analytic platforms that let you directly connect to transactional and other “source” data, many teams prefer to consolidate data into a data warehouse. Will you be connecting directly to your data? Will you be using a data warehouse? Do you have a combination of these options?
Have you determined how data will be transported (if at all) to storage?
How will data get from the source systems to the data warehouse or other storage? It seems like a trivial consideration, but the processes to move data require careful planning and shouldn’t be overlooked.
- Do you need/have a way to track the transformed data back to its source?
In many cases, it’s critical that you maintain “data fidelity” to the original source. That is, after transforming data and making it ready for analytics, can you still trace the data back to its origin? This might not be important for some use cases but for situations like health care, pharmaceuticals, and finance it’s mission-critical. Make sure you’ve considered your requirements.
- Have you determined who will be preparing data — both during implementation and when changes are required?
As you can see from the other checklist questions, data preparation is a non-trivial activity. Who will be performing this vital role for your analytic project? A consultant? Your team? Who on your team? Make sure that you identify the responsible party before you find that you’re under pressure to make a change.
- Have you set a schedule for auditing data cleanliness?
Cleansed data unfortunately doesn’t stay clean forever. Over time, you’ll need to review data sources and transformations to ensure that things are still as intended. Make sure you’ve got a process in place to handle this essential activity.
(You can download a PDF of this checklist here.)
That’s Checklist #2: Data Readiness. It’s not as glamorous as building metrics and charts, but it’s a key step to making sure that what you’ll do later in the project is accurate.
In the next checklist in the series, we’ll cover data visualization planning.