Data quality: Garbage in, garbage out?

Email LinkedIn
Tools


We're seeing quite a bit written these days about the need to move forward on IT initiatives and decisions with great speed. When it comes to data-related projects, increasingly the advice seems to be to get moving regardless of the quality of the data. (Send me an email and I'll send you back some links.)

A data set can never be perfect, we're told, so why sweat it? Data analysis can be useful even if the input isn't exactly accurate, so don't get hung up on facts.

Who's to say what's factual anyhow? Everyone in an enterprise seems to have his or her own version of the truth, so just go ahead and pick one and the pieces will fall into place.

To me, this sounds like a good approach for people planning to be around for the short term. It certainly saves time, money and effort, and anyone who can cut back on those resources and still affect change is likely to win some accolades. It is a beneficial approach for anyone selling data analytics technology and services as well.

But for IT executives who plan to stay in the game for the long haul, I'm not so sure that playing fast and loose with the data is the ticket. The movement toward data-driven business cultures rests on the understanding that data--unlike hunches, distorted perceptions or egos--has immovable integrity and trustworthiness. Decisions based on hard data, therefore, are more likely to withstand the test of time.

Not worrying about data quality may make IT look efficient now, but how will it look when business decisions based on sloppy data prove wrong? Even small inaccuracies can lead to large errors. What happens when someone demonstrates that the data was no more accurate than a hunch might have been?

There is an ever-growing body of strategies and tools available to help organizations clean their information, weed out false versions of the "truth," and develop more complete, accurate and valid data sets. Maybe before racing to put a company's data to work, it is worth the time and effort it takes to check them out. Maybe integrity--of data and of decisions--should still be a business goal. - Caron