{"id":404,"date":"2018-06-27T12:53:38","date_gmt":"2018-06-27T19:53:38","guid":{"rendered":"http:\/\/www.lightsondata.com\/?p=404"},"modified":"2020-09-06T22:05:54","modified_gmt":"2020-09-07T05:05:54","slug":"estimating-cost-poor-data-quality","status":"publish","type":"post","link":"https:\/\/www.lightsondata.com\/estimating-cost-poor-data-quality\/","title":{"rendered":"Estimating the cost of poor data quality in 5 steps"},"content":{"rendered":"

According to the IBM Big Data & Analytics Hub<\/a>, poor data costs the US economy $3.1 trillion every year. Ovum Research reported<\/a> that poor data can cost businesses at least 30% of revenues. These numbers are staggering and in my opinion, they should be enough to motivate any organization to start investing in data governance, but we know that’s not happening for everyone.<\/p>\n

What is the cost of poor data for your organization?<\/h2>\n

As you might recall from the best data quality management trifecta<\/a>, analyzing the business impact and cost of bad data is one of the first steps one should undertake before cleansing it and implementing data quality controls. Analyzing this cost can be a daunting undertake and it usually involves both qualitative and quantitative methods, but here are some simple steps to get you started. For the sake of simplicity, we assume the default time frame is a year, and that your organization only manages one data system.<\/p>\n

\"estimating<\/p>\n

Step 1: Number of added and updated data<\/h3>\n

Estimate the number of new data entries your organization enters into your system as well as the number of data updates being made. This can usually be taken from audit logs or comparisons of data snapshots taken at different times.<\/p>\n

Step 2:\u00a0 The error rate<\/h3>\n

Note the percentage of the data entries that contain errors. This can be estimate either from data quality logs<\/a>, data profiling, or interviews and survey responses which were conducted during the situational analysis phase.<\/p>\n

Step 3: Cost of poor data entry<\/h3>\n

This cost usually considers the allotted time to cleanse the data, the cost per hour of the position tasked with cleansing it and\/or any licensing and servicing fees for 3rd party tools in place to cleanse this data. 3 categories should be considered in estimating the cost of poor data entry, each with its own criteria:<\/p>\n

    \n
  1. if caught on entry – the default cost range is is $0.5-$1<\/li>\n
  2. if cleansed once it’s in the system – the default cost range is $1-$10<\/li>\n
  3. if left unchecked – the default cost range is $10-$100. This can be quite high as it takes into account reputation erosion, risk of regulatory noncompliance, lost opportunity, misinformation to decision makers,\u00a0 etc.<\/li>\n<\/ol>\n
    \n
    <\/span><\/span><\/div>\n
    <\/div>\n

    Step 4: Percentage of detected poor data<\/h3>\n

    For the 3 above categories, what is the percentage of data errors caught and corrected at each step? The three values should add up to 100%<\/p>\n

    Ex: caught on entry: 20%, cleansed in the system: 50%, left unchecked: 30%<\/p>\n

    Step 5: Get the total cost<\/h3>\n

    Now it’s time to put it all together. Here is an example of the total cost calculations:<\/p>\n

    \"cost<\/p>\n

    <\/h4><\/div>\n <\/div><\/h4>\n

    Conclusion<\/h3>\n

    For more advanced\u00a0 calculations you can also include the rate of data decay. For customers records there is a natural data quality decay occurring even for those cleansed data points due to death, change of address, change of name, contact preferences, etc. As a rule of thumb, this is at least 10% of your entire customer database, and you can assume the same cost as for those records left unchecked.<\/p>\n

    Have you done any preliminary poor data quality costs? What other criteria did you consider?<\/p>\n","protected":false},"excerpt":{"rendered":"

    According to the IBM Big Data & Analytics Hub, poor data costs the US economy $3.1 trillion every year. Ovum Research reported that poor data can cost businesses at least 30% of revenues. These numbers are staggering and in my opinion, they should be enough to motivate any organization to start investing in data governance, […]<\/p>\n","protected":false},"author":1,"featured_media":407,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[4],"tags":[66,8],"class_list":["post-404","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-data-quality","tag-data-quality-cost","tag-template","post-wrapper","thrv_wrapper"],"yoast_head":"\nEstimating the cost of poor data quality in 5 steps | LightsOnData<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.lightsondata.com\/estimating-cost-poor-data-quality\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Estimating the cost of poor data quality in 5 steps | LightsOnData\" \/>\n<meta property=\"og:description\" content=\"According to the IBM Big Data & Analytics Hub, poor data costs the US economy $3.1 trillion every year. Ovum Research reported that poor data can cost businesses at least 30% of revenues. These numbers are staggering and in my opinion, they should be enough to motivate any organization to start investing in data governance, […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.lightsondata.com\/estimating-cost-poor-data-quality\/\" \/>\n<meta property=\"og:site_name\" content=\"LightsOnData\" \/>\n<meta property=\"article:published_time\" content=\"2018-06-27T19:53:38+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2020-09-07T05:05:54+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/i0.wp.com\/www.lightsondata.com\/wp-content\/uploads\/2018\/06\/estimating-cost-of-poor-data.png?fit=700%2C553&ssl=1\" \/>\n\t<meta property=\"og:image:width\" content=\"700\" \/>\n\t<meta property=\"og:image:height\" content=\"553\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"George Firican\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@georgefirican\" \/>\n<meta name=\"twitter:site\" content=\"@georgefirican\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"George Firican\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.lightsondata.com\/estimating-cost-poor-data-quality\/\",\"url\":\"https:\/\/www.lightsondata.com\/estimating-cost-poor-data-quality\/\",\"name\":\"Estimating the cost of poor data quality in 5 steps | LightsOnData\",\"isPartOf\":{\"@id\":\"https:\/\/www.lightsondata.com\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.lightsondata.com\/estimating-cost-poor-data-quality\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.lightsondata.com\/estimating-cost-poor-data-quality\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/i0.wp.com\/www.lightsondata.com\/wp-content\/uploads\/2018\/06\/estimating-cost-of-poor-data.png?fit=700%2C553&ssl=1\",\"datePublished\":\"2018-06-27T19:53:38+00:00\",\"dateModified\":\"2020-09-07T05:05:54+00:00\",\"author\":{\"@id\":\"https:\/\/www.lightsondata.com\/#\/schema\/person\/a6c554e2c0ae016f437ab91c59d65622\"},\"breadcrumb\":{\"@id\":\"https:\/\/www.lightsondata.com\/estimating-cost-poor-data-quality\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.lightsondata.com\/estimating-cost-poor-data-quality\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.lightsondata.com\/estimating-cost-poor-data-quality\/#primaryimage\",\"url\":\"https:\/\/i0.wp.com\/www.lightsondata.com\/wp-content\/uploads\/2018\/06\/estimating-cost-of-poor-data.png?fit=700%2C553&ssl=1\",\"contentUrl\":\"https:\/\/i0.wp.com\/www.lightsondata.com\/wp-content\/uploads\/2018\/06\/estimating-cost-of-poor-data.png?fit=700%2C553&ssl=1\",\"width\":700,\"height\":553,\"caption\":\"estimating the cost of poor data quality\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.lightsondata.com\/estimating-cost-poor-data-quality\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.lightsondata.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Estimating the cost of poor data quality in 5 steps\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.lightsondata.com\/#website\",\"url\":\"https:\/\/www.lightsondata.com\/\",\"name\":\"LightsOnData\",\"description\":\"Practical resources, online courses, free articles and videos for data management, data governance, data quality, and business intelligence\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.lightsondata.com\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.lightsondata.com\/#\/schema\/person\/a6c554e2c0ae016f437ab91c59d65622\",\"name\":\"George Firican\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.lightsondata.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/3ded8f815b6acadce87408824c260bc65adf043dd0eb7accc6799e576d011254?s=96&d=retro&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/3ded8f815b6acadce87408824c260bc65adf043dd0eb7accc6799e576d011254?s=96&d=retro&r=g\",\"caption\":\"George Firican\"},\"description\":\"George Firican is the Director of Data Governance and Business Intelligence at the University of British Columbia, which is ranked among the top 20 public universities in the world. His passion for data led him towards award-winning program implementations in the data governance, data quality, and business intelligence fields. Due to his desire for continuous improvement and knowledge sharing, he founded LightsOnData, a website which offers free templates, definitions, best practices, articles and other useful resources to help with data governance and data management questions and challenges. He also has over twelve years of project management and business\/technical analysis experience in the higher education, fundraising, software and web development, and e-commerce industries.\",\"sameAs\":[\"https:\/\/www.lightsondata.com\",\"https:\/\/www.linkedin.com\/in\/georgefirican\/\",\"https:\/\/x.com\/georgefirican\"],\"url\":\"https:\/\/www.lightsondata.com\/author\/juni_83yahoo-com\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Estimating the cost of poor data quality in 5 steps | LightsOnData","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.lightsondata.com\/estimating-cost-poor-data-quality\/","og_locale":"en_US","og_type":"article","og_title":"Estimating the cost of poor data quality in 5 steps | LightsOnData","og_description":"According to the IBM Big Data & Analytics Hub, poor data costs the US economy $3.1 trillion every year. Ovum Research reported that poor data can cost businesses at least 30% of revenues. These numbers are staggering and in my opinion, they should be enough to motivate any organization to start investing in data governance, […]","og_url":"https:\/\/www.lightsondata.com\/estimating-cost-poor-data-quality\/","og_site_name":"LightsOnData","article_published_time":"2018-06-27T19:53:38+00:00","article_modified_time":"2020-09-07T05:05:54+00:00","og_image":[{"width":700,"height":553,"url":"https:\/\/i0.wp.com\/www.lightsondata.com\/wp-content\/uploads\/2018\/06\/estimating-cost-of-poor-data.png?fit=700%2C553&ssl=1","type":"image\/png"}],"author":"George Firican","twitter_card":"summary_large_image","twitter_creator":"@georgefirican","twitter_site":"@georgefirican","twitter_misc":{"Written by":"George Firican","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.lightsondata.com\/estimating-cost-poor-data-quality\/","url":"https:\/\/www.lightsondata.com\/estimating-cost-poor-data-quality\/","name":"Estimating the cost of poor data quality in 5 steps | LightsOnData","isPartOf":{"@id":"https:\/\/www.lightsondata.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.lightsondata.com\/estimating-cost-poor-data-quality\/#primaryimage"},"image":{"@id":"https:\/\/www.lightsondata.com\/estimating-cost-poor-data-quality\/#primaryimage"},"thumbnailUrl":"https:\/\/i0.wp.com\/www.lightsondata.com\/wp-content\/uploads\/2018\/06\/estimating-cost-of-poor-data.png?fit=700%2C553&ssl=1","datePublished":"2018-06-27T19:53:38+00:00","dateModified":"2020-09-07T05:05:54+00:00","author":{"@id":"https:\/\/www.lightsondata.com\/#\/schema\/person\/a6c554e2c0ae016f437ab91c59d65622"},"breadcrumb":{"@id":"https:\/\/www.lightsondata.com\/estimating-cost-poor-data-quality\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.lightsondata.com\/estimating-cost-poor-data-quality\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.lightsondata.com\/estimating-cost-poor-data-quality\/#primaryimage","url":"https:\/\/i0.wp.com\/www.lightsondata.com\/wp-content\/uploads\/2018\/06\/estimating-cost-of-poor-data.png?fit=700%2C553&ssl=1","contentUrl":"https:\/\/i0.wp.com\/www.lightsondata.com\/wp-content\/uploads\/2018\/06\/estimating-cost-of-poor-data.png?fit=700%2C553&ssl=1","width":700,"height":553,"caption":"estimating the cost of poor data quality"},{"@type":"BreadcrumbList","@id":"https:\/\/www.lightsondata.com\/estimating-cost-poor-data-quality\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.lightsondata.com\/"},{"@type":"ListItem","position":2,"name":"Estimating the cost of poor data quality in 5 steps"}]},{"@type":"WebSite","@id":"https:\/\/www.lightsondata.com\/#website","url":"https:\/\/www.lightsondata.com\/","name":"LightsOnData","description":"Practical resources, online courses, free articles and videos for data management, data governance, data quality, and business intelligence","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.lightsondata.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.lightsondata.com\/#\/schema\/person\/a6c554e2c0ae016f437ab91c59d65622","name":"George Firican","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.lightsondata.com\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/3ded8f815b6acadce87408824c260bc65adf043dd0eb7accc6799e576d011254?s=96&d=retro&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/3ded8f815b6acadce87408824c260bc65adf043dd0eb7accc6799e576d011254?s=96&d=retro&r=g","caption":"George Firican"},"description":"George Firican is the Director of Data Governance and Business Intelligence at the University of British Columbia, which is ranked among the top 20 public universities in the world. His passion for data led him towards award-winning program implementations in the data governance, data quality, and business intelligence fields. Due to his desire for continuous improvement and knowledge sharing, he founded LightsOnData, a website which offers free templates, definitions, best practices, articles and other useful resources to help with data governance and data management questions and challenges. He also has over twelve years of project management and business\/technical analysis experience in the higher education, fundraising, software and web development, and e-commerce industries.","sameAs":["https:\/\/www.lightsondata.com","https:\/\/www.linkedin.com\/in\/georgefirican\/","https:\/\/x.com\/georgefirican"],"url":"https:\/\/www.lightsondata.com\/author\/juni_83yahoo-com\/"}]}},"jetpack_publicize_connections":[],"jetpack_featured_media_url":"https:\/\/i0.wp.com\/www.lightsondata.com\/wp-content\/uploads\/2018\/06\/estimating-cost-of-poor-data.png?fit=700%2C553&ssl=1","jetpack_shortlink":"https:\/\/wp.me\/p9BPV6-6w","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/www.lightsondata.com\/wp-json\/wp\/v2\/posts\/404","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.lightsondata.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.lightsondata.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.lightsondata.com\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.lightsondata.com\/wp-json\/wp\/v2\/comments?post=404"}],"version-history":[{"count":11,"href":"https:\/\/www.lightsondata.com\/wp-json\/wp\/v2\/posts\/404\/revisions"}],"predecessor-version":[{"id":2274,"href":"https:\/\/www.lightsondata.com\/wp-json\/wp\/v2\/posts\/404\/revisions\/2274"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.lightsondata.com\/wp-json\/wp\/v2\/media\/407"}],"wp:attachment":[{"href":"https:\/\/www.lightsondata.com\/wp-json\/wp\/v2\/media?parent=404"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.lightsondata.com\/wp-json\/wp\/v2\/categories?post=404"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.lightsondata.com\/wp-json\/wp\/v2\/tags?post=404"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}