{"id":1257,"date":"2019-09-25T12:01:38","date_gmt":"2019-09-25T19:01:38","guid":{"rendered":"https:\/\/www.lightsondata.com\/?p=1257"},"modified":"2020-09-06T21:47:50","modified_gmt":"2020-09-07T04:47:50","slug":"3-ideas-data-warehouse-lifecycle-quality-process","status":"publish","type":"post","link":"https:\/\/www.lightsondata.com\/3-ideas-data-warehouse-lifecycle-quality-process\/","title":{"rendered":"3 new ideas on improving the data warehouse lifecycle quality process"},"content":{"rendered":"<p>Data warehousing for business intelligence and \u201cbig data initiatives\u201d continues to gain significance as organizations become more aware of the benefits of decision oriented data warehouses. However, a key issue, with the rapid development and implementation of data warehouses, is that data quality defects are often injected during the multiple lifecycle stages of the warehouse.<\/p>\n<p>A primary requirement is an efficient data warehouse (DW\/BI) system process that reliably extracts, transforms, cleanses, and loads data from source systems on a 24 by 7 basis without impacting overall performance, scalability or reliability.<\/p>\n<p>In this article, we present new ideas on a \u201cbeginning-to-end\u201d data warehouse lifecycle quality process.<\/p>\n<p><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" data-attachment-id=\"1259\" data-permalink=\"https:\/\/www.lightsondata.com\/3-ideas-data-warehouse-lifecycle-quality-process\/dw-lifecycle-quality-process\/\" data-orig-file=\"https:\/\/i0.wp.com\/www.lightsondata.com\/wp-content\/uploads\/2019\/09\/dw-lifecycle-quality-process.png?fit=558%2C167&amp;ssl=1\" data-orig-size=\"558,167\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"dw lifecycle quality process\" data-image-description=\"\" data-image-caption=\"\" data-medium-file=\"https:\/\/i0.wp.com\/www.lightsondata.com\/wp-content\/uploads\/2019\/09\/dw-lifecycle-quality-process.png?fit=300%2C90&amp;ssl=1\" data-large-file=\"https:\/\/i0.wp.com\/www.lightsondata.com\/wp-content\/uploads\/2019\/09\/dw-lifecycle-quality-process.png?fit=558%2C167&amp;ssl=1\" class=\"aligncenter size-full wp-image-1259\" src=\"https:\/\/i0.wp.com\/www.lightsondata.com\/wp-content\/uploads\/2019\/09\/dw-lifecycle-quality-process.png?resize=558%2C167&#038;ssl=1\" alt=\"dw lifecycle quality process\" width=\"558\" height=\"167\" srcset=\"https:\/\/i0.wp.com\/www.lightsondata.com\/wp-content\/uploads\/2019\/09\/dw-lifecycle-quality-process.png?w=558&amp;ssl=1 558w, https:\/\/i0.wp.com\/www.lightsondata.com\/wp-content\/uploads\/2019\/09\/dw-lifecycle-quality-process.png?resize=300%2C90&amp;ssl=1 300w\" sizes=\"auto, (max-width: 558px) 100vw, 558px\" \/><\/p>\n<h2>1. Eliminating data quality errors<\/h2>\n<p>Injections of data quality problems occur during all phases of data warehousing:<\/p>\n<ul>\n<li>data warehouse modeling and schema design,<\/li>\n<li>ETL (extract, transformation, loading) design and coding,<\/li>\n<li>integrating data from varied sources, and<\/li>\n<li>running the ETL processes for data staging, cleaning, and loading.<\/li>\n<\/ul>\n<p>It\u2019s common, although undesirable, for problems and defects to emerge when populating the warehouse.<\/p>\n<p>Data testing is often planned for the later phases of data warehouse projects. However, most Quality Assurance (QA) professionals agree that establishing a successful test planning and execution effort for the early project phases is one of the keys to success. As with other software development, the earlier data errors are detected, the less expensive it is to find and correct them. Besides that, planning early testing activities to be carried out during the design and before implementation gives project managers an effective means to regularly measure and document the project progress state.<\/p>\n<h2>2. QA efforts should begin early<\/h2>\n<p>Since the quality of a DW can be measured best with reference to a set of data requirements, a successful testing process begins with the gathering and documentation of end-user data requirements. As most end-users data requirements are about data analysis and data quality, it is inevitable that data warehouse testing will focus primarily on the ETL process on the one hand (this is sometimes called back-end testing), then on reporting and Online Analytical Processing (<a href=\"https:\/\/en.wikipedia.org\/wiki\/Online_analytical_processing\" target=\"_blank\" rel=\"noopener noreferrer\">OLAP<\/a>) on the other (front-end testing).<\/p>\n<p>After gathering requirements, analysts develop conceptual, then detailed schemas to represent the user\u2019s needs as an important reference for testing. DW and BI designers are responsible for logical schemata of data repositories and for data staging definitions that should be tested for efficiency and robustness.<\/p>\n<p>The data architecture and model are necessary as a blueprint for any data warehouse. Understanding these artifacts and discovering potential defects help the QA team to comprehend the bigger picture of a data warehouse. The data model aids comprehension of the methods used for the key relationships between the major data sources. The relationship hierarchies and the depth of data throw light on the complexity of transformation rules.<\/p>\n<p>In order to gain greater value from the QA team, it\u2019s recommended to include them for quality assessments in all phases of data warehouse design and development and testing. Figure 1 shows typical DW project testing categories. Representative validations are shown associated with four significant QA contributions:<\/p>\n<ul>\n<li>Database \/ schema testing<\/li>\n<li>Data validation testing<\/li>\n<li>Performance testing<\/li>\n<li>Graphical User Interface (GUI) and business rule testing<\/li>\n<\/ul>\n<p>Reviews, verifications, recommendations for improvement are among QA team contributions that aid in early removal of defects.<\/p>\n<figure id=\"attachment_1258\" aria-describedby=\"caption-attachment-1258\" style=\"width: 687px\" class=\"wp-caption aligncenter\"><img data-recalc-dims=\"1\" loading=\"lazy\" decoding=\"async\" data-attachment-id=\"1258\" data-permalink=\"https:\/\/www.lightsondata.com\/3-ideas-data-warehouse-lifecycle-quality-process\/categories-of-dw-testing\/\" data-orig-file=\"https:\/\/i0.wp.com\/www.lightsondata.com\/wp-content\/uploads\/2019\/09\/categories-of-dw-testing.png?fit=687%2C419&amp;ssl=1\" data-orig-size=\"687,419\" data-comments-opened=\"1\" data-image-meta=\"{&quot;aperture&quot;:&quot;0&quot;,&quot;credit&quot;:&quot;&quot;,&quot;camera&quot;:&quot;&quot;,&quot;caption&quot;:&quot;&quot;,&quot;created_timestamp&quot;:&quot;0&quot;,&quot;copyright&quot;:&quot;&quot;,&quot;focal_length&quot;:&quot;0&quot;,&quot;iso&quot;:&quot;0&quot;,&quot;shutter_speed&quot;:&quot;0&quot;,&quot;title&quot;:&quot;&quot;,&quot;orientation&quot;:&quot;0&quot;}\" data-image-title=\"categories of dw testing\" data-image-description=\"\" data-image-caption=\"&lt;p&gt;Figure 1. Categories of data warehouse testing to discover quality issues (graphic courtesy of Virtusa Corp.)&lt;\/p&gt;\n\" data-medium-file=\"https:\/\/i0.wp.com\/www.lightsondata.com\/wp-content\/uploads\/2019\/09\/categories-of-dw-testing.png?fit=300%2C183&amp;ssl=1\" data-large-file=\"https:\/\/i0.wp.com\/www.lightsondata.com\/wp-content\/uploads\/2019\/09\/categories-of-dw-testing.png?fit=687%2C419&amp;ssl=1\" class=\"size-full wp-image-1258\" src=\"https:\/\/i0.wp.com\/www.lightsondata.com\/wp-content\/uploads\/2019\/09\/categories-of-dw-testing.png?resize=687%2C419&#038;ssl=1\" alt=\"categories of dw testing\" width=\"687\" height=\"419\" srcset=\"https:\/\/i0.wp.com\/www.lightsondata.com\/wp-content\/uploads\/2019\/09\/categories-of-dw-testing.png?w=687&amp;ssl=1 687w, https:\/\/i0.wp.com\/www.lightsondata.com\/wp-content\/uploads\/2019\/09\/categories-of-dw-testing.png?resize=300%2C183&amp;ssl=1 300w, https:\/\/i0.wp.com\/www.lightsondata.com\/wp-content\/uploads\/2019\/09\/categories-of-dw-testing.png?resize=600%2C366&amp;ssl=1 600w\" sizes=\"auto, (max-width: 687px) 100vw, 687px\" \/><figcaption id=\"caption-attachment-1258\" class=\"wp-caption-text\"><strong>Figure 1.<\/strong> Categories of data warehouse testing to discover quality issues (graphic courtesy of Virtusa Corp.)<\/figcaption><\/figure>\n<hr \/>\n<h3 style=\"text-align: center;\"><em>Here&#8217;s <a href=\"https:\/\/www.lightsondata.com\/master-test-plan-for-dw-bi-project\/\" target=\"_blank\" rel=\"noopener noreferrer\">what no one tells you about developing a Master Test Plan for your DW\/ BI project<\/a><\/em><\/h3>\n<hr \/>\n<p>&nbsp;<\/p>\n<h2>3. QA team contributions during the DW\/BI project lifecycle<\/h2>\n<p>Following are sample contributions and benefits from the QA team during the DW development lifecycle:<\/p>\n<p><strong>Planning for data integration and the ETL (data model, low-level DW design)<\/strong><\/p>\n<ul>\n<li>Testers gain an understanding of data to be reported by the application (e.g., profiling) and the tables upon which BI and other reports will be based<\/li>\n<li>Testers review and understand the data model \u2013 gain an understanding of keys, ETL workflows and more from sources to targets<\/li>\n<li>Testers review and become familiar with data Low Level Design (LLD) and mappings: add, update sequences for all sources for each target element<\/li>\n<\/ul>\n<p><strong>Planning for ETL verifications<\/strong><\/p>\n<ul>\n<li>Testers participate in ETL design reviews<\/li>\n<li>The QA team gains an in-depth knowledge of ETL workflows \/ sessions, the order of job executions, restraints, transformations<\/li>\n<li>Testers develop ETL test scenarios and distribute for reviews<\/li>\n<\/ul>\n<p><strong>Assessing ETL run and diagnostic logs: sessions, workflows, errors<\/strong><\/p>\n<ul>\n<li>After ETL\u2019s are run, testers use checklists and test scenarios for QA assessments of rejects, session failures, errors<\/li>\n<li>QA teams review ETL workflow outputs, source to target counts<\/li>\n<li>Verify source to target mapping docs with loaded tables using a database management toolset (ex: TOAD, Navicat, MS SQL Server Management Studio, etc.)<\/li>\n<li>After ETL runs or manual data loads, assess data in every table with a focus on key fields (dirty data, incorrect formats, duplicates, etc.). Use database management tools or simple spreadsheet tools. (SQL queries, filtering, etc.)<\/li>\n<\/ul>\n<p>During DW design and development, testers plan and organize for the following DW quality categories:<\/p>\n<p style=\"padding-left: 40px;\"><strong>Data completeness:<\/strong> Ensuring that all expected data is loaded<br \/>\n<strong>Data transformation:<\/strong> Ensuring that all data is transformed correctly according to business rules and\/or design specifications<br \/>\n<strong>Data quality:<\/strong> Ensuring that the ETL system correctly rejects, substitutes default values, corrects (or ignores) invalid data<br \/>\n<strong>Performance and scalability:<\/strong> Ensuring that data loads and queries perform within expected time frames and that the technical architecture is scalable<br \/>\n<strong>Integration testing:<\/strong> Ensuring that the ETL process functions well with other upstream and downstream processes<br \/>\n<strong>User-acceptance testing:<\/strong> Ensuring the warehouse solution meets users&#8217; current requirements and anticipates their future expectations<br \/>\n<strong>Regression testing:<\/strong> Ensuring that current functionality remains intact each time a new release of code is completed<\/p>\n<p>Testing cannot guarantee that there will be no data errors. There are too many combinations and permutations, therefore, it is not practical to test each one. However, by joining forces with business analysts, database designers, developers and ranking the types of errors as suggested above, DW\/BI projects will avoid wasting time on creating test scripts and test scenarios for less important possibilities and not having time to create test scripts and test scenarios for possibilities in which errors could significantly diminish or destroy the value of the data warehouse to the users.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Data warehousing for business intelligence and \u201cbig data initiatives\u201d continues to gain significance as organizations become more aware of the benefits of decision oriented data warehouses. However, a key issue, with the rapid development and implementation of data warehouses, is that data quality defects are often injected during the multiple lifecycle stages of the warehouse. [&hellip;]<\/p>\n","protected":false},"author":12,"featured_media":1263,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"3 new ideas on improving the data warehouse life-cycle quality process\r\n#BI #datawarehouse #lightsondata #data","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[4],"tags":[52],"class_list":["post-1257","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-data-quality","tag-data-warehouse","post-wrapper","thrv_wrapper"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v26.4 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>3 new ideas on improving the data warehouse lifecycle quality process | LightsOnData<\/title>\n<meta name=\"description\" content=\"Data quality defects often happen during the lifecycle stages of the warehouse. Here are 3 ideas on improving the data warehouse lifecycle quality process.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.lightsondata.com\/3-ideas-data-warehouse-lifecycle-quality-process\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"3 new ideas on improving the data warehouse lifecycle quality process | LightsOnData\" \/>\n<meta property=\"og:description\" content=\"Data quality defects often happen during the lifecycle stages of the warehouse. Here are 3 ideas on improving the data warehouse lifecycle quality process.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.lightsondata.com\/3-ideas-data-warehouse-lifecycle-quality-process\/\" \/>\n<meta property=\"og:site_name\" content=\"LightsOnData\" \/>\n<meta property=\"article:published_time\" content=\"2019-09-25T19:01:38+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2020-09-07T04:47:50+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/i2.wp.com\/www.lightsondata.com\/wp-content\/uploads\/2019\/09\/3-new-ideas-improving-dw-lifecycle-quality-process.png?fit=800%2C450&ssl=1\" \/>\n\t<meta property=\"og:image:width\" content=\"800\" \/>\n\t<meta property=\"og:image:height\" content=\"450\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Wayne Yaddow\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@georgefirican\" \/>\n<meta name=\"twitter:site\" content=\"@georgefirican\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Wayne Yaddow\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.lightsondata.com\/3-ideas-data-warehouse-lifecycle-quality-process\/\",\"url\":\"https:\/\/www.lightsondata.com\/3-ideas-data-warehouse-lifecycle-quality-process\/\",\"name\":\"3 new ideas on improving the data warehouse lifecycle quality process | LightsOnData\",\"isPartOf\":{\"@id\":\"https:\/\/www.lightsondata.com\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.lightsondata.com\/3-ideas-data-warehouse-lifecycle-quality-process\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.lightsondata.com\/3-ideas-data-warehouse-lifecycle-quality-process\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/i0.wp.com\/www.lightsondata.com\/wp-content\/uploads\/2019\/09\/3-new-ideas-improving-dw-lifecycle-quality-process.png?fit=800%2C450&ssl=1\",\"datePublished\":\"2019-09-25T19:01:38+00:00\",\"dateModified\":\"2020-09-07T04:47:50+00:00\",\"author\":{\"@id\":\"https:\/\/www.lightsondata.com\/#\/schema\/person\/4503a79021fcf6acf4850c36356b6ffe\"},\"description\":\"Data quality defects often happen during the lifecycle stages of the warehouse. Here are 3 ideas on improving the data warehouse lifecycle quality process.\",\"breadcrumb\":{\"@id\":\"https:\/\/www.lightsondata.com\/3-ideas-data-warehouse-lifecycle-quality-process\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.lightsondata.com\/3-ideas-data-warehouse-lifecycle-quality-process\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.lightsondata.com\/3-ideas-data-warehouse-lifecycle-quality-process\/#primaryimage\",\"url\":\"https:\/\/i0.wp.com\/www.lightsondata.com\/wp-content\/uploads\/2019\/09\/3-new-ideas-improving-dw-lifecycle-quality-process.png?fit=800%2C450&ssl=1\",\"contentUrl\":\"https:\/\/i0.wp.com\/www.lightsondata.com\/wp-content\/uploads\/2019\/09\/3-new-ideas-improving-dw-lifecycle-quality-process.png?fit=800%2C450&ssl=1\",\"width\":800,\"height\":450,\"caption\":\"3 new ideas improving datawarehouse lifecycle quality process\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.lightsondata.com\/3-ideas-data-warehouse-lifecycle-quality-process\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.lightsondata.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"3 new ideas on improving the data warehouse lifecycle quality process\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.lightsondata.com\/#website\",\"url\":\"https:\/\/www.lightsondata.com\/\",\"name\":\"LightsOnData\",\"description\":\"Practical resources, online courses, free articles and videos for data management, data governance, data quality, and business intelligence\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.lightsondata.com\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.lightsondata.com\/#\/schema\/person\/4503a79021fcf6acf4850c36356b6ffe\",\"name\":\"Wayne Yaddow\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.lightsondata.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/0c5eaab6104044bec265d829746d99a451bd1acb77e74af42b55c757e191fe76?s=96&d=retro&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/0c5eaab6104044bec265d829746d99a451bd1acb77e74af42b55c757e191fe76?s=96&d=retro&r=g\",\"caption\":\"Wayne Yaddow\"},\"description\":\"Wayne Yaddow is an independent consultant with more than 20 years\u2019 experience leading data integration, data warehouse, and ETL testing projects with J.P. Morgan Chase, Credit Suisse, Standard and Poor\u2019s, AIG, Oppenheimer Funds, and IBM. He taught IIST (International Institute of Software Testing) courses on data warehouse and ETL testing and wrote DW\/BI articles for Better Software, The Data Warehouse Institute (TDWI), Tricentis, and others. Wayne continues to lead numerous ETL testing and coaching projects on a consulting basis. You can contact him at wyaddow@gmail.com.\",\"url\":\"https:\/\/www.lightsondata.com\/author\/wyaddowgmail-com\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"3 new ideas on improving the data warehouse lifecycle quality process | LightsOnData","description":"Data quality defects often happen during the lifecycle stages of the warehouse. Here are 3 ideas on improving the data warehouse lifecycle quality process.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.lightsondata.com\/3-ideas-data-warehouse-lifecycle-quality-process\/","og_locale":"en_US","og_type":"article","og_title":"3 new ideas on improving the data warehouse lifecycle quality process | LightsOnData","og_description":"Data quality defects often happen during the lifecycle stages of the warehouse. Here are 3 ideas on improving the data warehouse lifecycle quality process.","og_url":"https:\/\/www.lightsondata.com\/3-ideas-data-warehouse-lifecycle-quality-process\/","og_site_name":"LightsOnData","article_published_time":"2019-09-25T19:01:38+00:00","article_modified_time":"2020-09-07T04:47:50+00:00","og_image":[{"width":800,"height":450,"url":"https:\/\/i2.wp.com\/www.lightsondata.com\/wp-content\/uploads\/2019\/09\/3-new-ideas-improving-dw-lifecycle-quality-process.png?fit=800%2C450&ssl=1","type":"image\/png"}],"author":"Wayne Yaddow","twitter_card":"summary_large_image","twitter_creator":"@georgefirican","twitter_site":"@georgefirican","twitter_misc":{"Written by":"Wayne Yaddow","Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/www.lightsondata.com\/3-ideas-data-warehouse-lifecycle-quality-process\/","url":"https:\/\/www.lightsondata.com\/3-ideas-data-warehouse-lifecycle-quality-process\/","name":"3 new ideas on improving the data warehouse lifecycle quality process | LightsOnData","isPartOf":{"@id":"https:\/\/www.lightsondata.com\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.lightsondata.com\/3-ideas-data-warehouse-lifecycle-quality-process\/#primaryimage"},"image":{"@id":"https:\/\/www.lightsondata.com\/3-ideas-data-warehouse-lifecycle-quality-process\/#primaryimage"},"thumbnailUrl":"https:\/\/i0.wp.com\/www.lightsondata.com\/wp-content\/uploads\/2019\/09\/3-new-ideas-improving-dw-lifecycle-quality-process.png?fit=800%2C450&ssl=1","datePublished":"2019-09-25T19:01:38+00:00","dateModified":"2020-09-07T04:47:50+00:00","author":{"@id":"https:\/\/www.lightsondata.com\/#\/schema\/person\/4503a79021fcf6acf4850c36356b6ffe"},"description":"Data quality defects often happen during the lifecycle stages of the warehouse. Here are 3 ideas on improving the data warehouse lifecycle quality process.","breadcrumb":{"@id":"https:\/\/www.lightsondata.com\/3-ideas-data-warehouse-lifecycle-quality-process\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.lightsondata.com\/3-ideas-data-warehouse-lifecycle-quality-process\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.lightsondata.com\/3-ideas-data-warehouse-lifecycle-quality-process\/#primaryimage","url":"https:\/\/i0.wp.com\/www.lightsondata.com\/wp-content\/uploads\/2019\/09\/3-new-ideas-improving-dw-lifecycle-quality-process.png?fit=800%2C450&ssl=1","contentUrl":"https:\/\/i0.wp.com\/www.lightsondata.com\/wp-content\/uploads\/2019\/09\/3-new-ideas-improving-dw-lifecycle-quality-process.png?fit=800%2C450&ssl=1","width":800,"height":450,"caption":"3 new ideas improving datawarehouse lifecycle quality process"},{"@type":"BreadcrumbList","@id":"https:\/\/www.lightsondata.com\/3-ideas-data-warehouse-lifecycle-quality-process\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.lightsondata.com\/"},{"@type":"ListItem","position":2,"name":"3 new ideas on improving the data warehouse lifecycle quality process"}]},{"@type":"WebSite","@id":"https:\/\/www.lightsondata.com\/#website","url":"https:\/\/www.lightsondata.com\/","name":"LightsOnData","description":"Practical resources, online courses, free articles and videos for data management, data governance, data quality, and business intelligence","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.lightsondata.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/www.lightsondata.com\/#\/schema\/person\/4503a79021fcf6acf4850c36356b6ffe","name":"Wayne Yaddow","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.lightsondata.com\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/0c5eaab6104044bec265d829746d99a451bd1acb77e74af42b55c757e191fe76?s=96&d=retro&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/0c5eaab6104044bec265d829746d99a451bd1acb77e74af42b55c757e191fe76?s=96&d=retro&r=g","caption":"Wayne Yaddow"},"description":"Wayne Yaddow is an independent consultant with more than 20 years\u2019 experience leading data integration, data warehouse, and ETL testing projects with J.P. Morgan Chase, Credit Suisse, Standard and Poor\u2019s, AIG, Oppenheimer Funds, and IBM. He taught IIST (International Institute of Software Testing) courses on data warehouse and ETL testing and wrote DW\/BI articles for Better Software, The Data Warehouse Institute (TDWI), Tricentis, and others. Wayne continues to lead numerous ETL testing and coaching projects on a consulting basis. You can contact him at wyaddow@gmail.com.","url":"https:\/\/www.lightsondata.com\/author\/wyaddowgmail-com\/"}]}},"jetpack_publicize_connections":[],"jetpack_featured_media_url":"https:\/\/i0.wp.com\/www.lightsondata.com\/wp-content\/uploads\/2019\/09\/3-new-ideas-improving-dw-lifecycle-quality-process.png?fit=800%2C450&ssl=1","jetpack_shortlink":"https:\/\/wp.me\/p9BPV6-kh","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/www.lightsondata.com\/wp-json\/wp\/v2\/posts\/1257","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.lightsondata.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.lightsondata.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.lightsondata.com\/wp-json\/wp\/v2\/users\/12"}],"replies":[{"embeddable":true,"href":"https:\/\/www.lightsondata.com\/wp-json\/wp\/v2\/comments?post=1257"}],"version-history":[{"count":4,"href":"https:\/\/www.lightsondata.com\/wp-json\/wp\/v2\/posts\/1257\/revisions"}],"predecessor-version":[{"id":1297,"href":"https:\/\/www.lightsondata.com\/wp-json\/wp\/v2\/posts\/1257\/revisions\/1297"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.lightsondata.com\/wp-json\/wp\/v2\/media\/1263"}],"wp:attachment":[{"href":"https:\/\/www.lightsondata.com\/wp-json\/wp\/v2\/media?parent=1257"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.lightsondata.com\/wp-json\/wp\/v2\/categories?post=1257"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.lightsondata.com\/wp-json\/wp\/v2\/tags?post=1257"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}