Wayne Yaddow, Author at LightsOnData

What data quality testing skills are needed for data integration projects?

Wayne Yaddow — Wed, 12 Aug 2020 15:38:53 +0000

The impulse to cut project costs is often strong, especially in the final delivery phase of data integration and data migration projects. At this late phase of the project, a common mistake is to delegate testing responsibilities to resources with limited business and data testing skills. Data integrations are at the core of data warehousing, […]

The post What data quality testing skills are needed for data integration projects? appeared first on LightsOnData.

Managing DW/ BI data integration risks through data reconciliation and data lineage processes

Wayne Yaddow — Thu, 09 Jul 2020 00:40:54 +0000

Introduction Data lineage documents where data is coming from, where it is going, and what transformations are applied to it as it flows through multiple processes. It helps in understanding the data life cycle. It is one of the most critical pieces of information from a metadata management point of view. From data-quality and data-governance perspectives, it is essential […]

The post Managing DW/ BI data integration risks through data reconciliation and data lineage processes appeared first on LightsOnData.

Main considerations for testing BI reports

Wayne Yaddow — Wed, 03 Jun 2020 15:28:56 +0000

Among the most important phases of DW/BI testing is front-end report testing. The DW/BI project team should ensure that everything tested in the back-end DW and ETLs is now processed and displayed correctly in end-user reports. BI reports are often the most visible product of any DW/BI system. Data quality defects that are not discovered […]

The post Main considerations for testing BI reports appeared first on LightsOnData.

Data quality considerations during the DW/ BI design phase

Wayne Yaddow — Wed, 20 May 2020 15:05:59 +0000

Decisions in today’s organizations have become increasingly data-driven and real-time. Therefore, the business intelligence databases that support decision makers must be of exceptional quality. We sometimes confuse testing a data warehouse that produce business intelligence (BI) reports with backend or database testing or with testing the BI reports themselves. Data warehouse testing is much more […]

The post Data quality considerations during the DW/ BI design phase appeared first on LightsOnData.

What you need to know about regression testing on DW/ BI projects (part 2)

Wayne Yaddow — Wed, 29 Apr 2020 13:00:26 +0000

In the first article of this series on DW/BI regression testing, DW/BI regression test planning was defined accordingly: Testing performed after developing functional improvements or repairs to data and reports. Its purpose is to determine if changes have regressed other attributes of data and reports. A repetition of tests intended to show that the software’s […]

The post What you need to know about regression testing on DW/ BI projects (part 2) appeared first on LightsOnData.

What you need to know about regression testing on DW/ BI projects

Wayne Yaddow — Wed, 08 Apr 2020 15:03:19 +0000

Regression testing on large data integration DW/BI development projects is challenging. The number of test cases is often massive and many change impacts may be widely spread. For high-level integration regression testing, the retest-all approach is always time and resource consuming. To counter challenges with regression testing, this article proposes test scenarios based on changes […]

The post What you need to know about regression testing on DW/ BI projects appeared first on LightsOnData.

ETL test automation for DWH and BI – free white paper

Wayne Yaddow — Wed, 29 Jan 2020 19:58:09 +0000

When your organization depends on business intelligence reporting, you will need a high level of process automation within your DWH/BI application development. When you can build automation into your DWH development process, you will need fewer highly skilled people to perform tedious and repetitive testing processes – usually, every day. It is practical to increase […]

The post ETL test automation for DWH and BI – free white paper appeared first on LightsOnData.

The what, the why, and the how of shift left testing for BI and Data Warehouse Projects

Wayne Yaddow — Wed, 15 Jan 2020 23:33:22 +0000

Shift left testing is an approach to Data Warehouse (DW) / BI development where testing is performed early in the lifecycle. Shifting left refers not only to dynamic data and BI application testing- it also refers to static testing, conducting reviews and inspections, and even more unit and integration testing. In this article, we highlight: […]

The post The what, the why, and the how of shift left testing for BI and Data Warehouse Projects appeared first on LightsOnData.

5 crucial data warehouse testing checklists

Wayne Yaddow — Wed, 11 Dec 2019 16:27:35 +0000

You probably use checklists to record and efficiently execute a wide range of daily work tasks. But if you don’t use data warehouse testing checklists for developing and monitoring your data warehouse quality assurance (QA), you’re missing an enormous boost in productivity and proficiency. Procedural data warehouse checklists serve as concrete reminders of which jobs we […]

The post 5 crucial data warehouse testing checklists appeared first on LightsOnData.

Best practices to achieve optimal source data profiling

Wayne Yaddow — Wed, 27 Nov 2019 18:28:40 +0000

Today’s Data Warehoues (DW) source data often originates from a large variety of data formats as illustrated in Figure 1. Original sources may consist of external data, reference data, business transactions, production raw data and more. The multitude of data sources each has its own methods, systems, and architectures for storing data. When working with […]

The post Best practices to achieve optimal source data profiling appeared first on LightsOnData.