Preparing heterogeneous data landscapes for a data analytics platform using data quality checks

In practice, the input of incomplete and incorrect data, for example caused by empty mandatory fields, incorrect links or values, is not uncommon and poses a problem for later analysis. The goal should be a quantitative analysis of the summarized company data in order to use it, for example, to optimize marketing strategies.

In practice, the entry of incomplete and incorrect data, for example caused by empty mandatory fields, incorrect links or values, is not uncommon and represents a problem for later analysis. The goal should be a quantitative analysis of the summarized company data in order to use it, for example, for the optimization of marketing strategies.

Our department Software & Information Engineering of Fraunhofer IML was assigned to prepare a high amount of heterogeneous data for a consistent use and application in a data analytics platform. This is a cooperation with CDQ AG.

In order to create a homogeneous data structure, data must be checked by means of data quality checks and incorrect data sets must be displayed before integration.

This is where the cooperation with CDQ AG as an industrial consultant in the area of master data and data quality and our Software & Information Engineering department of IML as a technically experienced partner starts:

  •  Initially, CDQ AG was responsible for the creation of the data quality check and for the development of the concept and methodology for the revision
  •  In the next step, our IML team dealt with the software implementation of the data quality checks in the data analytics platform

Here our team proved its competence in the development and in the training in external systems within a given IT infrastructure as well as in the interpretation of the data. An interface between the cooperation partners was created, as experience and architectural as well as technical knowledge interacted.

If you have any feedback, please contact our staffmember Timo Erler