Some time ago, I was asked: How is the company's data quality management controlled? At present, most of the data departments of Internet companies Whatsapp Database will build group data warehouses, and the data sources of Whatsapp Database upper-level data products basically come from data warehouses. Therefore, I understand this question as: how to ensure the data quality in the enter. Drise data warehouse? Combined with previous data project experience, I made a simple answer: (1)
Data infrastructure construction If you want to Whatsapp Database have a high-quality data warehouse, first of all, from the design of the data warehouse, we must have a perfect subject domain with clear layers (usually divided into ODS [data source surface layer], Whatsapp Database DWD [data detail layer], DWS [ A data warehouse system with clear data consumption scenarios and clear data processing links. With this foundation, we can monitor data at different subject areas and at different levels. (2)
Data processing monitoring Through data blood relationship management, monitor and locate the execution node with problems on the data Whatsapp Database processing link, and notify the corresponding person in charge through the system or email or enterprise employee management platform. (3) Business system adjustment response The first is to add new Whatsapp Database business modules, resulting in new data that needs to be connected to the data warehouse in time. The second is the change of the business module, which has led to changes in the historical statistical caliber of some indicators in the data warehouse.