一种开放式数据清理框架

从提高制造业信息化中数据质量的角度出发,分析了建立基于规则库和算法库的可扩展数据清理框架的必要性,设计了一种基于多种规则结合不同算法以流程方式完成清理任务的可重构的数据清理框架。在框架的设计中,以实例方式总结了数据质量问题的分类,详细阐述了该框架的结构、组成部分以及工作流程。最后,通过数据清理实例说明了该框架中流程的实现及其工作过程。本文提出的数据清理框架功能将随着不同领域清理任务的完成而不断完善。

Open Data Cleaning Frame

To improve the data quality in manufacturing information,the necessity of the data cleaning frame(DCF) is analyzed and an extensible DCF is proposed based on the rule warehouse and the arithmetic algorithm warehouse.And an extensible DCF implemented by the flow mode based on multi-rule combining the variable algorithm is designed.In the frame design,the problem of the data quality is summarized using examples,then the structure,components and the workflow of the frame are expatiated.Finally,an example for data cleaning shows the realization process of the workflow in the frame.The function of the proposed DCF becomes more and more perfect and powerful with the success of data cleaning practice in different fields.