基于粗糙集特征选择的过拟合现象及应对策略

在粗糙集方法中,利用向前启发式算法进行特征选择,是一个逐步加入重要度最高的特征的过程,直至满足所给定的约束条件。但使用这一策略选择出来的特征子集有可能产生过拟合现象。鉴于此,设计了一种新的启发式算法,其主要思想是借助交叉验证的方法对特征的重要度进行计算,当过拟合出现时,则采用截断式机制终止算法。使用邻域粗糙集模型,在UCI数据集上将启发式算法与所提算法进行对比分析,实验结果表明:所提算法能够有效地降低过拟合的程度;利用所提算法得到的特征子集能够带来更好的分类性能。

Over-Fitting and Its Countermeasure in Feature Selection Based on Rough Set

In rough set theory, forward heuristic algorithm selects the most important feature in the process of feature selection until the given constraint is satisfied. However, the feature subset selected by such strategy may bring us over-fitting. To solve this problem, a new heuristic algorithm is designed. The importance of the feature is obtained by cross validation and then the early stopping is employed to terminate the algorithm when over-fitting occurs. Based on the neighborhood rough set, the new method is compared with the heuristic algorithm over several UCI data sets. The experimental results show that: the proposed algorithm can effectively reduce the degree of over-fitting, and the feature subset obtained by the new algorithm may offer better classification performances.