Dataset Weighting via Intrinsic Data Characteristics for Pairwise Statistical Comparisons in Classification

  1. José A. Sáez 1
  2. Pablo José Villacorta Iglesias 2
  3. Emilio Santiago Corchado Rodríguez 1
  1. 1 Universidad de Salamanca

    Universidad de Salamanca

    Salamanca, España


  2. 2 Universidad de Granada

    Universidad de Granada

    Granada, España


Hybrid Artificial Intelligent Systems. 14th International Conference, HAIS 2019: León, Spain, September 4–6, 2019. Proceedings
  1. Hilde Pérez García (ed. lit.)
  2. Lidia Sánchez González (ed. lit.)
  3. Manuel Castejón Limas (ed. lit.)
  4. Héctor Quintián Pardo (ed. lit.)
  5. Emilio Santiago Corchado Rodríguez (ed. lit.)

Publisher: Springer Suiza

ISBN: 978-3-030-29859-3 978-3-030-29858-6

Year of publication: 2019

Pages: 61-72

Congress: Hybrid Artificial Intelligent Systems (14. 2019. León)

Type: Conference paper


In supervised learning, some data characteristics (e.g. presence of errors, overlapping degree, etc.) may negatively influence classifier performance. Many methods are designed to overcome the undesirable effects of the aforementioned issues. When comparing one of those techniques with existing ones, a proper selection of datasets must be made, based on how well each dataset reflects the characteristic being specifically addressed by the proposed algorithm. In this setting, statistical tests are necessary to check the significance of the differences found in the comparison of different methods. Wilcoxon’s signed-ranks test is one of the most well-known statistical tests for pairwise comparisons between classifiers. However, it gives the same importance to every dataset, disregarding how representative each of them is in relation to the concrete issue addressed by the methods compared. This research proposes a hybrid approach which combines techniques of measurement for data characterization with statistical tests for decision making in data mining. Thus, each dataset is weighted according to its representativeness of the property of interest before using Wilcoxon’s test. Our proposal has been successfully compared with the standard Wilcoxon’s test in two scenarios related to the noisy data problem. As a result, this approach stands out properties of the algorithms easier, which may otherwise remain hidden if data characteristics are not considered in the comparison.