On feature selection protocols for very low-sample-size data.

•Feature selection with very few instances, possibly high-dimensional.•Widely used protocol: 1) feature selection, 2) cross-validation to test a classifier.•Alternative, proper, protocol includes both steps in a single cross-validation loop.•Experiment using 24 datasets, 3 feature selection methods and 5 classifier models.•The proper protocol accuracy is significantly closer to the true accuracy.