Extended pre-processing pipeline for text classification: On the role of meta-feature representations, sparsification and selective sampling

•We propose and orchestrate new pre-processing steps for text classification pipelines.•We explore meta-feature representations, sparsification and selective sampling.•We provide thorough evaluations of the trade-offs between costs and effectiveness.•Our final representations are more effective than word embeddings (up to 46%).•Our processes induce large reductions in computational costs and memory consumption.