Local parallelization of pleasingly parallel stream processing on multiple CPU cores

Data stream processing addresses the need for high-throughput near real-time data processing, which can be considered as one part of Big Data or Fast Data. In this paper, we study the local parallelization of stream processing on a single multi-core Central Processing Unit (CPU) computer system, which, in our opinion, was not sufficiently addressed yet. In distributed systems, optimizing the local throughput can help to improve the overall system. In less resource demanding scenarios, it may be beneficial to use more lightweight local parallelization instead of more complex distributed approaches. We present our work-in-progress on locally parallelizing stream processing on multiple CPU cores and on ways for further improving the local data processing. In order to study the fundamental mechanisms and effects, we focused on pleasingly parallel workloads. While pleasingly parallel tasks, by definition, can be easily parallelized, our results show that stream processing adds important aspects and that the outcomes strongly vary depending on use case and parallelization approach. Furthermore, we present early stages of a stream transformation Domain Specific Language and of a self-adaptive mechanism for easing and optimizing the processing. We published our implementations as Open Source Software.