Automated Byproduct Segmentation In Grain Images

Within the farming industry using computer vision technologies provides great support for various processes. One example is the grain harvest with a combine harvester, which requires excellent expert knowledge to reach a certain level of quality. This means the amount of byproducts including straw and hulls should be as small as possible. Inside a combine harvester several sensors are installed which deliver important information about the current harvesting state. This includes a visual based sensor that constantly delivers images to the driver about the composition of the processed grain. Normally, it is the driver's task to decide if the quality is sufficient or if machine settings need to be adjusted to improve the outcome. Yet, resolving this task entirely manually is rather error prone, due to the high amount of fairly similar images that need to be analyzed in a short time. Therefore we designed and implemented a system that is able to automatically detect unwanted byproducts within the wheat harvest and highlight those parts inside images delivered by the visual sensor. The system itself is a combination of an automated preprocessing step and a variation of a U-Net. The preprocessing step filters unwanted byproducts and is able to automatically generate training data for the U-Net. The U-Net is lightweight, easy to train and can potentially be used onboard the combine harvester.