Sorry to bother you: designing bots for effective recommendations

Bots have been proposed as a way to encourage developer actions and support software development activities. Many bots make recommendations to users, however humans may find these recommendations ineffective or problematic. In this paper, we argue that while bots can help automate many tasks, ultimately bots still need to find ways to interact with humans and handle all of the associated social and cognitive problems entailed. To illustrate this problem, we performed a small study where we generated 52 pull requests making tool recommendation to developers. As a result, we only convinced two developers to accept the pull request, while receiving several forms of feedback on why the pull request was ineffective. We summarize this feedback and suggest design principles for bot recommendations, including how psychology frameworks, such as nudge theory, can be used to improve human-bot interactions.