Multimodal Recipe Recommendation System Using Deep Learning and Rule-Based Approach

In today’s age of the internet, there is tremendous growth in information. Moreover, this information can be easily accessed through different combinations of large-scale interconnected networks. But the information available is highly unstructured and requires a certain degree of intelligence to garner the requisite information required by the user. Hence, present age recommendation engines provide the most suitable answer considering different modalities of data, the most common form of which are text and images. Due to the wide range of possibilities and increased awareness among users, the service should be highly satisfactory. This paper has contributed a recipe recommendation system that relies on deep learning and a rule-based approach. It aims to recommend recipes based on input ingredients or food images. To test the proficiency of the designed recommendation system, we have created a dataset of recipes that mainly includes the recipe of food and their images. Finally, the designed recommendation system allows users to look for the desired information through text and image searches.