What is this

• A simple web application that aims to recommend products to customers based on their emotions about other products.
• This was the final project to finish the 3rd year in university.

Requirements

• Python 3.6+
• Flask 1.0.2 for the entire web app
• passlib 1.7.1 for hashing passwords
• Keras 2.2.4 for 5-star sentiment analyzer
• Tensorflow 1.13.1 Keras backend (atleast in my case)
• FFmpeg for converting wav files to 16000Hz sample rate (install here)
• SciPy for the recommender system ( Used Matrix-Factorization )
• NumPy
• Pandas

pip3 install -r requirements.txt

Home Page

Figure 1: Home page

The customer at first is going to register for a new user and fill the form shown in Figure 2, once finished, you can see when he/she clicks at "Recommended Products" in the navigation bar shown in Figure 3, no products and content will be shown, that is because no rating ( or emotion ) has been successfully registered by the customer, the page will show as captured in Figure 4

Figure 2: Registration Form

Figure 3: Navigation Bar

Figure 4: No products available

When the user likes a dish, for example let us say "Grilled Chicken", he/she needs to click the card of the specified product as displayed in Figure 5, once finished, a new page will appear with a high quality image of the dish he chose, he/she can now express his/her emotion and rating ( Note that the recognizer will detect the emotion and star rating in speech tone and not text ) about that food using the microphone. At first no emotion will be visible as shown in Figure 6, but when the four seconds of recording is completed, the system will automatically capture the emotion & star rating and will show the results very clearly as shown in Figure 7

Figure 5: The desired example food chosen by a customer

Figure 6: The emotion status before expressing any emotion

Figure 7: The emotion status after expressing the emotion

When the customer now wants to view his recommended products in the navbar mentioned earlier, the top three dishes actually contains chicken and meat as shown in Figure 8

Figure 8: Top 3 Recommended Dishes to the Chicken fan

Models used

• Star review rating: Converts 3 emotions ( sad, neutral and happy ) to a number between 0 and 5
• Emotion Recognizer: Different models for different emotion sets, such as (sad, neutral and happy) or (angry, sad, neutral, surprised, happy), etc. Accomplished with the help of this repository.
• Recommender System: A model that recommend products based on review ratings about products, the method used is Matrix Factorization and the code is in recommender/core.py

Thanks to

• Anis LC00L for helping me accomplish this college project.