Hello. Welcome to our course and building recommendation systems with TensorFlow on Google Cloud Platform. My name is Lak, and I lead the team that is putting together this course and this specialization. This is the fifth and final course of the advanced machine-learning on GCP specialization. In this course, we will learn how to build recommendation systems. As usual, you will get hands-on practice building machine learning models on a variety of datasets in the labs that we will work on together. We start out by explaining what recommendation systems are. When you watch a video on YouTube and you see a list of suggested videos to watch next, that list is being built by a recommendation machine-learning model, often called a recommendation engine. Of course, just the ML model, just the recommendation engine, is not enough, someone needs to build a data pipeline to collect whatever input data the model needs. Inputs like the last five videos that you watched. That is done by a recommendation system. Recommendation systems are not just about suggesting products to users, sometimes they can be about suggesting users for products. For example, in marketing applications, you may have a new promotion and you want to find the thousand most relevant current customers. That's called targeting, and that is also done by a recommendation system. Many times, recommendation systems are not about what you would think of as products. When Google Maps suggests the route that avoids toll roads, that's a recommendation system. When Smart Reply in Gmail suggests possible replies to an email that you received, that too is a recommendation system. Recommendation systems are about personalization. It's about taking your product that works for everyone and personalizing it for an individual user. Let's say you want to recommend movies to users. You can do this in several ways. The first of these is to use a content-based recommendation system. In a content-based recommendation system, you use the metadata about your products. For example, perhaps you know which movies are cartoons and which movies are Sci-Fi. Now, suppose you have a user who has seen and rated a few movies, some that she liked and give a thumbs up, and that some that she didn't. We would like to know which movie in our database we need to recommend next. Remember, that we have metadata about the movies. We know that this particular user likes Sci-Fi and doesn't like cartoons, so we might use that information to recommend popular Sci-Fi dramas to this user. Perhaps we recommend The Dark Knight Rises. Notice that this recommendation is based on knowing something about the content. You're simply recommending the most popular items in a category that the user likes. Maybe you don't even have the individual user's preferences. All you might have is a market segmentation, which movies are liked by users in which regions of the country, and that is enough to build a content-based recommendation system. Arguably, there is no machine-learning here. It's a simple rule that relies on the builder of the recommendation system to assign proper tags to items and users. That's a content-based recommendation system. In collaborative filtering, you don't have any metadata about the products. Instead, you learn about item similarity and user similarity from the ratings data itself. We might store our user movie data in a matrix like this, with checkmarks indicating whether the user watched the complete movie, or commented on it, or gave it a star rating, or however it is that we measure that a specific user liked a specific movie. This matrix is very large. Since you might have millions to billions of users and a hundreds to millions of movies, an individual will have tend to have watched only a handful of these movies. Most of this matrix is not only large, this matrix is sparse. The idea behind collaborative filtering is that this very large, very sparse user by item matrix can be approximated by the product of two smaller matrices called user-factors and item factors. Then, if we need to find whether a particular user will like a particular movie, it's as simple as taking the row corresponding to the user and the column corresponding to the movie, and multiplying them to get the predicted rating. To recommend movies to users, we recommend the movies that we predict, they will rate the highest. Of course, you will learn how to implement collaborative filtering in TensorFlow. One of the really cool things about collaborative filtering, is that you don't need to know any metadata about your items. You also don't need to do market segmentation of your users. As long as you have an interactions matrix, you're ready to go. But if you do have metadata about users and items, you can use content-based recommendation systems. But what if you have both? You have metadata and you have an interactions matrix. If you have both, you can use neural networks to combine all of the advantages, and eliminate the disadvantages of all three types of recommendation systems. Three types. Well, there's actually a third type of recommendation system called a knowledge-based recommendation system that can be used to provide business impact inputs to systems. We'll covered this in this course. The hybrid models of all three of these types of systems use all the data available and connect all of these models together into an ML pipeline. Incidentally, this is how YouTube works. We'll discuss some of the inner workings of the YouTube video hybrid recommendation system as an inspiration to building your own hybrid model. Finally, we will see how we can productionize and automate much of the necessary pipeline using the greater Google Cloud Platform ecosystem. Because, as we know, from core seven of the machine learning series, the machine learning model is only a small part of the overall system. Consequently, we'll look at the architecture of an end-to-end system for recommendations so that we can orchestrate the continuous retraining of the recommendation system as new ratings data comes in from users. The real-world example that we'll use is to use a week of data extracted from the web blogs of kurier.at, a large news provider in Austria. Recommendation systems are often the machine learning systems that you encounter the most often in enterprise settings. From a business impact standpoint, recommendation systems can allow you to sell products you could never sell before, to users you could never reach before. To learn how to build effective recommendation systems though, you will have to know all the topics that we have covered in all the courses so far. The data you have, may be structured data, might be images, might be text, and you will need to bring in all the data into a powerful machine learning model. We have quite a journey ahead of us. Let's get started.
