spark-knn-recommender
spark-knn-recommender copied to clipboard
findItemPairs returning only first item pair for each user
findItemPairs returning only first item pair for each user: def findItemPairs(user_id,items_with_rating): ''' For each user, find all item-item pairs combos. (i.e. items with the same user) ''' for item1,item2 in combinations(items_with_rating,2): return ((item1[0],item2[0]),(item1[1],item2[1]))
Solution: change return to yield to get the generator.
I using a other way to fix this bug. replace map with flatMap and return a list with all Pairs in findItemPairs
def findItemPairs(user_id, items_with_rating): list = [] for item1,item2 in combinations(items_with_rating,2): list.append(((item1[0],item2[0]),(item1[1],item2[1]))) return list
pairwise_items = user_item_pairs.filter( lambda p: len(p[1]) > 1).flatMap( lambda p: findItemPairs(p[0], p[1])).groupByKey()