Guillaume Allain gave an interesting talk at the recent PyData London 2017 event. It was about the implementation of recommender systems using TensorFlow. The talk is shared in the YouTube video below.
I recommend the reader to also fork the GitHub pull request/repository Tensorflow-based Recommendation systems, where a detailed description of this developement is available as well as all the code base:
Tensorflow-based Recommendation systems
Factorization models are very popular in recommendation systems because they can be used to discover latent features underlying the interactions between two different kinds of entities. There are many variations of factorization algorithms (SVD, SVD++, factorization machine, …). When implementing them or developing new ones, you probably spend a lot of time on the following areas rather than modeling:
- Derivative calculation
- Variant SGD algorithm exploration
- Multi-thread acceleration
- Vectorization acceleration
Tensorflow is a general computation framework using data flow graphs although deep learning is the most important application of it. With Tensorflow, derivative calculation can be done by auto differentiation, which means that you only need to write the inference part. It provides variant fancy SGD learning algorithms, CPU/GPU acceleration, and distributed training in a computer cluster. Since Tensorflow has some embedding modules for word2vec-like application, it is supposed to be a good platform for factorization models as well, even in production. Please note that embedding in deep learning is equivalent to factorization in shallow learning!
This talk will demonstrate how to harness a deep-learning framework such as Tensorflow, together with the usual suspects such as Pandas and Numpy, to implement recommendation models for news and classified ads.
Recommender systems are used across the digital industry to model users’ preferences and increase engagement. Popularised by the seminal Netflix prize, collaborative filtering techniques such as matrix factorisation are still widely used, with modern variants using a mix of meta-data and interaction data in order to deal with new users and items. We will demonstrate how to implement a variety of models using Tensorflow, from simple bi-linear models expressed as shallow neural nets to the latest deep incarnations of Amazon DSSTNE and Youtube neural networks. We will also use TensorBoard and particularly the embedding projector to visualise the latent space for items and metadata.
The final part of the talk was an important question and answer by Guillaume, where he answered some pertinent questions. The comparison between TensorFlow and the more Pythonesque PyTorch was highlighted on several occasions, with the speaker finally giving his own opinions, regarding TensorFlow a more robust tool set for the kinds of compute workloads in distributed computing for recommenders systems with matrix factorization, but the code declaration is static whereas PyTorch is dynamically compiled providing a much more flexible code manipulation between Python software ecosystems.
featured image: Guillaume Allain – Recommender systems with Tensorflow