Are We Really Making Much Progress? A Worrying Analysis of Recent Neural Recommendation Approaches
by Maurizio Ferrari Dacrema, Paolo Cremonesi, Dietmar Jannach · ACM RecSys 2019 / arXiv
RecSys 2019 best paper that tried to reproduce 18 neural top-n recommendation methods and found most could not be reproduced or were beaten by tuned nearest-neighbour and simple baselines, exposing weak baselines and evaluation flaws. Covers the topic's evaluation pillar: teaches why offline comparisons mislead and why strong baselines matter. Essential corrective after reading about deep models.
More resources on Recommender Systems
How Algorithmic Confounding in Recommendation Systems Increases Homogeneity and Decreases Utility
RecSys 2018 paper using simulation to show how retraining recommenders on data their own recommendations shaped creates feedback loops that homogenise user behaviour, reduce utility, and bias offline evaluation.
System Design for Recommendations and Search
The best free bridge from textbook models to how real systems are wired. Synthesis of primary industry sources with links, not a rehash. Essay comparing how Alibaba, Facebook, JD, DoorDash, LinkedIn and others structure recommendation and search systems into offline and online components and retrieval and ranking stages, with links to each company's primary write-up.
Deep Neural Networks for YouTube Recommendations
Primary source for deep recommenders at scale. RecSys 2016 paper describing YouTube's two-stage deep recommender: a candidate generation network trained as extreme multiclass classification and a separate ranking network, plus practical lessons on features, example age and watch-time objectives.
Matrix Factorization Techniques for Recommender Systems
The primary source for matrix factorization as practised in industry; short (8 pages) and readable. The 2009 IEEE Computer article by members of the Netflix Prize-winning team explaining latent factor models: SGD and ALS training, user and item biases, implicit feedback, temporal dynamics and confidence weighting, with results on Netflix data.
Recommender Systems: The Textbook
The single reference text for the field: comprehensive graduate textbook covering neighborhood and model-based collaborative filtering, latent factor models, content-based, knowledge-based and hybrid systems, evaluation methodology, context-aware and time-sensitive recommendation, and attack-resistant and social recommenders.
Mining of Massive Datasets, Chapter 9: Recommendation Systems
Free chapter from the Stanford CS246 textbook covering the utility matrix, content-based recommendation, user-user and item-item collaborative filtering, UV-decomposition by gradient descent, and a case study of the Netflix Prize. Compact, mathematically clean treatment of the classical toolkit (CF, UV-decomposition) with exercises.