Fitting a deeply nested hierarchical model to a large book review dataset using a moment-based estimator
Ningshan Zhang, Kyle Schmaus, Patrick O. Perry
The Annals of Applied Statistics · 2019 · Volume 13 · Issue 4
Abstract
We consider a particular instance of a common problem in recommender systems, using a database of book reviews to inform user-targeted recommendations. In our dataset, books are categorized into genres and subgenres. To exploit this nested taxonomy, we use a hierarchical model that enables information pooling across across similar items at many levels within the genre hierarchy. The main challenge in deploying this model is computational. The data sizes are large and fitting the model at scale using off-the-shelf maximum likelihood procedures is prohibitive. To get around this computational bottleneck, we extend a moment-based fitting procedure proposed for fitting single-level hierarchical models to the general case of arbitrarily deep hierarchies. This extension is an order of magnitude faster than standard maximum likelihood procedures. The fitting method can be deployed beyond recommender systems to general contexts with deeply nested hierarchical generalized linear mixed models.
Keywords
Generalized linear mixed model, hierarchical model, method of moments, recommender system
Enable JavaScript for citation tools and additional paper interactions.