The Power of Bidirectional Estimators: Personalized search, Markov Chain estimation and beyond

Speaker: Siddhartha Banerjee
School of Operations Research and Information Engineering (ORIE) at Cornell

Venue: Packard 101
Time: 4:15 pm to 5 pm
Date: Thursday, February 25, 2016

Abstract

fundamental problem in Markov chains is of estimating the probability of transitioning from a given starting state to a given terminal state in a fixed number of steps. This has received much attention in recent years as Markov chains form the basis of many graph centrality measures, in particular, PageRank and Personalized PageRank (PPR). Standard approaches to this problem use either linear-algebraic iterative techniques (such as the power iteration) or Monte Carlo - both however have a running time which scales linearly in the size of the network. This is too slow for real-time computation on large networks - consequently, PPR, which has long been recognized as an effective measure for ranking search results, is rarely used in practice. I’ll present a new approach towards designing bidirectional estimators, which combines linear algebraic and random walk techniques. Our approach provides the first algorithm for PageRank estimation which has sublinear running-time guarantees in theory, and which is much faster than existing algorithms in practice. In particular, we show that it returns estimates with additive error in time in undirected networks, and in sparse directed networks. Moreover, our approach extends to general Markov chains – this may have applications in many diverse settings, and I look forward to some suggestions from the audience! This is joint work with Peter Lofgren and Ashish Goel.

Speaker Bio

Sid Banerjee is an assistant professor in the School of Operations Research and Information Engineering (ORIE) at Cornell, where he works on stochastic modeling, and the design of algorithms and incentives for large-scale systems. He received his PhD in 2013 from the ECE Department at UT Austin, and was a postdoctoral researcher in the Social Algorithms Lab at Stanford from 2013 to 2015. He was a technical consultant at Lyft in 2014, and has previously interned at the Technicolor Paris Research Lab and AlcatelLucent Bell Labs. His current research focusses on the design of scalable algorithms for large networks, online marketplaces and social-computing platforms.