Online Allocation and Pricing: Constant Regret via Bellman Inequalities
Alberto Vera, Siddhartha Banerjee, Itai Gurvich
Operations Research · 2021 · Volume 69 · Issue 3
Abstract
We develop a framework for designing simple and efficient policies for a family of online allocation and pricing problems that includes online packing, budget-constrained probing, dynamic pricing, and online contextual bandits with knapsacks. In each case, we evaluate the performance of our policies in terms of their regret (i.e., additive gap) relative to an offline controller that is endowed with more information than the online controller. Our framework is based on Bellman inequalities, which decompose the loss of an algorithm into two distinct sources of error: (1) arising from computational tractability issues, and (2) arising from estimation/prediction of random trajectories. Balancing these errors guides the choice of benchmarks, and leads to policies that are both tractable and have strong performance guarantees. In particular, in all our examples, we demonstrate constant-regret policies that only require resolving a linear program in each period, followed by a simple greedy action-selection rule; thus, our policies are practical as well as provably near optimal.
Keywords
dynamic programming/optimal control; applications, Revenue Management and Market Analytics, stochastic optimization, approximate dynamic programming, online resource allocation, dynamic pricing, online packing, network revenue management
Enable JavaScript for citation tools and additional paper interactions.