A New Approach to Optimal Design under Model Uncertainty Motivated by Multi-Armed Bandits
Mingyao Ai, Holger Dette, Zhengfu Liu, Jun Yu
Journal of the American Statistical Association · 2025 · Volume 120 · Issue 552
DOI: 10.1080/01621459.2025.2487223
Abstract
An optimal design is usually model-dependent and is sub-optimal if the postulated model is not correctly specified. Furthermore, it is far from ideal even if it is efficient for model selection but has a poor performance for estimating parameters in the selected model. In practice, it is common that a researcher has a list of candidate models at hand and a design has to be found that is efficient for both model discrimination and parameter estimation in the (unknown) “true” model. In this article, we use a multi-armed bandits approach to balance these two competing goals in the design of experiments. We develop a sequential algorithm to provide a design that has asymptotically the same performance as an optimal design when the “true” model could be correctly specified in advance. A lower bound is established to quantify the relative efficiency between the proposed design and an optimal design for the “true” model. Some comparisons with other state-of-the-art algorithms for model discrimination and parameter estimation are discussed. The advantages of the proposed method are illustrated by several numerical examples. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.
Keywords
Model discrimination, Optimal design, Reinforcement learning, Sequential design, Thompson sampling
Enable JavaScript for citation tools and additional paper interactions.