Joint modeling for learning decision-making dynamics in behavioral experiments

The Annals of Applied Statistics · 2025 · Volume 19 · Issue 4

Abstract

Major depressive disorder (MDD), a leading cause of disability and mortality, is associated with reward-processing abnormalities and concentration difficulties. Motivated by the probabilistic reward task from the Establishing Moderators and Biosignatures of Antidepressant Response in Clinical Care (EMBARC) study, we propose a novel framework that integrates the reinforcement learning (RL) model and drift-diffusion model (DDM) to jointly analyze reward-based decision making with response times. To account for emerging evidence suggesting that decision making may alternate between multiple interleaved strategies, we model latent state switching using a hidden Markov model (HMM). In the engaged state, decisions follow an RL-DDM, simultaneously capturing reward processing, decision dynamics and temporal structure. In contrast, in the lapsed state, decision making is modeled using a simplified DDM, where specific parameters are fixed to approximate random guessing with equal probability. The proposed method is implemented using a computationally efficient generalized expectation-maximization (EM) algorithm with forward-backward procedures. Through extensive numerical studies, we demonstrate that our proposed method outperforms competing approaches across various reward-generating distributions, under both strategy-switching and nonswitching scenarios, as well as in the presence of input perturbations. When applied to the EMBARC study, our framework reveals that MDD patients exhibit lower overall engagement than healthy controls and experience longer responses when they do engage. Additionally, we show that neuroimaging measures of brain activities are associated with decision-making characteristics in the engaged state but not in the lapsed state, providing evidence of a brain-behavior association specific to the engaged state.

Keywords

Brain-behavior association; Cognitive modeling; drift-diffusion models; mental health; reinforcement learning; state switching;

Enable JavaScript for citation tools and additional paper interactions.