Effect of Valence & Uncertainty on Reinforcement Learning

Abstract

In prior work, a bias towards negative valence had been observed in humans and animals. This bias was explained through attention, memory, and re-sponse biases. Previous studies also indicated that uncertainty could interact with these behavioral biases, but the underlying mechanisms were not well understood. In this study, we approached this multifaceted problem by con-sidering how valence, levels of expected uncertainty, and traits modulate the output bias using a modified version of the cue association learning task. We used a newly derived axis and reinforcement learning drift-diffusion model (RL-DDM) to assess the bias in behavioral data. Data collected includes 70 participants(40 in 70% and 30 in 80% correct feedback blocks) . Supporting our hypothesis, we found that 1) uncertainty modulated the negative bias and 2) this modulatory effect was captured in the prior bias parameter of the RL-DDM( p< 0.05 for 70% but not in 80% correct feedback blocks). Additionally, we found that the bias coded in a new normalized variable was explained by trait variables(STAI)(β=-0.03, p=0.02), indicating that the bias is not invariant and depends on the population. This provided a more generalizable approach for addressing such biases and their potential implications in clinical studies.