I did it my way

Standard reinforcement learning (RL) models often focus on finding a single optimal strategy, but in the real world, no two animals or people would solve a problem in the exact same way. RL treats behavioral variation as unwanted noise rather than a core feature of natural life. In their newest article, the Collective Information Processing Group (Pawel Romanczuk) criticises this standard practice. They argue that it overlooks the individual differences and behavioral diversity essential to biological systems. To bridge the gap between computational models and biological reality, they propose a framework that generates behavioral heterogeneity through biologically plausible mechanisms across three main dimensions: Expanding decision spaces, incorporating sequential temporal dynamics and embedding individuals within populations. They conclude that RL is not just an optimization tool, but a mechanistic framework for understanding how individual variation naturally emerges and stabilizes in adaptive biological populations.

Abstract

Reinforcement learning (RL) is primarily known as a computational method for optimizing control tasks, but it is increasingly used to explain biological behavior. While RL successfully captures key aspects of biology, a major gap remains: between-agent behavioral variability. Consistent individual differences naturally permeate biological populations, yet RL models typically present only the single best individual or the population average. Addressing this gap requires moving beyond current practices to generate behavioral diversity using biologically plausible mechanisms. Here, we examine approaches from various subfields of RL and outline potential paths forward to close the gap between biology and simulation.