To Remember or Not to Remember

Does a system really have a memory—or are we just being fooled by noisy data? Many physical and biological processes are described using so-called Markov models, where the future depends only on the present and not on the past. Detecting when a system truly deviates from this "memoryless" behavior is therefore a fundamental challenge. The Theory of Complex Systems and Neurophysics Group (Benjamin Lindner) and the Statistical Physics and Nonlinear Dynamics Group (Igor Sokolov) examined a recently proposed test for identifying memory effects and asked a practical question: when does an apparent signal of memory reflect the system itself, and when is it simply the result of limited experimental data? Using a simple model system, they showed that the reliability of the test depends critically on the strength of the applied perturbation. Surprisingly, neither very weak nor very strong perturbations perform best. Instead, the most reliable results are obtained when the induced response is roughly the same size as the system's natural fluctuations. Curious how scientists test whether a system remembers its past? Explore their Physical Review E Article!

Abstract

Markov processes are widely applied in modeling random phenomena, and the corresponding models can be analyzed within an established theoretical framework. It is thus desirable to have reliable tests for the (non-)Markovianity of a physical or biological system at hand. One such test, suggested by Engbring et al. [Phys. Rev. X 13, 021034 (2023)], is based on a nonlinear fluctuation-dissipation relation (NL-FDR) that compares spontaneous fluctuations with the response to (nonweak) external perturbations. A violation of this NL-FDR indicates non-Markovianity. However, estimating the relevant statistics for the variable conjugated to the perturbation from finite experimental data can be challenging. Comparing noisy estimates of the spontaneous and the response statistics can lead to apparent deviations from the prediction of the NL-FDR. So, when is a violation of the NL-FDR due to non-Markovian features of the system and when is it due to a finite sample size? We consider this question for the simplest Markovian example, the overdamped harmonic oscillator with thermal white noise and subject to a constant force perturbation. We derive analytical expressions for the variances of the estimators for the spontaneous correlation function and for the time-dependent mean response. We confirm previous findings that there is an optimal amplitude for the perturbation at which the test of non-Markovianity is most reliable. In terms of the original variable, the optimal amplitude is such that the response to the perturbation is of the order of one standard deviation of the spontaneous fluctuations. Put differently, the optimal amplitude neither corresponds to the linear-response regime of very weak perturbations nor to the regime of very strong perturbations where the response is largely deterministic. At large amplitudes, we encounter heavy-tailed distributions of certain estimators, which we discuss.