Nonsense correlations in neuroscience
Abstract
Many neurophysiological signals exhibit slow continuous trends over time. Because standard correlation analyses assume that all samples are independent, they can yield apparently significant "nonsense correlations" even for signals that are completely unrelated. Here we compare the performance of several methods for assessing correlations between timeseries, using simulated slowly drifting signals with and without genuine correlations. The best performance was obtained from a "pseudosession method", which relies on one of the signals being randomly generated by the experimenter, or a "session perturbation" method which requires multiple recordings under the same conditions. If neither of these is applicable, a "linear shift" method can be used when one of the signals is stationary. Methods based on cross-validation, circular shifting, phase randomization, or detrending gave up to 100% false positive rates in our simulations. We conclude that analysis of neural timeseries is best performed when stationarity and randomization is built into the experimental design.
02 /Specification & Classification
Click any tag to browse matching papers- Citations
- 86
- Assay
- Research Focus
- Approach
In-depth summary
Claude Opus 5.5 · xhigh effort · 8 Nov 2026 · from the full text of version 3 (19 Jun 2021) · CC BY 4.0
With slowly drifting simulated signals, pseudosession, session permutation and (if one series is stationary) linear shift tests stayed calibrated; circular shift, phase randomization, cross-validation and auto-decorrelation did not.
Neural and behavioral variables often drift slowly over a session, so tests that treat each timepoint as independent can declare unrelated signals significantly correlated; the paper notes the issue is widely discussed in econometrics but rarely addressed in neuroscience. It defines correlation between time series as dependence of whole histories across repeats of an experiment, and uses that definition to compare ten candidate tests on simulated data.
Key results
ShownOrdinary multiple linear regression with an F-test declared slowly drifting neural rates significantly predictive of behavior in every simulation, including when the two were generated independently.
With C = 10 simulated cells and T = 200 trials, the F-test was significant in all four scenarios (binary or continuous behavior, with or without genuine coupling) for each of 1000 simulations, so it gave a false positive every time the null was true. · What are nonsense correlations?; Figure legends
SuggestedTwo tests gave calibrated null p-values and detected genuine coupling in all four scenarios: pseudosession, which resamples the experimenter-randomized variable from its known distribution, and session permutation, which scores each session's neural data against other sessions' behavior.
Across 1000 simulations per scenario, p-values were flat when there was no true correlation and piled up near zero when coupling was present. Pseudosession needs only one session, and any association statistic gives a valid test; session permutation needs at least 5 sessions (5! = 120 permutations) but not the same neurons in each session. · Pseudosession method; Session permutation method; Figure legends
SuggestedThe conservative linear shift test compares the unshifted association with shifted ones on a central segment. When one series is stationary, it kept false-positive rates under the 0.05 level and still rejected the null in most coupled simulations. Its approximate variant added little.
With N = 19 and no coupling, the conservative test rejected in 2.7% (block) and 2.9% (continuous) of simulations and the approximate test in 4.2% and 5.1%. With coupling, both rejected in 100% of block runs and 89% and 91% of continuous runs. Shifting the nonstationary block variable instead gave 1.9% and 3.0% false rejections. · Linear shift method; Figure legends
ShownCross-validation does not protect against nonsense correlations: slow drifts let a model fitted on one stretch of a session generalize to another. Forecasting folds helped only for the block variable.
With 10-fold cross-validation on random trials, held-out predictions beat the training-mean baseline in 100% of simulations even for unrelated signals. Contiguous folds still gave false positives in 54% (block) and 92% (continuous) of simulations. Forecasting from earlier blocks gave 1% (block) but 61% (continuous). · Cross validation; Figure legends
ShownCircular shifting, Fourier phase and wavelet randomization, and AR(1) (first-order autoregressive) auto-decorrelation before an F-test all inflated false positives. For all but wavelet, the text blames mishandled slow structure (wrap-around jumps, imposed cyclic continuity, leftover drift).
In 1000 simulations, the three surrogate nulls gave more false positives than linear shift (wavelet fewer than Fourier); after AR(1), trends persisted and F-tests stayed inflated. The abstract cites up to 100% false positives for methods based on cross-validation, circular shifting, phase randomization or detrending; the body reports no surrogate rates, and its only 100% is for random-fold cross-validation. · Abstract; Phase/wavelet randomization; Auto-decorrelation; Figure legends
Methods, data and reproducibility
Defined correlation between time series as dependence of whole histories across experiment repeats. Simulated drifting firing rates for 10 neurons over 200 trials with a binary block or continuous behavioral variable, with or without weak coupling, then applied ten tests (linear-regression prediction error as the association measure) over 1000 simulations per scenario.
- Simulated firing rates (C = 10 cells, T = 200 trials; logistic sigmoid drifts plus pink noise) paired with a binary block or continuous behavioral variable, with or without weak coupling · n = 1000 simulations per scenario, four scenarios
- Methods & tools
- multiple linear regression with F-test, Fourier phase randomization, Daubechies(4) wavelet transform, first-order autoregressive model, 10-fold cross-validation, Google Colab notebook (nonsense.ipynb)
- Code
- https://github.com/kdharris101/nonsense-correlations/https://colab.research.google.com/github/kdharris101/nonsense-correlations/blob/main/nonsense.ipynb
- Reproducibility gaps
- No software versions or environment file are stated. All data are simulated, so nothing is deposited beyond the simulation code.
Limitations (6)
- The pseudosession method requires one series to be randomly generated by the experimenter from a known distribution, so it cannot be applied to uncontrolled variables such as choices or running speed. · stated by the authors
- Session permutation needs at least 5 sessions under identical conditions and cannot establish causality; it can pick up shared trends across sessions (e.g. recording quality and performance degrading together, or subject differences). With different neurons per session, only population-level conclusions hold. · stated by the authors
- The conservative linear shift test assumes one series is stationary and accepts a twofold loss in power. The approximate test and shifting a nonstationary series behaved well here, but the author notes this is not guaranteed in general. · stated by the authors
- For partial correlation with a confounder (e.g. block versus choice), neither pseudosession nor session permutation applies. The proposed alternative is approximate, needs multiple experiments and assumes near-symmetric statistic distributions. · stated by the authors
- All results come from one simulation generator (sigmoid steps plus pink noise, 10 cells, 200 trials, linear-regression error as the statistic). No recorded neural data are analysed, so false-positive rates on real recordings are not measured. · noted in this summary
- The partial-correlation test is only described. It is not evaluated in simulation in this paper. · noted in this summary
Who should read:Read this if you correlate neural recordings with behavior across trials and need a significance test that will not report slow drifts in unrelated signals as real effects.
Reproducibility ledger
Nobody has reported trying to reproduce this paper yet. Did you replicate an experiment or run the code? Your report saves the next lab a week.
Sign in to file a reproducibility report.
Related papers
- A new family of statistical tests for neuronal spiking and autocorrelated timeseries data
Jorrit Steven Montijn, Guido T. Meijer, J. Alexander Heimel·bioRxiv·Oct 30, 2023
- Estimating mutual information and Pearson correlation on neural evoked responses
Anni Hukari, Silvia Federica Cotroneo, Riitta H. Salmelin·NeuroinformaticsPeer-reviewed·Sep 16, 2026
- The unbiased estimation of r2 between two sets of noisy neural responses
Dean A. Pospisil, Wyeth Bair·bioRxiv·Mar 29, 2021
- Conservative Significance Testing of Tripartite Interactions in Multivariate Neural Data
Aleksejs Fomins, Yaroslav Sych, Fritjof Helmchen·Network NeurosciencePeer-reviewed·Jan 1, 2022
- Third-order motifs are sufficient to fully anduniquely characterize spatiotemporal neuralnetwork activity
Sarita S. Deshpande, Graham A. Smith, Wim van Drongelen·Scientific ReportsPeer-reviewed·Jan 5, 2023
- Rhythmic variations in prefrontal inter-neuronal correlations, their underlying mechanisms and their behavioral correlates
Suliann Ben Hamed, Corentin Gaillard, … Claire Wardak·bioRxiv·Sep 27, 2019