Intact early visual representations, not phosphene-adapted features, account for human perceptual behavior with retinal prostheses

Abstract

AO_SCPLOWBSTRACTC_SCPLOWEfforts to restore vision via neural implants have outpaced the ability to predict what users will perceive, leaving patients and clinicians without reliable tools for surgical planning or device selection. To bridge this critical gap, we introduce a computational virtual patient (CVP) pipeline that integrates anatomically grounded phosphene simulation with task-optimized deep neural networks (DNNs) to forecast patient perceptual capabilities across diverse prosthetic designs and tasks. We evaluate performance across six visual tasks, six electrode configurations, and two artificial vision models, establishing our CVP approach as a scalable pre-implantation method. Several chosen tasks align with the Functional Low-Vision Observer Rated Assessment (FLORA), revealing correspondence between model-predicted difficulty and real-world patient outcomes. Further, the CVP paradigm exhibited strong correspondence with psychophysical data collected from normally sighted subjects viewing phosphene simulations, capturing both overall task difficulty and performance variation across implant configurations. Comparing frozen-feature linear probing with full end-to-end fine-tuning reveals that preserving natural-image representations--rather than adapting them to phosphene-specific statistics--better reproduces human perceptual behavior, consistent with the constrained plasticity of adult visual cortex. The findings position CVP as a scientific tool for probing perception under prosthetic vision, an engine to inform device development, and a clinically relevant framework for pre-surgical forecasting.