Neural Networks and Applications

Constructing Biologically Constrained RNNs via Dale's Backprop and Topologically-Informed Pruning

The paper develops RNNs that incorporate biological constraints like Dale's law and structured connectivity to better model cortical function and validate insights through mouse calcium imaging data.

Bounds on the computational complexity of neurons due to dendritic morphology

This paper examines the computational limits of neurons with complex dendritic structures, showing that their capacity exceeds simple linear units and varies with morphological architecture.

Communication subspaces align with training in ANNs

This study examines if communication subspaces develop with training in ANNs and finds increased alignment between activations and weights in trained models.

Robustness in Hopfield neural networks with biased memory patterns

The paper studies how pattern bias affects robustness in Hopfield neural networks under different noise types and finds that higher pattern load favors more biased representations.

Competitive plasticity to reduce the energetic costs of learning

This paper proposes energy-saving plasticity algorithms for neural networks, showing they reduce synaptic updates while maintaining learning performance.

Brain-like learning with exponentiated gradients

This paper introduces exponentiated gradient (EG) as a learning algorithm for artificial neural networks that better aligns with biological principles than gradient descent.

Equivalence between representational similarity analysis, centered kernel alignment, and canonical correlations analysis

This paper shows that representational similarity analysis and centered kernel alignment are equivalent when mean-centering is applied to RSA, unifying two methods for comparing neural representations.

When and why modular representations emerge

This study shows that modular representations in neural networks emerge for complex behavior when inputs are low-dimensional, without anatomical constraints, and supports rapid learning and generalization.

The underlying mechanisms of alignment in error back-propagation through arbitrary weights

This paper investigates the mechanisms of weight alignment in BP-TAW, showing it is governed by statistical properties of neural signals and can be improved with weight normalization.

Flexible multitask computation in recurrent networks utilizes shared dynamical motifs

This paper identifies dynamical motifs in recurrent neural networks that enable flexible multitask computation through modular reuse across different tasks.

Contribute to balance, wire in accordance: Emergence of backpropagation from a simple, bio-plausible neuroplasticity rule

This paper proposes a neuroplasticity rule that mathematically replicates backpropagation in neural networks without requiring biologically implausible assumptions.

Transition to chaos separates learning regimes and relates to measure of consciousness in recurrent neural networks

The paper shows that chaotic dynamics in recurrent neural networks separate learning regimes and correlate with a consciousness measure, suggesting a link between critical dynamics, learning, and consciousness.

Scaling Properties for Artificial Neural Network Models of a Small Nervous System

This study examines how data and model size affect the accuracy of ANN models predicting neural activity in C. elegans, finding log-linear data scaling and nonlinear model scaling.

Probing the Structure and Functional Properties of the Dropout-induced Correlated Variability in Convolutional Neural Networks

This study examines the structure of trial-by-trial neural co-variance in dropout neural networks and finds it similar to biological neural variability, suggesting potential implications for network performance and modeling.

Unsupervised Learning of Perceptual Feature Combinations

This paper introduces an unsupervised learning mechanism that enables neurons to specialize in feature combinations, showing convergence to specific representations through a correlation-based learning rule.

Attractor neural networks with double well synapses

This paper introduces a neural network model with double-well synapses, showing it has a power law storage capacity scaling and improved robustness to noise compared to single-well synapse models.

Structured flexibility in recurrent neural networks via neuromodulation

This paper introduces a neuromodulated RNN model that uses synaptic gain scaling to improve training and generalization on canonical tasks, and shows how it implements gating mechanisms found in artificial RNNs.

AI without networks

This paper introduces a network-free AI framework based on data interpolation, demonstrating its effectiveness in generative modeling across ethology, control theory, and mathematics.

Iterative convergent computation may not be a useful inductive bias for residual neural networks

This paper investigates whether iterative convergent computation improves ResNets but finds it does not enhance classification accuracy on standard or challenging visual tasks.

The effective number of shared dimensions: A simple method for revealing shared structure between datasets

The paper introduces a model-free measure called effective number of shared dimensions (ENSD) to quantify shared structure between datasets, demonstrating its utility across neural networks and neuroscience data.

The Flip-flop neuron - A memory efficient alternative for solving challenging sequence processing and decision making problems

The paper introduces a JK flip-flop neuron model for sequence processing tasks, showing it outperforms LSTMs with fewer parameters.

ElegansNet: a brief scientific report and initial experiments.

ElegansNet uses the C. elegans connectome to design deep learning systems that outperform random and artificial networks in image classification and digit reconstruction tasks.

Training biologically plausible recurrent neural networks on cognitive tasks with long-term dependencies

This paper trains biologically plausible RNNs on cognitive tasks with long-term dependencies using specialized skip-connections, achieving faster training and better performance than conventional methods.

A reduced-memory multi-layer perceptron withsystematic network weights generated and trainedthrough distribution hyper-parameters

This paper introduces a reduced-memory MLP framework that uses distribution-based weights and genetic algorithm-trained hyperparameters to improve efficiency and performance.

Cognitive Model Discovery via Disentangled RNNs

This paper learns cognitive models from behavioral data using disentangled RNNs, recovering interpretable models from synthetic and rat behavioral data.

Privileged representational axes in biological and artificial neural networks

The paper identifies privileged representational axes in both biological and artificial neural networks, showing alignment between DCNNs and perceptual cortices and benefits of axis-sensitive coding.

Efficient neural codes naturally emerge through gradient descent learning

The paper shows that gradient descent learning in neural networks creates representations sensitive to common image features, mirroring efficient coding principles observed in animal sensory systems.

Sparse RNNs can support high-capacity classification

This paper demonstrates that sparse RNNs can perform high-capacity classification through distributed computation and resonance amplification of inputs.

Biologically plausible unsupervised learning in neural networks with sparse and asymmetric connectivity

The paper develops biologically plausible variants of RBMs with sparse and asymmetric connectivity and benchmarks their performance on MNIST.

Training stochastic stabilized supralinear networks by dynamics-neutral growth

This paper trains a biologically realistic neural network model, the stabilized supralinear network, on challenging tasks using a dynamics-neutral growth method during training.