Speech and Audio Processing

Vib2Sound: Separation of Multimodal Sound Sources

This paper introduces a neural network system for separating overlapping vocalizations using multi-channel microphone and accelerometer data from zebra finches.

Neural Tracking of Audiovisual Effects in Noise Using Deep Neural Network-Generated Virtual Humans

This study evaluates DNN-generated virtual humans for enhancing audiovisual speech perception in noise using behavioral and neural measures, finding they improve speech intelligibility compared to human speakers.

A brain-inspired algorithm improves cocktail party listening for individuals with hearing loss

A biologically inspired algorithm improves speech intelligibility in noisy environments for people with hearing loss, outperforming standard beamforming approaches.

Adversarial Learning For End-To-End Cochlear Speech Denoising Using Lightweight Deep Learning Models

This paper improves cochlear implant speech denoising by relocating a deep envelope detector and adding a discriminator network, enhancing performance and reducing parameters for real-time use.

Decoding the Unintelligible: Neural Speech Tracking in Low Signal-to-Noise Ratios

This study shows that neural speech tracking remains reliable in low signal-to-noise ratios, suggesting the brain retains sufficient information for decoding attended speech.

Brain-controlled augmented hearing for spatially moving conversations in multi-talker environments

This paper develops a brain-controlled assistive hearing system for multi-talker environments with moving conversations, improving speech intelligibility and tracking.

Deep Learning Restores Speech Intelligibility in Multi-Talker Interference for Cochlear Implant Users

This study applies deep learning models to improve speech intelligibility for cochlear implant users in multi-talker noise, showing significant improvements over conventional methods.

The Effect on Speech-in-Noise Perception of Real Faces and Synthetic Faces Generated with either Deep Neural Networks or the Facial Action Coding System

A biologically orientated algorithm for spatial sound segregation

This paper introduces a biologically inspired algorithm for spatial sound segregation that mimics auditory processing and achieves performance comparable to multi-microphone systems without requiring training data.

Auditory spatial analysis in reverberant audio-visual multi-talker environments with congruent and incongruent visual room information

The paper investigates how listeners analyze audio-visual scenes with varying talker numbers and reverberation, finding that reverberation and talker count affect accuracy and response time, while incongruent visual room information does not.

Algorithms for Estimating Time-Locked Neural Response Components in Cortical Processing of Continuous Speech

This paper compares and evaluates methods for estimating time-locked neural response components in speech processing using simulated and real MEG data.

Speech-driven Facial Animations Improve Speech-in-Noise Comprehension of Humans

This study shows that synthesized facial animations from speech improve speech comprehension in noise, though natural facial motions provide a greater benefit.

Speech-In-Noise Comprehension is Improved When Viewing a Deep-Neural-Network-Generated Talking Face

This study shows that using a deep neural network-generated talking face improves speech comprehension in noise, performing better than audio-only but less than natural video.

Speech intelligibility with various head-related transfer functions: A computational modeling approach

The paper uses an auditory model to predict speech intelligibility with different HRTFs, finding that intelligibility varies across HRTFs, suggesting individual differences in access to speech cues.

Experiments on localization accuracy with non-individual and individual HRTFs comparing static and dynamic reproduction methods

This study compares localization accuracy using individual and non-individual HRTFs with static and dynamic reproduction methods, finding dynamic methods reduce front-back confusions more effectively than individual HRTFs.

An Interpretable Performance Metric for Auditory Attention Decoding Algorithms in a Context of Neuro-Steered Gain Control

The paper introduces an interpretable performance metric for auditory attention decoding algorithms to optimize gain control in hearing aids by resolving the trade-off between accuracy and decision time.

MobiLab - A Mobile Laboratory for On-Site Listening Experiments in Virtual Acoustic Environments

The paper describes the design and evaluation of a mobile hearing laboratory for conducting on-site listening experiments in virtual acoustic environments to address logistical challenges in studying noise effects on vulnerable populations.

Impact of Different Acoustic Components on EEG-based Auditory Attention Decoding in Noisy and Reverberant Conditions

This paper evaluates an EEG-based method for decoding auditory attention in various acoustic conditions and finds it performs well even with degraded reference and training signals.