Speech and Audio

This AI system generates high-quality multi-track music arrangements from simple lead sheets using a novel style prior modeling approach, significantly improving both efficiency and musical coherence.

Energy-efficient neuromorphic SSL model achieves state-of-the-art accuracy and robustness using Resonate-and-Fire neurons and a novel multi-auditory attention module.

SongCreator: a novel AI system generates complete, high-quality songs from lyrics, surpassing existing methods in lyrics-to-song and lyrics-to-vocals generation.

SLIM: A novel audio deepfake detection model leverages style-linguistics mismatch for superior generalization and explainability.

SepReformer: Asymmetric encoder-decoder model for efficient speech separation, achieving state-of-the-art performance with less computation.

SCOREQ: a novel triplet loss contrastive regression approach for superior speech quality prediction, addressing generalization issues in no-reference metrics.

FINALLY achieves studio-like speech enhancement speed and quality using a novel GAN-based approach with WavLM-integrated perceptual loss, outperforming existing diffusion models.

CA-SSLR: a novel self-supervised learning model dynamically adapts to various speech tasks by integrating language and speaker embeddings, improving performance and reducing reliance on audio features…

Annealed Multiple Choice Learning (aMCL) overcomes limitations of Winner-takes-all in multiple choice learning by using annealing, improving robustness and performance.

Acoustic Volume Rendering (AVR) revolutionizes realistic audio synthesis by adapting volume rendering to model acoustic impulse responses, achieving state-of-the-art performance in novel pose synthesi…