LABERT: A Combination of Local Aggregation and Self-Supervised Speech Representation Learning for Detecting Informative Hidden Units in Low-Resource ASR Systems

Fatehi, Kavan; Kucukyilmaz, Ayse

Abstract

With advances in deep learning methodologies, Automatic Speech Recognition (ASR) systems have seen impressive results. However, ASR in Low-Resource Environments (LREs) are challenged by a lack of training data for the specific target domain. We propose that data sampling criteria for choosing more informative speech samples can be critical to addressing the problem of training data bottleneck. Our proposed Local Aggregation BERT (LABERT) method for self-supervised speech representation learning fuses an active learning model with an adapted local aggregation metric. Active learning is used to pick informative speech units, whereas the aggregation metric forces the model to move similar data together in the latent space while separating dissimilar instances to detect hidden units in LRE tasks. We evaluate LABERT with two LRE datasets: I-CUBE and UASpeech to explore the performance of our model in the LRE ASR problems.

Citation

Fatehi, K., & Kucukyilmaz, A. (2023). LABERT: A Combination of Local Aggregation and Self-Supervised Speech Representation Learning for Detecting Informative Hidden Units in Low-Resource ASR Systems. In Interspeech 2023

• Presentation Conference Type: Edited Proceedings
• Conference Name: Interspeech 2023
• Start Date: Aug 20, 2023
• End Date: Aug 24, 2023
• Acceptance Date: May 17, 2023
• Online Publication Date: Aug 21, 2023
• Publication Date: Aug 21, 2023
• Deposit Date: Jun 22, 2023
• Publicly Available Date: Aug 21, 2023
• Series Title: Interspeech Conference
• Series ISSN: 1990-9772
• Book Title: Interspeech 2023
• Keywords: Self-Supervised Learning; BERT; Local Aggre- gation Function; Low-Resource Environment ASR
• Public URL: https://nottingham-repository.worktribe.com/output/22183323
• Related Public URLs: https://www.isca-speech.org/archive/pdfs/interspeech_2023/fatehi23_interspeech.pdf