A Mixed supervised Learning Framework for Target Sound Detection

Abstract: Target sound detection (TSD) aims to detect the target sound from mixture audio given the reference information. Previous works have shown that TSD models can be trained on fully-annotated (frame-level label) or weakly-annotated (clip-level label) data. However, there are some clear evidences show that the performance of the model trained on weakly-annotated data is worse than that trained on fully-annotated data. To fill this gap, we provide a mixed supervision perspective, in which learning novel categories (target domain) using weak annotations with the help of full annotations of existing base categories (source domain). To realize this, a mixed supervised learning framework is proposed, which contains two mutually-helping student models (\textit{f_student} and \textit{w_student}) that learn from fully-annotated and weakly-annotated data, respectively. The motivation is that \textit{f_student} learned from fully-annotated data has a better ability to capture detailed information than \textit{w_student}. Thus, we first let \textit{f_student} guide \textit{w_student} to learn the ability to capture details, so \textit{w_student} can perform better in the target domain. Then we let \textit{w_student} guide \textit{f_student} to fine-tune on the target domain. The process can be repeated several times so that the two students perform very well in the target domain. To evaluate our method, we built three TSD datasets based on UrbanSound and Audioset. Experimental results show that our methods offer about 8\% improvement in event-based F-score as compared with a recent baseline.