Humans perceive the world through various dimensions, relying on fundamental sensory modalities like audition and vision. Beyond these foundational senses, we interact with higher-level, abstract forms of expression, such as natural language and artistic creations. My research is dedicated to establishing connections between these diverse perceptual dimensions, with the ultimate goal of creating a unified representation for machine modeling. Specifically, my research centers on the modeling of music understanding. Using the conceptual framework mentioned earlier, we can map the intricate landscape of music into the central representation. This process enables us to translate musical data into various arbitrary dimensions or modalities, resulting in a comprehensive understanding of music. Currently, I am working on content-based music audio generation, trying to translate multiple modalities into music.
Publications
- Liwei Lin, Gus Xia, Junyan Jiang, Yixiao Zhang. "Content-based Controls For Music Large Language Modeling." https://arxiv.org/abs/2310.17162.
- Liwei Lin, Gus Xia, Junyan Jiang, Yixiao Zhang. "A Unified Model for Zero-shot Music Source Separation, Transcription and Synthesis." (ISMIR 2021) the 22nd International Society for Music Information Retrieval Conference.
- Liwei Lin, Qiuqiang kong, Junyan Jiang, Gus Xia. "Guided learning for weakly-labeled semi-supervised sound event detection." ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
- Liwei Lin, Xiangdong Wang, Hong Liu, Yueliang Qian. "Specialized decision surface and disentangled feature for weakly-supervised polyphonic sound event detection." IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Liwei Lin, Xiangdong Wang, Hong Liu, Yueliang Qian. "Guided learning convolution system for dcase 2019 task 4." Proceedings of the Detection and Classification of Acoustic Scenes and Events 2019 Workshop (DCASE2019)
- Computer music
- Computer audition
- Machine learning