1 code implementation • 28 May 2024 • Yixiao Zhang, Yukara Ikemiya, Woosung Choi, Naoki Murata, Marco A. Martínez-Ramírez, Liwei Lin, Gus Xia, Wei-Hsiang Liao, Yuki Mitsufuji, Simon Dixon
Recent advances in text-to-music editing, which employ text queries to modify music (e. g.\ by changing its style or adjusting instrumental components), present unique challenges and opportunities for AI-assisted music creation.
no code implementations • 19 May 2024 • Daniel Chin, Yuxuan Wang, Gus Xia
Large Language Model (LLM) -in-the-loop applications have been shown to effectively interpret the human user's commands, make plans, and operate external tools/systems accordingly.
2 code implementations • 16 May 2024 • Ziyu Wang, Lejun Min, Gus Xia
A cascaded diffusion model is trained to model the hierarchical language, where each level is conditioned on its upper levels.
no code implementations • 9 Apr 2024 • Xingwei Qu, Yuelin Bai, Yinghao Ma, Ziya Zhou, Ka Man Lo, Jiaheng Liu, Ruibin Yuan, Lejun Min, Xueling Liu, Tianyu Zhang, Xinrun Du, Shuyue Guo, Yiming Liang, Yizhi Li, Shangda Wu, Junting Zhou, Tianyu Zheng, Ziyang Ma, Fengze Han, Wei Xue, Gus Xia, Emmanouil Benetos, Xiang Yue, Chenghua Lin, Xu Tan, Stephen W. Huang, Wenhu Chen, Jie Fu, Ge Zhang
In this paper, we explore the application of Large Language Models (LLMs) to the pre-training of music.
1 code implementation • 25 Feb 2024 • Ruibin Yuan, Hanfeng Lin, Yi Wang, Zeyue Tian, Shangda Wu, Tianhao Shen, Ge Zhang, Yuhang Wu, Cong Liu, Ziya Zhou, Ziyang Ma, Liumeng Xue, Ziyu Wang, Qin Liu, Tianyu Zheng, Yizhi Li, Yinghao Ma, Yiming Liang, Xiaowei Chi, Ruibo Liu, Zili Wang, Pengfei Li, Jingcheng Wu, Chenghua Lin, Qifeng Liu, Tao Jiang, Wenhao Huang, Wenhu Chen, Emmanouil Benetos, Jie Fu, Gus Xia, Roger Dannenberg, Wei Xue, Shiyin Kang, Yike Guo
It is based on continual pre-training and finetuning LLaMA2 on a text-compatible music representation, ABC notation, and the music is treated as a second language.
1 code implementation • 14 Feb 2024 • Liwei Lin, Gus Xia, Yixiao Zhang, Junyan Jiang
We apply this method to fine-tune MusicGen, a leading autoregressive music generation model.
no code implementations • 9 Feb 2024 • Yixiao Zhang, Yukara Ikemiya, Gus Xia, Naoki Murata, Marco A. Martínez-Ramírez, Wei-Hsiang Liao, Yuki Mitsufuji, Simon Dixon
This paper introduces a novel approach to the editing of music generated by such models, enabling the modification of specific attributes, such as genre, mood and instrument, while maintaining other aspects unchanged.
no code implementations • 3 Dec 2023 • Qisheng Liao, Zhinuo Wang, Muhammad Abdul-Mageed, Gus Xia
Chinese calligraphy can be viewed as a unique form of visual art.
1 code implementation • 26 Oct 2023 • Liwei Lin, Gus Xia, Junyan Jiang, Yixiao Zhang
We aim to further equip the models with direct and content-based controls on innate music languages such as pitch, chords and drum track.
1 code implementation • 25 Oct 2023 • Jingwei Zhao, Gus Xia, Ye Wang
The first component is a piano arranger that generates piano accompaniment for the lead sheet by transferring texture styles to the chords using latent chord-texture disentanglement and heuristic retrieval of texture donors.
1 code implementation • 19 Oct 2023 • Yixiao Zhang, Akira Maezawa, Gus Xia, Kazuhiko Yamamoto, Simon Dixon
Creating music is iterative, requiring varied methods at each stage.
1 code implementation • 19 Sep 2023 • Yuxuan Wu, Roger B. Dannenberg, Gus Xia
Music motif, as a conceptual building block of composition, is crucial for music structure analysis and automatic composition.
1 code implementation • 19 Jul 2023 • Lejun Min, Junyan Jiang, Gus Xia, Jingwei Zhao
We propose Polyffusion, a diffusion model that generates polyphonic music scores by regarding music as image-like piano roll representations.
no code implementations • 11 Jul 2023 • Yinghao Ma, Ruibin Yuan, Yizhi Li, Ge Zhang, Xingran Chen, Hanzhi Yin, Chenghua Lin, Emmanouil Benetos, Anton Ragni, Norbert Gyenge, Ruibo Liu, Gus Xia, Roger Dannenberg, Yike Guo, Jie Fu
Our findings suggest that training with music data can generally improve performance on MIR tasks, even when models are trained using paradigms designed for speech.
1 code implementation • NeurIPS 2023 • Ruibin Yuan, Yinghao Ma, Yizhi Li, Ge Zhang, Xingran Chen, Hanzhi Yin, Le Zhuo, Yiqi Liu, Jiawen Huang, Zeyue Tian, Binyue Deng, Ningzhi Wang, Chenghua Lin, Emmanouil Benetos, Anton Ragni, Norbert Gyenge, Roger Dannenberg, Wenhu Chen, Gus Xia, Wei Xue, Si Liu, Shi Wang, Ruibo Liu, Yike Guo, Jie Fu
This is evident in the limited work on deep music representations, the scarcity of large-scale datasets, and the absence of a universal and community-driven benchmark.
1 code implementation • 31 May 2023 • Yizhi Li, Ruibin Yuan, Ge Zhang, Yinghao Ma, Xingran Chen, Hanzhi Yin, Chenghao Xiao, Chenghua Lin, Anton Ragni, Emmanouil Benetos, Norbert Gyenge, Roger Dannenberg, Ruibo Liu, Wenhu Chen, Gus Xia, Yemin Shi, Wenhao Huang, Zili Wang, Yike Guo, Jie Fu
Although SSL has been proven effective in speech and audio, its application to music audio has yet to be thoroughly explored.
no code implementations • 30 May 2023 • Qisheng Liao, Gus Xia, Zhinuo Wang
In this paper, we propose Calliffusion, a system for generating high-quality Chinese calligraphy using diffusion models.
1 code implementation • NeurIPS 2023 • Xuanjie Liu, Daniel Chin, Yichen Huang, Gus Xia
We have recently seen great progress in learning interpretable music representations, ranging from basic factors, such as pitch and timbre, to high-level concepts, such as chord and texture.
1 code implementation • 10 Nov 2022 • Runbang Zhang, Yixiao Zhang, Kai Shao, Ying Shan, Gus Xia
In this study, we explore the representation mapping from the domain of visual arts to the domain of music, with which we can use visual arts as an effective handle to control music generation.
1 code implementation • 31 Oct 2022 • Junyan Jiang, Gus Xia
We propose a novel method to model hierarchical metrical structures for both symbolic music and audio signals in a self-supervised manner with minimal domain knowledge.
1 code implementation • 21 Sep 2022 • Junyan Jiang, Daniel Chin, Yixiao Zhang, Gus Xia
In this paper, we explore a data-driven approach to automatically extract hierarchical metrical structures from scores.
1 code implementation • 21 Sep 2022 • Yang Qu, Yutian Qin, Lecheng Chao, Hangkai Qian, Ziyu Wang, Gus Xia
The relationship between perceptual loudness and physical attributes of sound is an important subject in both computer music and psychoacoustics.
1 code implementation • 1 Sep 2022 • Li Yi, Haochen Hu, Jingwei Zhao, Gus Xia
We propose AccoMontage2, a system capable of doing full-length song harmonization and accompaniment arrangement based on a lead melody.
1 code implementation • 24 Aug 2022 • Yixiao Zhang, Junyan Jiang, Gus Xia, Simon Dixon
Lyric interpretations can help people understand songs and their lyrics quickly, and can also make it easier to manage, retrieve and discover songs efficiently from the growing mass of music archives.
no code implementations • 13 Feb 2022 • Shiqi Wei, Gus Xia
Learning symbolic music representations, especially disentangled representations with probabilistic interpretations, has been shown to benefit both music understanding and generation.
1 code implementation • 30 Dec 2021 • Ziyu Wang, Dejing Xu, Gus Xia, Ying Shan
This is the audio-to-symbolic arrangement problem we tackle in this paper.
1 code implementation • 25 Aug 2021 • Jingwei Zhao, Gus Xia
Accompaniment arrangement is a difficult music generation task involving intertwined constraints of melody, harmony, texture, and music structure.
1 code implementation • 7 Aug 2021 • Liwei Lin, Qiuqiang Kong, Junyan Jiang, Gus Xia
We propose a unified model for three inter-related tasks: 1) to \textit{separate} individual sound sources from a mixed music audio, 2) to \textit{transcribe} each sound source to MIDI notes, and 3) to\textit{ synthesize} new pieces based on the timbre of separated sources.
2 code implementations • 17 Aug 2020 • Ziyu Wang, Dingsu Wang, Yixiao Zhang, Gus Xia
While deep generative models have become the leading methods for algorithmic composition, it remains a challenging problem to control the generation process because the latent variables of most deep-learning models lack good interpretability.
2 code implementations • 17 Aug 2020 • Ziyu Wang, Yiyi Zhang, Yixiao Zhang, Junyan Jiang, Ruihan Yang, Junbo Zhao, Gus Xia
The dominant approach for music representation learning involves the deep unsupervised model family variational autoencoder (VAE).
1 code implementation • 17 Aug 2020 • Ziyu Wang, Ke Chen, Junyan Jiang, Yiyi Zhang, Maoran Xu, Shuqi Dai, Xianbin Gu, Gus Xia
The main body of the dataset contains the vocal melody, the lead instrument melody, and the piano accompaniment for each song in MIDI format, which are aligned to the original audio files.
1 code implementation • 21 Jul 2020 • Tongyu Lu, Lyucheng Yan, Gus Xia
This paper proposes a word representation strategy for rhythm patterns.
1 code implementation • 5 Feb 2020 • Ke Chen, Gus Xia, Shlomo Dubnov
Automatic music generation is an interdisciplinary research topic that combines computational creativity and semantic analysis of music to create automatic machine improvisations.
3 code implementations • 9 Jun 2019 • Ruihan Yang, Dingsu Wang, Ziyu Wang, Tianyao Chen, Junyan Jiang, Gus Xia
Analogy-making is a key method for computer algorithms to generate both natural and creative music pieces.
no code implementations • 18 Apr 2019 • Ruihan Yang, Tianyao Chen, Yiyi Zhang, Gus Xia
Variational Autoencoders(VAEs) have already achieved great results on image generation and recently made promising progress on music generation.
no code implementations • 28 Dec 2018 • Ziyu Wang, Gus Xia
Second, the melody generation model generates the lead melody and other voices (melody lines) of the accompaniment using seasonal ARMA (Autoregressive Moving Average) processes.
1 code implementation • 20 Nov 2018 • Ke Chen, Weilin Zhang, Shlomo Dubnov, Gus Xia, Wei Li
With recent breakthroughs in artificial neural networks, deep generative models have become one of the leading techniques for computational creativity.
no code implementations • 14 Nov 2018 • Yixing Guan, Jinyu Zhao, Yiqin Qiu, Zheng Zhang, Gus Xia
Automated melodic phrase detection and segmentation is a classical task in content-based music information retrieval and also the key towards automated music structure analysis.