Google has released a machine learning tool that can generate pieces of music itself based on entered text. MusicLM is not yet available to everyone, but Google has put a research paper and samples online.
Google writes in a paper that it created MusicML, a hierarchical sequence-to-sequence machine learning model. The tool can create music pieces with a clarity of 24KHz that last several minutes based on a text prompt. In addition to text, the prompt can also create music based on whistling or humming, or in response to a photo or a painting. Google gives an example of a painting by Salvador Dali, from which MusicLM composes its own song.
The tool itself cannot yet be used by everyone. However, Google on a separate website has put samples online with the corresponding prompts. Those are descriptions like “slow tempo, bass-and-drums-led reggae song.” Sustained electric guitar. High pitched bongos with ringing tones. Vocals are relaxed with a laid-back feel, very expressive’. MusicLM can also create multi-minute songs in a so-called narrative mode where the prompt tells you what’s happening at different times in the song.
Google has trained the tool on a dataset of 280,000 hours of music. In addition to the tool, Google has also made a dataset called MusicCaps publicly available to researchers. That dataset consists of 5500 music descriptions including their original music. In the paper, Google does not write anything about copyrighted material and how the tool handles it.
