Tip Of AI
Discover what you can do with AI
AudioCraft is an AI research project developed by Meta AI that serves as a comprehensive platform for generative audio tasks. It offers a centralized code base for creating music, sound effects, and audio compression after training on raw audio signals. The design of AudioCraft simplifies the process of creating generative models for audio in comparison to previous methods. It consists of two main components: MusicGen and AudioGen. Both components utilize a single autoregressive Language Model (LM) that operates on compressed discrete music representations or tokens. By leveraging the internal structure of parallel token streams and employing an interleaving pattern, AudioCraft efficiently models audio sequences, captures long-term dependencies, and generates high-quality audio. To facilitate audio token learning, AudioCraft utilizes the EnCodec neural audio codec. EnCodec maps the audio signal to parallel streams of discrete tokens, which are then recursively modeled by the autoregressive language model. The generated tokens are fed into the EnCodec decoder to transform them back into audio signals, resulting in the final output waveform. Additionally, AudioCraft supports various conditioning models, such as using a pretrained text encoder for text-to-audio applications, to control the generation process. AudioCraft offers two specific audio generation tasks: text-to-sound generation and text-to-music generation. AudioGen focuses on generating audio from environmental sounds based on text inputs, while MusicGen produces diverse and extended music samples from user-provided text inputs. For additional resources and technical details, users can refer to the AudioCraft section on Meta AI's website. The platform provides sample audio outputs, a blog, technical information, the code base, and model cards. ©2023 Meta AI reserves all rights to AudioCraft.