Tip Of AI
Discover AI tools
YuE2 is an advanced AI model that unifies symbolic music representation and full audio output in a single end to end workflow. It generates an editable score through symbolic planning and then transforms this plan into a high quality song complete with vocals and accompaniment. The core architecture consists of a 3.59 billion parameter transformer with 28 layers and separate autoregressive and non autoregressive experts sharing an attention computation backbone. Symbolic planning allows users to input lyrics and style prompts and receive ABC notation which can be modified interactively before rendering. Features include high fidelity VAE based audio generation, multi turn agentic music editing in collaboration with language models, interactive score playback, and a genre explorer with over ninety nine songs across seventy genres in multiple languages. The system integrates components such as MERT2 for robust representations, SheetSage2 for audio to score transcription, and benchmarks on WildSongBench and MARBLE that demonstrate state of the art performance across generation and transcription metrics. Training data spans hundreds of thousands of hours of CC0 and licensed synthetic data under ethical guidelines. The open source release on GitHub ensures transparency, reproducibility, and community contributions for continued research and development.
