Everything runs locally.
One Windows PC.
One RTX 4080.
One open-weight music model, free for non-commercial use.

Spoiler: yes. Especially if you know a bit about computers and music.

This blog is about YuE2, an open-weight music generation model free for non-commercial use, and how I used it on my PC to create a complete track.

Why I did this experiment

I wanted to see how far a small open music model could go on a normal high-end PC. The goal was simple: generate a complete song locally, understand how the model works, and see if the result could become something I would actually keep and finish in Ableton.

YuE2

YuE2 is an open music generation model developed by M-A-P (Multimodal Art Projection), an open AI research community working on foundation models and creative AI. It takes lyrics + a style prompt and generates a complete song with vocals and accompaniment. Under the hood, it uses one AR–NAR Transformer backbone to generate musical and semantic tokens, then acoustic latents, and finally a VAE turns them into 48 kHz stereo audio.

YuE2: https://arxiv.org/pdf/2609.33757

YuE2 is open-weight, but that does not mean its complete training dataset is public. The team says YuE2 was trained on around 346,000 hours of primarily CC0 music (no copyright) and synthetic data.

YuE2 works on a compressed musical representation first, then converts it into sound. This makes the model closer to a music-generation pipeline than a simple text-to-waveform model.

The configuration

I used my Alienware desktop with an RTX 4080 and 16 GB of VRAM, running Windows 11.

For the model, I used YuE2-3B in Q8 format with the F16 VAE, running through audio.cpp + CUDA.

Q8 was a good compromise for this test. It keeps the model small enough to fit comfortably on the GPU while preserving much more precision than lower-bit quantization.

Learning how to prompt YuE2

YuE2 works best when I separate lyrics from production instructions.

The lyrics define the structure with tags such as [Verse], [Chorus], [Drum Solo] or [Guitar Solo].

The style prompt carries the rest: scale, BPM, instruments, vocal tone, mix, energy and atmosphere.

I also tuned a few generation parameters. I kept num_inference_steps=8 for fast iteration, fixed the seed when I wanted to compare two versions, and used the default sampling values: temperature=1, top_p=0.95, top_k=100 and repetition_penalty=1.2.

One funny lesson: when I wrote detailed instructions inside the lyrics field, YuE2 sometimes sang them. So I kept the lyrics clean and moved all production details into style.

The first surprise: it is fast (and good)

The first real surprise was the speed.

One of my complete generations took about 59 seconds on the GPU. Most of that time was spent generating the semantic music tokens. The acoustic synthesis and VAE decoding were much faster.

And the second surprise was the quality.

The first tracks were basic, but already coherent enough to hear structure, groove, vocals and instrumentation. After a few prompt iterations, some generations became genuinely interesting rather than just technically impressive.

For a local 3B model running on one consumer GPU, that changed the experiment very quickly from “does it work?” to “how far can I push it?”

The weird experiments were the best ones

The best results came when I stopped trying to write “proper” songs.

I tested French, English, Kryptonian-inspired lyrics, then a completely invented Nordic-sounding language. After that, I went one step further with vocals that sound English without really being English.

That was surprisingly effective. Once meaning became less important, the voice could behave more like another instrument. Rhythm, phonetics and groove started to matter more than the words themselves.

That is where YuE2 became really fun.

The final track: UpTown

The final version used pseudo-English lyrics over an early-2000s alternative hip-hop/reggae style: A major for a brighter mood, 94 BPM, dark boom-bap drums, warm bass, dusty piano and a low male vocal (real prompt for the style is my own recipe ;-)

YuE2 generated the full song locally. I then moved to Ableton Live for EQ, compression, balance and mastering.

At that point, the experiment had become a real track. Not a masterpiece, but that was never the purpose.

https://open.spotify.com/album/0mLOLjzKqw3E4rFbItNFku

What YuE2 is good at, and bad at

YuE2 is good at structure, groove and experimentation. It understands sections, styles, tempo, vocal character and basic arrangement surprisingly well. It also reacts nicely to unusual phonetics, which made the fake-language tests much more interesting than expected.

Its limits are also clear. The songs tend to stay quite linear: little harmonic movement, almost no scale or key changes, few production FX, and transitions that a musician can often predict. The guitar and drum solos work, but they are usually simple rather than expressive or technically interesting (still better than me on guitar; on piano, I’m not so sure...).

High frequencies can also become metallic, as with other models. Vocals can repeat too much, and detailed structure instructions are followed with some freedom. The raw mix can be uneven.

For me, YuE2 is a very good creative generator. It gives you a strong first composition and useful material, then a DAW is still the place to add movement, surprise and final production.

Conclusion XP

Yes, it is possible to run a capable music generation model for free, for non-commercial use, on a local PC.

For an artist, that is interesting as a source of material: ideas, textures, vocal directions, arrangements, strange experiments, or simply a way to get unstuck and try something quickly.

I would not use YuE2 to release a complete song directly. The output still needs musical judgment, editing and production.

And there is one practical condition: you need some computer skills. Installing the runtime, choosing the right model format, using CUDA, managing prompts and understanding what the model is doing still requires more than clicking a button.

For me, that is part of what makes the experiment interesting. It is fun. YuE2 feels less like a finished product and more like a creative laboratory for artists.

The objective is to create a rap / hip-hop instrumental without vocals, using The Infinite Crate as a source of generated audio material inside Ableton Live.

This is not a benchmark of audio quality and not a full review of the plugin. It is a practical production experiment focused on workflow, control, usability, and the role of the producer when a real-time music model is integrated directly into a DAW.

Short history: Magenta and Lyria

Magenta started as a Google research project exploring machine learning as a tool for music and art creation. For years, many Magenta projects were closer to research environments: Python notebooks, TensorFlow models, MIDI experiments, browser demos, and open-source prototypes.

This history matters because The Infinite Crate is not an isolated plugin. It is part of a broader trajectory from research models toward creative interfaces.

Google DeepMind introduced Lyria in 2023 as a music generation model. The model family later evolved toward more advanced music generation systems, including Lyria 3 for high-quality text-to-music generation and Lyria RealTime for interactive music creation.

Lyria RealTime changes the interaction model:

prompt + controls → continuous audio stream → capture inside DAW

The generated music is no longer only a static file produced after a prompt. It becomes a stream that can be steered, recorded, sampled, and transformed while the producer is working.

This is the technical shift tested here.

What is The Infinite Crate?

The Infinite Crate is an experimental VST / DAW plugin from Google Magenta. It integrates Lyria RealTime into a music production environment and allows the user to steer a live music model with text prompts.

The plugin feeds generated audio directly into the DAW. The intended uses include sampling, live performance, and backing tracks for jamming or practice.

The plugin behaves less like a traditional instrument and more like a source of sound. It is similar to digging through an imaginary crate of records, except the crate is generated by a model in real time.

From a technical point of view, the open-source project is also interesting because it connects a real-time generative model with a plugin architecture. The public implementation is built around a DAW plugin stack, with audio processing and user interface components connected to the Lyria RealTime model through the Gemini API.

The most relevant point for producers is not the internal architecture alone. The key point is the workflow:

generate → record → cut → sample → rebuild → mix

Another possible use is live performance for musicians or DJs, where the plugin acts as a real-time generative source.

Test method and limits

Test setup:

• DAW: Ableton Live 12

• Plugin: The Infinite Crate

• Model: Lyria RealTime

• Target: rap / hip-hop instrumental

• Vocals: none

The test focuses on instrumental production only.

Limits:

• The test is based on one production session, not a large-scale evaluation.

• The result depends strongly on prompt choices, manual selection, and chance.

• The model output is evaluated as raw material, not as a final track.

• No access is available to the internal details of Lyria RealTime beyond public documentation.

• The experiment does not address copyright, dataset provenance, or watermarking in depth.

https://www.youtube.com/watch?v=MSUtcnglvK8

First observations

The strongest value of The Infinite Crate is speed of exploration.

Starting a DAW session from silence can be difficult. The plugin immediately provides audio material to react to. Some results are generic, some are unusable, but some contain interesting textures, harmonic fragments, bass movements, or short loops.

The real-time aspect changes the experience compared with a classic text-to-music generator. The output is not only requested and downloaded. It is heard as it evolves. This makes the tool feel closer to a live source than to an export function.

The main limitation is control.

Prompts can guide the model, but they do not precisely define rhythm, swing, arrangement, density, or mix balance.

In several cases, the generated audio contained too much movement or too much harmonic information to work directly as a rap instrumental. In other cases, the texture was useful, but the rhythmic foundation needed to be rebuilt manually.

The best results came when the plugin was treated as a sample source rather than a complete beat generator.

What I like

The plugin worked best for:

• generating textures;

• creating short harmonic fragments;

• producing background layers;

• finding unexpected samples;

• starting a beat without a blank session;

• creating material that can be chopped and transformed.

The most useful workflow was to record longer streams, then search inside them for short usable sections. This is close to traditional sampling. The producer does not need the full generated output to be good. A few seconds can be enough.

This is an important difference from full-song generation. In a full-song workflow, the expected output is a coherent finished piece. In a sampling workflow, the expected output is material with potential.

For this type of use, The Infinite Crate is convincing.

What I don't like

First, I had reliability difficulties with the connection to the Lyria servers.

A DAW workflow is based on precision. The producer controls tempo, grid, MIDI, audio clips, automation, routing, effects, gain staging, and arrangement.

Real-time GenAI is more exploratory and less deterministic. It gives a direction more than a fully controlled result.

This tension is not necessarily negative. It is useful during ideation, but less useful during final production.

For this experiment, the most practical approach was to use The Infinite Crate early in the process, then return to traditional Ableton editing and mixing for the final structure.

XP Conclusion

The Infinite Crate is a real experiment in what a virtual GenAI instrument could become in the future. It gives a taste of what may come next to support artists.

It does not remove the need for selection, editing, arrangement, sound design, mixing, or taste. It is also not yet precise enough to behave like a normal instrument or a standard sample library.

Its value is different. It provides a fast source of generated audio directly inside the DAW.

For a hip-hop production, this positioning is relevant. The plugin is most useful when treated as a crate, not as a composer. The generated stream becomes material to dig through, cut, and transform.

The model generates. The producer decides.

This distinction is important for the future of music GenAI. Full-song generation will continue to improve, but DAW-native GenAI tools may have a deeper impact on how music is actually produced.

A web generator produces an output.

A DAW plugin enters the workflow.

That difference may be more important than the quality of one isolated generation.

Like in many other domains, several foundational open-source models for music emerged between 2022 and 2024, including AudioCraft from Meta, MusicLM from Google, Stable Audio, and others. These models marked an important step forward, yet none established a sustainable business model on their own.

Some companies seized the opportunity and developed proprietary systems, likely trained on large-scale music catalogs gathered from multiple sources, including streaming platforms. Udio builds on Stable Audio by Stability AI, while Suno relies on BARK, itself derived from MuLAN developed by Google. Beyond the ongoing debates around ethics, both for artists and researchers, the quality of the generated music has reached a remarkably high level, often creating a real “wow” effect.

Recently, a new generation of GenAI plugins has started to emerge within DAWs. Tools such as ACE or Synthesizer V Studio illustrate this shift. These solutions remain recent, yet they already show strong potential for producers working directly inside their production environments.

The problem: artificial streaming

Streaming platforms face a growing challenge with artificial streaming. Bot farms generate large volumes of plays to capture royalties, creating a system that can be exploited at scale. The rise of GenAI further lowers the barrier to entry, making it easier to produce large quantities of music tailored for this purpose.

GenAI music generation itself is not the core issue. The real challenge lies in how it can be used at scale within these systems, as well as in the question of copyright and the use of artists’ data to train these models. This is why streaming platforms are developing different approaches to address the problem.

The problem for the artist: the revenue

For artists, revenue distribution becomes increasingly diluted, and the value of genuine listening is harder to preserve within an ecosystem shaped by automated activity. This pressure extends an existing shift that began with the rise of streaming platforms more than a decade ago, pushing artists to adapt their models toward merchandising, public support, and live performances.

Copyright protection has always been a challenge, with gaps between countries. The way foundational models are trained, often to reproduce similar outputs from large datasets, also raises important questions.

Why I did this experiment?

As an engineer, curiosity naturally leads to experimentation. I recently released a music album, mainly for fun, on streaming platforms where I combined generative AI, instruments, and traditional production tools. GenAI was mainly used for vocals to sing my lyrics, along with contributions on other stems, resulting in a hybrid workflow that even included some Python-based mathematical generation for the bass.

After the release, Deezer flagged the album with the label “AI-generated content: This album includes tracks detected as generated with the use of AI.” Some tracks indeed incorporated Suno v5.0 before further work in the DAW. This triggered a deeper curiosity about how such detection is performed. Other streaming platforms, such as Spotify or Apple Music, did not flag the album as AI-generated, although they may also use detection systems.

Initial theory about Deezer’s AI detection

I initially suspected a clustering approach based on musical features extracted from each stem. Another hypothesis pointed toward a fingerprinting mechanism, similar to Shazam, applied to individual stems, such as vocal signatures, spectral artifacts, or structural patterns. In both cases, the first logical step would be to separate the stems before analysis.

Test method and limits

The approach consisted of releasing four singles, one every four days, and observing whether each track was flagged on Deezer. Each track used GenAI as a starting point, then underwent different levels of rearrangement and restructuring within the DAW. The objective was to identify which transformations would trigger Deezer’s GenAI detection.

Minimalist electronic music is particularly well suited for computational processing. I created four singles with a similar artistic direction while applying different production approaches to each. The lyrics and musical ideas were written by me, kept intentionally simple, with a focus on a recognizable style, clear scales/chords, and coherent, distinctive melodies. The primary goal of these tracks is experimentation.

Test cases:

Material:

• Generation: Suno v5.5 with stems separation

• Digital Audio Workstation (DAW): Ableton Live with plugins

• Controller / sound library: Arturia & PML

• Hardware: Mac, Focusrite audio interface, Pioneer/Shure monitors

Limits:

• No use of real instruments, only sound libraries and MIDI

• Limited hardware and software setup

• No access to Deezer’s internal detection systems

T2: Be a Better Man - A Minor

The objective of T2 was to apply minimal transformation to the Suno v5.5 output after stem separation. The work focused on standard DAW processing, including EQ, Dynamic EQ, compression, limiting, and saturation, with no use of GenAI plugins. Mastering followed a traditional approach.

The expected outcome was detection as AI-generated content.

The result confirmed this expectation, as the track was flagged by Deezer. This suggests that basic post-processing is not sufficient to alter the underlying signal used for detection. The system appears to rely on deeper audio characteristics rather than simple clustering or surface-level features.

https://open.spotify.com/album/2GhkDN9RMtmCzPGWVFdT3D

Also available in all other streaming platforms or here

T3: AI Rising - A Minor

The objective of T3 was to go further than T2 by restructuring the track after stem separation. This included rearranging sections, introducing cuts and offsets, and modifying the overall flow of the song within the DAW. I also added a layer of white noise and additional effects (FX) to further alter the texture. The same set of standard tools was used, including EQ, dynamic EQ, compression, limiting, and saturation, with no GenAI plugins. Mastering followed a traditional approach.

The expected outcome was less certain. The hypothesis was that structural changes, combined with added noise and effects, might weaken the detection signal or alter the classification.

The results shows that the track was still flagged by Deezer (possibly slower than T2). This indicates that modifying the structure and adding surface-level perturbations are not sufficient to remove the underlying characteristics identified by the detection system. The key takeaway is that the algorithm likely relies on deeper audio signatures that persist despite rearrangement and added noise, rather than on temporal structure or superficial texture alone.

https://open.spotify.com/album/7LAekp9yIkQU1mEnMVMvzu?si=s0S70PgOR3m42DM9-6VUlA

Also available in all other streaming platforms or here

T4a: Think Twice - G Minor

The objective of T4a was to go further by replacing a core musical component. After stem separation and restructuring in the DAW, I created a completely new drum track using a Drum Rack in Ableton, while keeping the other elements from the original workflow. Standard processing was applied, including EQ, Dynamic EQ, compression, limiting, and saturation, with no GenAI plugins. Mastering followed a traditional approach.

The expected outcome remained uncertain. Replacing the rhythmic foundation could alter the overall signature enough to impact detection.

The results shows that the track was not flagged by Deezer. This suggests that modifying a key component such as the drums can significantly affect detection outcome. The main takeaway is that the system appears sensitive to the internal consistency of the mix. Altering one major element may reduce the coherence of the original GenAI signal and bring it below the detection threshold.

https://open.spotify.com/album/4uWEhGSjdDaHokvy03BTlO?si=XjMCy1NoT1240eAYrz2sgg

Also available in all other streaming platforms or here

T4b: Cities - G Minor

The objective of T4b was to push the transformation further by rebuilding the track almost entirely. After stem separation and restructuring in the DAW, all musical elements were replaced with new stems created manually in Ableton, while keeping only the original vocal. The arrangement, instrumentation, and overall texture were redesigned within Ableton. Standard processing was applied, including EQ, Dynamic EQ, compression, limiting, and saturation, with no GenAI plugins. Mastering followed a traditional approach.

What the results seem to suggest so far

Based on the experiments, a realistic assumption is that Deezer relies on a compact set of well-known audio ML building blocks rather than a single complex system. The pipeline can be simplified into a few core stages, each associated with concrete methods.

Step 1: Representation learning (feature extraction)
The audio is transformed into a format suitable for machine learning. This is most likely done using log-mel spectrograms or pretrained embedding models such as OpenL3 or VGGish. A more advanced setup could rely directly on transformer-based encoders like AST (Audio Spectrogram Transformer). The goal is to map raw audio into a feature space where generative signatures become separable.

Step 2: Core classification model
A supervised model is trained to distinguish between human-produced and GenAI-generated tracks. Likely candidates include:

• CNN-based models such as PANNs (CNN14) for robustness and efficiency

• Transformer-based models like AST or HTS-AT for capturing long-range temporal patterns

• Or a hybrid approach combining embeddings with a lighter classifier such as XGBoost or a small MLP

This model learns the global “texture” of GenAI audio, including spectral smoothness, transient behavior, and harmonic consistency.

Step 3: Component-aware analysis
To go beyond the full mix, the system likely incorporates some form of component-level reasoning. This can be done in two ways:

• explicit separation using models such as Demucs (INRIA for Meta ..) or Spleeter (created by Deezer), followed by per-stem analysis

• or implicit decomposition within the model using attention mechanisms that specialize in different regions of the spectrum

Each component can then be processed with the same type of classifier as in Step 2.

Step 4: Cross-component consistency modeling
A key assumption is that the system evaluates how different parts of the track relate to each other. This can be implemented with:

• similarity measures such as cosine similarity between embedding vectors

• or learned fusion layers using multi-head attention or a small fusion MLP

This stage captures whether vocals, drums, and harmonic layers share a coherent generative signature.

Step 5: Decision layer and thresholding
The outputs from global and component-level models are aggregated. This can be done with:

• a simple weighted average (simple but efficient)

• or a learned ensemble model such as logistic regression or gradient boosting

A confidence threshold determines whether the track is labeled as AI-generated. Multiple thresholds or passes may exist, which would explain delayed detection in some cases. If one track in the album is flagged, the entire album is flagged, probably to save computing resources.

In this setup, the system combines a deep audio classifier (AST or PANNs) with optional stem-aware analysis and a fusion mechanism that evaluates consistency across components. This aligns well with the experiments, where structural edits alone have limited impact, while modifications to a key element of the mix can influence the final outcome.

Limits of the experiment

Producing a track, even with the support of GenAI, still requires time, inspiration, and sustained effort. This naturally limited the scope of the experiment to four singles, each representing a distinct transformation approach.

The results should also be understood within a changing context. Additional variations between T3 and T4a could provide more data points, yet the detection system itself is likely evolving over time. Each observation reflects a specific moment, and future outcomes may differ as the platform updates its models and thresholds.

Conclusion

The experiments indicate that Deezer’s detection system is highly effective at identifying tracks generated with tools such as Suno when post-processing remains limited. The model appears robust to standard DAW transformations and captures deeper characteristics of the audio.

For Deezer users, there may be some interest in knowing how music is produced and whether GenAI is involved.

For GenAI platforms such as Suno and Udio, this type of detection is a win. It creates a feedback loop that may encourage better supervision of training data and help maintain clearer boundaries over time, avoiding AI pollution.

However, this approach does not address the broader issue of artificial streaming. The underlying economic pressure on artists remains, with revenue distribution still impacted by large-scale automated activity. Other platforms, such as Spotify, appear to focus more directly on detecting artificial streaming patterns, whether driven by GenAI or by large volumes of low-cost production music. Copyright issues also remain and extend beyond music.

In my opinion, the current business model of platforms such as Suno and Udio, based on classical multimodal approaches, may have limitations over time, with constraints in data and a tendency to be more reactive than anticipatory. New approaches such as I-JEPA seem promising to support artists in their music creation. Detection will likely evolve in the same direction. For both, it remains a million-dollar question: what is the purpose?

Final words

I truly enjoyed conducting this experiment. It opens the door to ongoing discussions about the future of music and reflects broader questions shared across many industries today. At its core, the issue remains simple: how we choose to use technology will shape what comes next.

Next experiment: Lyria RealTime from Google.