Real-Time Music GenAI

I explore Google Magenta’s *The Infinite Crate*, an experimental DAW (Digital Audio Workstation) plugin built on *Lyria RealTime*.

The objective is to create a rap / hip-hop instrumental without vocals, using The Infinite Crate as a source of generated audio material inside Ableton Live.

This is not a benchmark of audio quality and not a full review of the plugin. It is a practical production experiment focused on workflow, control, usability, and the role of the producer when a real-time music model is integrated directly into a DAW.

---

Short history: Magenta and Lyria

Magenta started as a Google research project exploring machine learning as a tool for music and art creation. For years, many Magenta projects were closer to research environments: Python notebooks, TensorFlow models, MIDI experiments, browser demos, and open-source prototypes.

This history matters because The Infinite Crate is not an isolated plugin. It is part of a broader trajectory from research models toward creative interfaces.

Google DeepMind introduced Lyria in 2023 as a music generation model. The model family later evolved toward more advanced music generation systems, including Lyria 3 for high-quality text-to-music generation and Lyria RealTime for interactive music creation.

Lyria RealTime changes the interaction model:

prompt + controls → continuous audio stream → capture inside DAW

The generated music is no longer only a static file produced after a prompt. It becomes a stream that can be steered, recorded, sampled, and transformed while the producer is working.

This is the technical shift tested here.

---

What is The Infinite Crate?

The Infinite Crate is an experimental VST / DAW plugin from Google Magenta. It integrates Lyria RealTime into a music production environment and allows the user to steer a live music model with text prompts.

The plugin feeds generated audio directly into the DAW. The intended uses include sampling, live performance, and backing tracks for jamming or practice.

The plugin behaves less like a traditional instrument and more like a source of sound. It is similar to digging through an imaginary crate of records, except the crate is generated by a model in real time.

From a technical point of view, the open-source project is also interesting because it connects a real-time generative model with a plugin architecture. The public implementation is built around a DAW plugin stack, with audio processing and user interface components connected to the Lyria RealTime model through the Gemini API.

The most relevant point for producers is not the internal architecture alone. The key point is the workflow:

generate → record → cut → sample → rebuild → mix

Another possible use is live performance for musicians or DJs, where the plugin acts as a real-time generative source.

---

Test method and limits

Test setup:

• DAW: Ableton Live 12

• Plugin: The Infinite Crate

• Model: Lyria RealTime

• Target: rap / hip-hop instrumental

• Vocals: none

The test focuses on instrumental production only.

Limits:

• The test is based on one production session, not a large-scale evaluation.

• The result depends strongly on prompt choices, manual selection, and chance.

• The model output is evaluated as raw material, not as a final track.

• No access is available to the internal details of Lyria RealTime beyond public documentation.

• The experiment does not address copyright, dataset provenance, or watermarking in depth.

https://www.youtube.com/watch?v=MSUtcnglvK8

---

First observations

The strongest value of The Infinite Crate is speed of exploration.

Starting a DAW session from silence can be difficult. The plugin immediately provides audio material to react to. Some results are generic, some are unusable, but some contain interesting textures, harmonic fragments, bass movements, or short loops.

The real-time aspect changes the experience compared with a classic text-to-music generator. The output is not only requested and downloaded. It is heard as it evolves. This makes the tool feel closer to a live source than to an export function.

The main limitation is control.

Prompts can guide the model, but they do not precisely define rhythm, swing, arrangement, density, or mix balance.

In several cases, the generated audio contained too much movement or too much harmonic information to work directly as a rap instrumental. In other cases, the texture was useful, but the rhythmic foundation needed to be rebuilt manually.

The best results came when the plugin was treated as a sample source rather than a complete beat generator.

---

What I like

The plugin worked best for:

• generating textures;

• creating short harmonic fragments;

• producing background layers;

• finding unexpected samples;

• starting a beat without a blank session;

• creating material that can be chopped and transformed.

The most useful workflow was to record longer streams, then search inside them for short usable sections. This is close to traditional sampling. The producer does not need the full generated output to be good. A few seconds can be enough.

This is an important difference from full-song generation. In a full-song workflow, the expected output is a coherent finished piece. In a sampling workflow, the expected output is material with potential.

For this type of use, The Infinite Crate is convincing.

---

What I don't like

First, I had reliability difficulties with the connection to the Lyria servers.

A DAW workflow is based on precision. The producer controls tempo, grid, MIDI, audio clips, automation, routing, effects, gain staging, and arrangement.

Real-time GenAI is more exploratory and less deterministic. It gives a direction more than a fully controlled result.

This tension is not necessarily negative. It is useful during ideation, but less useful during final production.

For this experiment, the most practical approach was to use The Infinite Crate early in the process, then return to traditional Ableton editing and mixing for the final structure.

---

XP Conclusion

The Infinite Crate is a real experiment in what a virtual GenAI instrument could become in the future. It gives a taste of what may come next to support artists.

It does not remove the need for selection, editing, arrangement, sound design, mixing, or taste. It is also not yet precise enough to behave like a normal instrument or a standard sample library.

Its value is different. It provides a fast source of generated audio directly inside the DAW.

For a hip-hop production, this positioning is relevant. The plugin is most useful when treated as a crate, not as a composer. The generated stream becomes material to dig through, cut, and transform.

The model generates. The producer decides.

This distinction is important for the future of music GenAI. Full-song generation will continue to improve, but DAW-native GenAI tools may have a deeper impact on how music is actually produced.

A web generator produces an output.

A DAW plugin enters the workflow.

That difference may be more important than the quality of one isolated generation.