What would it take to integrate VST plugins?

[tophee]

My specific interest is in using the GoodHertz CanOpener plugin, but I suppose other people might be interested in adding other VST plugins to the Music Assistant DSP pipeline.

So I'm wondering what it would take to integrate VST plugins. ChatGPT proposes two possible avenues (see below). I have no idea how much of that is correct, so I'm sharing it here for inspiration (and correction, where applicable)

Option 1: Native Python VST host (complex, but elegant)

To integrate VST plugins directly within Music Assistant’s Python code, you'd need a Python binding capable of loading and processing audio through VST plugins directly. Currently, there is no widely maintained, reliable pure-Python VST host.

However, you could:

• Develop a Python-C/C++ binding:
  Use existing native plugin hosting libraries (e.g., JUCE, DPF, Carla’s backend) and expose them through Python bindings via tools like:

  • pybind11
  • ctypes (if simpler wrappers suffice)

• Use Carla's backend:

  • Carla provides a headless audio plugin host with stable APIs.
  • You’d create a Python interface that routes audio buffers between Music Assistant’s Python DSP pipeline and Carla.

Challenges:

• Significant development effort.
• Requires C++/Python interfacing expertise.
• Introduces complexity (buffer handling, latency management).

Option 2: External Process Wrapper (simpler, quicker)

This approach is quicker to prototype and easier to maintain:

• Run your VST plugin in an external headless audio host (like Carla).
• Create a Python-based wrapper that pipes PCM audio streams from MA’s DSP pipeline via a low-latency audio pipe to Carla, and back into MA.

Possible Implementation:

• Use UNIX pipes, JACK, or shared-memory buffers to route audio.
• Python modules like jack-client or python-sounddevice can interface with JACK.
• Music Assistant’s DSP pipeline sends audio data to the external Carla host, applies VST processing, and the audio returns to MA.

Pros:

• Faster to prototype (weeks rather than months).
• Minimal changes to MA’s internal architecture.
• Allows integration of any existing VST/VST3 plugin.

Cons:

• Adds latency (manageable, but must be considered).
• Slightly higher CPU overhead due to inter-process communication.

[Axolord]

I am thinking an integration of something like easy effects would be nice. Though since easy effects works in conjunction with pipewire, the core concept is likely complety different than in music assistant, so true integration is likely not possible.

But it would be nice if one could apply effects, like in easy effects (like the Calf Bass Enhancer for example, so that small bookshelf speakers can be tuned for more perceived bass.

[OzGav]

Why not use the existing DSP settings?