Subject: [linux-audio-dev] Thoughts on VST, LADSPA etc
From: Richard W.E. Furse (richard_AT_muse.demon.co.uk)
Date: Mon Apr 03 2000 - 17:54:33 EEST
Hi there - here's a fairly random collection of thoughts on VST, LADSPA
etc. It's a bit long - sections (3), (4) and (7) are particularly core
IMHO.
(1) VST
(1.1) Originally the VST plugin SDK was written specifically to handle
plugin effect processors. Since then it has been extended to handle simple
synthesisers. However both effects processors and synthesisers are special
cases of the unit generator. For the purposes of this text, I use unit
generator in the Max Matthews/Csound sense and include event processing
within its capabilities.
(1.2) IMHO, VST has a good plugin API for effects processors and an okay
plugin API for MIDI synthesisers. But not for unit generators in general.
The biggest problem I have is the absence of control outputs on plugins -
it isn't even possible to build a control-rate envelope generator in VST.
Several other historical features are somewhat unfortunate, particularly
the rather clunky control input procedure (that will become very ineffic
ient when fed a high rate control signal) and the [-1, +1] and [0, +1]
constraints on audio and control data. It's also rather bulky for my
tastes, probably the result of piecewise development.
(1.3) VST has a large number of plugins that could be compiled for Linux
without too much trouble. This is an enormous benefit that cannot be
overstated.
(1.4) VST has potential licensing issues for source distribution but I
suspect these could be got around with a little discussion.
(2) LADSPA
(2.1) LADSPA is designed from the ground up as a plugin API for unit
generators (although explicit use of this terminology was stripped out at
an early stage). As a result, IMHO it does a better job at handling the
'general case' but isn't so good at the specialised cases, specifically the
synthesiser, because of the absence of event support.
(2.2) LADSPA2 could be generated by extending LADSPA1 just to include some
kind of very simple event mechanism, thus providing a logical superset of
VST SDK2 functionality. I'm not yet sure this is appropriate. Also see (4)
below.
(2.3) I think people should be welcome to extend LADSPA - perhaps with a
comment along the lines of "system <xyz> is a [backward compatible?]
extension of LADSPA version <n>, Copyright <etc>, available from
http://www.muse.demon.co.uk/ladspa.html." This allows people to play around
with less simplistic models of plugin behaviour but allows other folk
examining those implementations to find their way back to the simple
version. Personally I think that the strength of LADSPA as a plugin API
lies in its simplicity, but I can see other viewpoints and I'm happy for
people to experiment, as long as they use the LADSPA name responsibly and
avoid confusion or source tree branching.
(2.4) An observation about plugin libraries in general - these are
implemented as shared libraries. A shared library may contain more than one
way to access the code within it - there's no reason why a VST plugin
should not also have a ladspa_descriptor() function available. Also, if the
LADSPA descriptor structure is extended and renamed (e.g. for Benno's MuCoS
extensions), two functions could be provided: ladspa_descriptor() and
mucos_descriptor(). These could return the same structure, however LADSPA
hosts will only look at the first (LADSPA) part of the structure while
MuCoS hosts would see the full interface. This then allows the host to pick
the approach that suits it best. This approach could become somewhat
unpleasant for the plugin developer.
(3) Audio/Event Bus
(3.1) Linux currently does not have any standard audio/event bus facilities
that I'm aware of. Audio/event buses would allow communication between
hosts (perhaps including Cubase). Event (including MIDI) and audio data
could be streamed between running programs. ALSA could possibly be extended
to include audio buses, but this might turn out to be a bit of a corruption
of design. I don't know - I'm not that familiar with ALSA's design.
(3.2) An audio bus requires a tight negotiation of sample rates, block
sizes, data format etc. This just requires careful specification as not
that many format types will be required. Mechanisms could include IPC or
sockets, ATM etc in straightforward fashion.
(3.3) Event buses suffer an additional problem as event data needs to be
garbage collected. A possible mechanism could use IPC message queues,
shared memory and reference-counting locally and sockets and data copying
remotely. However although the client can be protected from the mechanism,
it will have to know how to interpret the event data itself. See (4)
(3.4) The audio/event transmission mechanism does not need to be specified
in the API itself - a library (and perhaps a hub process/kernel module?)
could be provided.
(4) Common Event Types
(4.1) In this text I use 'event' to refer to time-stamped data. I do not
assume a concept of duration.
(4.2) For different applications to share event data, they need to be able
to interpret that event data uniformly. Perhaps before looking at LADSPA2
or event buses we should consider a common registry for event types. Event
type IDs could be allocated on request from a central (automated?) body, as
we discussed for LADSPA plugin IDs.
(4.3) Events could have a simple single-block format starting out with a
type_id and size - and could then be implementation specific. Alternatively
they could use more complex data structures and provide plugins containing
read/write methods to handle persistence/neutral transmission. The latter
is how MN handles events but is probably somewhat over-the-top for a simple
plugin API.
(4.4) Having a common event registry could allow events to be exchanged
freely between LADSPA, MuCoS, VST plugins and event buses, let alone the
many hosts that could support them directly or wrapped.
(4.5) VST SDK2 has a crude concept of event type. Perhaps Steinberg would
let us hijack this.
(5) What do we want? [Okay okay, what do *I* want ;-)]
(5.1) Cubase running on Linux with our plugins running within it. Given the
former, the latter will follow if ANY sensible API is provided, so this is
probably not a problem (subject to 1.4).
(5.2) Access to the vast VST plugin library. This follows if the VST plugin
API for Linux is reasonably compatible or Windows developers are encouraged
to do ports.
(5.2) A plugin interface that can be used to share code between most Linux
hosts. In my opinion, the current VST plugin API is *not* adequate for this
so I think LADSPA needs to stay for the moment. Just as an example, one of
the unit generators on my to-do list for MN is a frequency tracker. This
would take an audio stream and apply cepstrum techniques to it to find a
fundamental frequency. The output would be a frequency control output. I
cannot construct this using VST as is - I'd have to use an audio output. I
can within LADSPA or MN - or most of the other unit generator based hosts
out there.
(5.3) An audio/event bus so that applications can work together.
(5.4) To achieve 5.2 and 5.3 nicely I think we need a common event type
pool.
(6) What do Steinberg want? [Guesswork]
(6.1) Market share.
(6.2) They hopefully won't want to write their own audio device drivers. We
probably should point them towards ALSA rather than OSS?
(6.3) They probably will want to be able to talk to other Linux
applications, particularly software synthesisers. For this they will need
an audio/event buses - which they probably don't want to write themselves
although if push comes to shove they could simply use a couple of
audio/MIDI devices and let other people connect to these - and porting
ReWire would probably be pretty trivial but proprietary. Does anyone know
what they did (if anything) for BeOS/IRIX?
(6.4) Innovation. (Well, I'm hoping they do - they certainly have a history
in this direction!) Hopefully they'll be interested in sensible extensions
to Cubase that increase its usefulness. I don't know what graphics
libraries they used for BeOS and IRIX ports, but I cannot help but suspect
they will have to do quite a bit of work to do a Linux port. Within this
budget I'd hope they have some flexibility to look at interface changes,
particularly given help they don't have to pay for! Also, there are an
awful lot of people doing "weird stuff" (i.e. innovation) on Linux and I'm
sure they would like these people on their side both technically and
politically.
(7) Some ideas for Things we can Do, ideally all in conjunction with
Steinberg
(7.1) Spec out an audio/event bus. Perhaps the ALSA people would like to
manage this although the bus probably shouldn't be part of ALSA itself.
(7.2) Spec out a common event type approach. I'm happy to do this - I think
a simple approach could be fairly trivial.
(7.3) Spec out a proper unit generator plugin API that covers both unit
generator requirements (i.e. proper control ports) and the event support
required by synthesisers etc. This can be done by extension/rewrite of the
VST API. It could also be done by persuading Steinberg to support LADSPA
plugins and adding very simple event support (or using a LADSPA-alternative
e.g. MuCoS if richer functionality is required). Otherwise MN will almost
certainly support both LADSPA1 and VST2 plugins - this is not too difficult
although the VST events will be a bit of a pain. I'm sure other hosts could
handle this too - but it would be much nicer to bring things more together.
Oof. That'll do for now. What do people think? Am I making any sense at
all?
-- Richard
This archive was generated by hypermail 2b28 : Mon Apr 03 2000 - 18:48:31 EEST