Subject: Plugin chains and EDLs, was Re: [linux-audio-dev] ardour, LADSPA, a marriage
From: Paul Winkler (slinkp23_AT_yahoo.com)
Date: Mon Nov 13 2000 - 19:00:51 EET
David Olofson wrote:
>
> On Mon, 13 Nov 2000, Paul Winkler wrote:
> > protocol... hmmm... MCOP? (I think that's the name of the protocol in
> > aRts?)
>
> The problem with most solutions mentioned in this thread: They result
> in too much latency for serious real time use. This may or may not be
> problematic to what some people are doing, but people who have used
> ProTools, Paris and other hardware based solutions are expecting
> latencies to be virtually zero for most practical matters, ie less
> than 5 ms.
Hmmm, yes,that would be an issue for me, probably sooner rather than
later.
> The only solution that works on any currently existing OS, except for
> hard RT kernels, such as QNX, RTAI or RTL, is to run everything in a
> single thread. Chains of processes are not possible with these kind of
> latency requirements, so the only option for a modular system is a
> host/plugin design, where the host application loads plugins as
> libraries and executes them through direct calls from it's engine
> thread. That is, LADSPA, VST, MuCoS, or whatever plugin API will be
> more suitable and mature when it's time to hack the code.
OK. let's look at the situation today with LADSPA. Is it / will it be
up to the job I have in mind? If not, this is a big stumbling block
for this virtual linux studio I and probably many of you are dreaming
of ... but there are possibilities beyond anything I've seen in any
software free or commercial.
In my message last night, which I wrote when really too tired and
incoherent, I first proposed building a mixer entirely out of LADSPA
plugins, and then deleted it, thinking it was too messy of a project
for a Simple Plugin Architecture. But if plugins are the only good way
to get this job done, I have to ask, can this be done with LADSPA?
From a user point of view, here's what I'd like: A mixer at least good
enough to be used for monitor mixing while tracking with Ardour.
(Leaving aside mixdown for now... I'm still tending to think that
could be done with a separate application.) Hopefully it would be
completely user-configurable. Could we patch together such a thing
with LADSPA plugins?
This raises two issues:
1) Could Ardour be a smart enough host to handle the patching? At this
point I tend to think that pbd can do anything, so I'm going to assume
yes. :) It's more a question of what pbd wants.
2) I had the impression that LADSPA is mostly intended for
single-input / single-output plugins, but a mixer has several
sub-sections where inputs != outputs.
Hmmm, browsing the CMT part of the LADSPA site I see several plugins
where inputs != outputs. OK.
Let's look at the requirements for a mixer. In a *very* simple but
still useful mixer, let's say we have N inputs, M bus outputs, and no
aux outs (just for simplicity for sake of this discussion).
Each input channel is simple at first - it's just a linear chain.
But then you hit bus assignment and/or panning. Let's leave out
panning for now (again for sake of discussion). We have reached a part
of the chain where we need to select which bus(ses) will get the
output of this channel. THis is pretty simple, it's just a switch
really; question is whether to do it with the host or with plugins?
The latter would be probably best in the long run from a user point of
view, in terms of flexibility.
Then there are the busses. Each is functionally an N-input, 1-output
adder. You could either do this by patching together 2-input, 1-output
adders (like the CMT mixer) until you reached N (simple for developer
but messy for the user if this is visible to the user... and requires
N -1 plugins). Or you'd need a plugin which had a configurable number
of inputs, selectable at startup time (no, I don't expect to add
channels to my mixer while running it). That would be cleaner from a
UI perspective, useful in a *lot* of situations, and I would think
more efficient (for example I assume having 23 2-to-1 plugins adds
overhead more than one big 24-input plugin).
THis leads me to another thought. Every application is going to want
to have a way to save descriptions of plugin networks so you can
re-create your session later. Maybe saved as XML for example. A
complete mixer setup could be saved as one such description. What
would be REALLY REALLY COOL would be if different applications could
read and write the same description.
For example, if I'm using snd and I discover a chain of effects that
does something really, really useful to me, I'd like to easily re-use
that same chain in ecasound, or ardour, or a video production suite or
whatever. From a user POV it should be as easy as
1) somehow select which part of your signal processing chain you want
to save
2) go to a menu, click "Save plugin setup",
3) go to my other app,
4)choose where I want to insert this chain
5) and then click "Insert plugin setup".
1 and 4 would depend entirely on the app of course.
Obviously none of this belongs inside of LADSPA at all. In fact these
same ideas could apply to VST or whatever. But since each host is
going to have to deal with these issues (at least if to decide not to
deal with them), we really should think about general solutions.
One question is: are plugin chains or networks just part of an edit
decision list? Or is it a separate issue?
To me they're conceptually separate, but I suppose a plugin setup
could just be saved as an EDL with no edits.
It's worth considering whether to think of a plugin chain as an object
that can be encapsulated *within* an EDL, and to be able to use them
independently. For example I'd like to be able to load in a chain of
plugins from another session, inserting it somewhere in my current
session without affecting any of my other edits in the current
session.
For that matter, it would be even more flexible if chains could
encapsulate other chains. So you could design several different kinds
of channel strips and describe a mixer at a higher level just by
saying it contains 6 fancy_channels, 12 simple_channels, etc.
This *could* all be handled by EDLs if the EDLs were sufficiently
smart about encapsulating other EDLs, but I still wonder if there's
something to be gained by separating the issues.
Of course for such plugin network descriptions to be portable, the
hosts supporting it would have to provide a common base of equivalent
patching functionality. So we'd have to hash out what that should be.
There could be various levels of support. Level 1 would be very simple
- just single-channel series chains. Everything above that would take
some work to define and agree on.
The possibilities here are quite exciting. My computer music
background is mostly in csound where we can save collections of
low-level building blocks (unit generators) in a higher-level module
(instrument) inside a yet higher-level module (orchestra). But it's
frustrating that these levels of abstraction are fixed and completely
inflexible. You can't wrap up an instrument as a new unit generator,
or wrap up an orchestra as a single instrument within another orc.
When you compare this to the flexibility of any decent programming
language, it's as if you said that functions cannot contain calls to
other functions. The mind boggles at how much extra work that would
create.
Now look at it the other way around. If a network of plugins can be
thought of as a discrete object; if networks of plugins can contain
sub-networks of plugins and keep the logical distinction between them;
if these networks can be used by a number of different applications...
I'm thinking this could be a step forward comparable to the
introduction of the plugin concept itself.
--
................. paul winkler ..................
slinkP arts: music, sound, illustration, design, etc.
web page: http://www.slinkp.com
A member of ARMS: http://www.reacharms.com
This archive was generated by hypermail 2b28 : Mon Nov 13 2000 - 20:07:08 EET