Subject: Re: Plugin chains and EDLs, was Re: [linux-audio-dev] ardour, LADSPA, a marriage
From: David Olofson (david_AT_gardena.net)
Date: Tue Nov 14 2000 - 12:33:39 EET
On Mon, 13 Nov 2000, Paul Winkler wrote:
> David Olofson wrote:
> >
> > On Mon, 13 Nov 2000, Paul Winkler wrote:
> > > protocol... hmmm... MCOP? (I think that's the name of the protocol in
> > > aRts?)
> >
> > The problem with most solutions mentioned in this thread: They result
> > in too much latency for serious real time use. This may or may not be
> > problematic to what some people are doing, but people who have used
> > ProTools, Paris and other hardware based solutions are expecting
> > latencies to be virtually zero for most practical matters, ie less
> > than 5 ms.
>
> Hmmm, yes,that would be an issue for me, probably sooner rather than
> later.
>
> > The only solution that works on any currently existing OS, except for
> > hard RT kernels, such as QNX, RTAI or RTL, is to run everything in a
> > single thread. Chains of processes are not possible with these kind of
> > latency requirements, so the only option for a modular system is a
> > host/plugin design, where the host application loads plugins as
> > libraries and executes them through direct calls from it's engine
> > thread. That is, LADSPA, VST, MuCoS, or whatever plugin API will be
> > more suitable and mature when it's time to hack the code.
>
> OK. let's look at the situation today with LADSPA. Is it / will it be
> up to the job I have in mind? If not, this is a big stumbling block
> for this virtual linux studio I and probably many of you are dreaming
> of ... but there are possibilities beyond anything I've seen in any
> software free or commercial.
>
> In my message last night, which I wrote when really too tired and
> incoherent, I first proposed building a mixer entirely out of LADSPA
> plugins, and then deleted it, thinking it was too messy of a project
> for a Simple Plugin Architecture.
On the contrary, this is the way I want the mixer in my new digital
studio to be built. I can't see any serious technical problems with
it.
> But if plugins are the only good way
> to get this job done, I have to ask, can this be done with LADSPA?
Sure, LADSPA should work for most things, and I can't see why you
would use a more complex API with more overhead for the majority of
the parts of a mixer.
(There *might* be an issue with extreme forms of mixer automation,
though, but I'd think it's rather unusual to modulate mixer controls
with higher frequencies than 100 Hz or so... No point in optimizing
the entire system for the extreme cases - that kind of stuff belongs
in modular synths.)
> >From a user point of view, here's what I'd like: A mixer at least good
> enough to be used for monitor mixing while tracking with Ardour.
> (Leaving aside mixdown for now... I'm still tending to think that
> could be done with a separate application.) Hopefully it would be
> completely user-configurable. Could we patch together such a thing
> with LADSPA plugins?
>
> This raises two issues:
>
> 1) Could Ardour be a smart enough host to handle the patching? At this
> point I tend to think that pbd can do anything, so I'm going to assume
> yes. :) It's more a question of what pbd wants.
>
> 2) I had the impression that LADSPA is mostly intended for
> single-input / single-output plugins, but a mixer has several
> sub-sections where inputs != outputs.
Can't see a problem - LADSPA supports all kinds of configurations me
might need, unless I've missed something...
What *might* be a problem is dynamic numbers of ins/outs. (Required?)
> Hmmm, browsing the CMT part of the LADSPA site I see several plugins
> where inputs != outputs. OK.
Yep.
> Let's look at the requirements for a mixer. In a *very* simple but
> still useful mixer, let's say we have N inputs, M bus outputs, and no
> aux outs (just for simplicity for sake of this discussion).
>
> Each input channel is simple at first - it's just a linear chain.
> But then you hit bus assignment and/or panning. Let's leave out
> panning for now (again for sake of discussion). We have reached a part
> of the chain where we need to select which bus(ses) will get the
> output of this channel. THis is pretty simple, it's just a switch
> really; question is whether to do it with the host or with plugins?
> The latter would be probably best in the long run from a user point of
> view, in terms of flexibility.
It's just a matter of using plugins that can do adding processing
with output gain control. Connect all outputs to the same buffer, and
there's your bus! :-)
As to pure switching/bus selection, it's just a batter of connecting
the plugins to the buffers representing the desired busses. (The same
goes for inputs tapping audio from busses.)
> Then there are the busses. Each is functionally an N-input, 1-output
> adder.
Not really. The bus is actually just a buffer where a bunch of
plugins have mixed down all of their output. Throw it at any plugin
input for further processing.
> You could either do this by patching together 2-input, 1-output
> adders (like the CMT mixer) until you reached N (simple for developer
> but messy for the user if this is visible to the user... and requires
> N -1 plugins).
Well, you'll have to do this for plugins that don't support adding
and the standardized output gain control.
> Or you'd need a plugin which had a configurable number
> of inputs, selectable at startup time (no, I don't expect to add
> channels to my mixer while running it). That would be cleaner from a
> UI perspective, useful in a *lot* of situations, and I would think
> more efficient (for example I assume having 23 2-to-1 plugins adds
> overhead more than one big 24-input plugin).
The visual abstraction may be anything you like, but it's actually
just a buffer and the settings of a bunch of plugins we're talking
about here; not a plugin of it's own.
> THis leads me to another thought. Every application is going to want
> to have a way to save descriptions of plugin networks so you can
> re-create your session later. Maybe saved as XML for example. A
> complete mixer setup could be saved as one such description. What
> would be REALLY REALLY COOL would be if different applications could
> read and write the same description.
Yes, that sounds very interesting... :-)
Besides, starting out with a standard like that, the next thing might
be applications that are actually only user interfaces/net editors for
other engines. That is, throw your net description and perhaps a
bunch of special plugins at any standard compliant engine, and fire
up. No need to hack your own engine.
Advantage: Multiple applications can use the same engine at the same
time without latency problems.
> For example, if I'm using snd and I discover a chain of effects that
> does something really, really useful to me, I'd like to easily re-use
> that same chain in ecasound, or ardour, or a video production suite or
> whatever.
Why not just having snd publish the chain in question, so that other
applications connected to the engine could connect to it right away?
That may, you can tweak the chain in real time without a stop A/save
from A/load from B/start B cycle.
> From a user POV it should be as easy as
> 1) somehow select which part of your signal processing chain you want
> to save
Rip the chain out and/or mark the connections that you want to make
public to the system.
> 2) go to a menu, click "Save plugin setup",
(do nothing)
> 3) go to my other app,
Yep!
> 4)choose where I want to insert this chain
Select the public connections, and connect them somewhere in you net
- the result will be as if the chain was set up by this application.
> 5) and then click "Insert plugin setup".
(do nothing)
> 1 and 4 would depend entirely on the app of course.
Yes; how to deal with public connection points.
> Obviously none of this belongs inside of LADSPA at all.
Nope, it should work with any plugin API, I think. The plugin UI
issues may get messier though, but these problems basically exist
anyway. It's just a matter of going from interthread communication
to interprocess communication.
> In fact these
> same ideas could apply to VST or whatever. But since each host is
> going to have to deal with these issues (at least if to decide not to
> deal with them), we really should think about general solutions.
Yeah, this is the messy part... :-)
> One question is: are plugin chains or networks just part of an edit
> decision list? Or is it a separate issue?
> To me they're conceptually separate, but I suppose a plugin setup
> could just be saved as an EDL with no edits.
Well, yeah, provided the *EDL* actually cares about what you insert.
Why not just make it deal only with connections, and have a different
layer deal with the routing (single plugins, chaing, public
connections etc) of these connections?
> It's worth considering whether to think of a plugin chain as an object
> that can be encapsulated *within* an EDL, and to be able to use them
> independently. For example I'd like to be able to load in a chain of
> plugins from another session, inserting it somewhere in my current
> session without affecting any of my other edits in the current
> session.
I think EDLs and chain/net descriptions should be considered two
strictly separate things. EDLs see chains, nets, external stuff etc
as black boxes, just as if they were all single plugins.
Hmm... How about "virtual plugins"?
That is, plugins that are actually descriptions of nets, rather than
real plugins. The logic behind that is that using this "chain manage
layer", everything is a chain, and all installed plugins (possibly
using different APIs) are registered as chains. Just pick and chose.
> For that matter, it would be even more flexible if chains could
> encapsulate other chains. So you could design several different kinds
> of channel strips and describe a mixer at a higher level just by
> saying it contains 6 fancy_channels, 12 simple_channels, etc.
Solved automatically with virtual plugins.
> This *could* all be handled by EDLs if the EDLs were sufficiently
> smart about encapsulating other EDLs, but I still wonder if there's
> something to be gained by separating the issues.
I don't think EDLs have got anything to do with this. They're
supposed to deal with edit desicions, not the details of how applied
effects are actually implemented.
> Of course for such plugin network descriptions to be portable, the
> hosts supporting it would have to provide a common base of equivalent
> patching functionality. So we'd have to hash out what that should be.
> There could be various levels of support. Level 1 would be very simple
> - just single-channel series chains. Everything above that would take
> some work to define and agree on.
Yep, especially if it's supposed to support more than one plugin API.
No problem for applications that don't host plugins themselves, but
engine implementations would be rather messy.
However, plugins hosting plugins is one way to sort things out...
Could be the way, at least for "normal" hosts. (A generic "standard"
host for remote control from host-less applications would probably
use a more optimized approach.)
> Now look at it the other way around. If a network of plugins can be
> thought of as a discrete object; if networks of plugins can contain
> sub-networks of plugins and keep the logical distinction between them;
> if these networks can be used by a number of different applications...
> I'm thinking this could be a step forward comparable to the
> introduction of the plugin concept itself.
Yes. This is exactly the way I'd like, although, I'd also like it all
to work in real time, without moving files around. :-)
//David
.- M u C o S -------------------------. .- David Olofson --------.
| A Free/Open Source | | Audio Hacker |
| Plugin and Integration Standard | | Linux Advocate |
| for | | Open Source Advocate |
| Professional and Consumer | | Singer |
| Multimedia | | Songwriter |
`-----> http://www.linuxdj.com/mucos -' `---> david_AT_linuxdj.com -'
This archive was generated by hypermail 2b28 : Tue Nov 14 2000 - 14:01:15 EET