Subject: [linux-audio-dev] Re: my take on the "virtual studio" (monolith vs plugins) ... was Re: [linux-audio-dev] ardour, LADSPA, a marriage
From: Paul Sladen (paul_AT_sladen.org)
Date: Sun Nov 19 2000 - 18:05:22 EET
On Sun, 19 Nov 2000, David Olofson wrote:
> On Sat, 18 Nov 2000, Benno Senoner wrote:
> > but what we need is a communication system which must be very lightweight
> > even when making/destroying connections (since we want to place patches in real
> > time).
>
> How about changing a pointer or two per connection, possibly using a
> very light weight function call? LADSPA does it that way, and MAIA
> (MuCoS) won't make it heavier. (It might even eliminate the function
> call if you can settle for connecting in between buffers. There might
> be a good reason to have that kind of notification for the plugin,
> though.)
By my reckoning there are 3 types of parameters that are going to get
passed to plugins:
* State:
These are so fundemental that the plugin needs a "power cycle" (an
"end()", lollowed by "set(parameter)" calls, followed by an "end()")
changing the order of a parametic equation requires re-evaluation to
produce new co-efficants for example.
* Syncronous:
Must be updated "as part of the RT chain"...
* Asyncronous:
Can be updated anytime, but you've got no guranttee that the new
values are going to be used till the next pass.
All three are updateable from outside the begin()..end() pair via a
function call. (the call to begin effectiveally makes a move from non-RT
to RT). Only Sync and Async variables can be updated while in "RT" mode,
and Sync variables only via a function call inline within the plugin
call.. Type 3, asyncronous parameters can just be updated via a plain old
"poke" into memory, so we would need some way to publish the memory
location (relative to a particular instance of a plugin).
Therefore async variables are much better from the POV of being updated
from a non-RT server process.
> > Plus for low latency apps, we need lock-free communication between the
> > RT and non-RT parts of the app.
>
> Pointers to linked lists of events, passed via lock-free FIFOs. (No
> lock-free operations on the single event level, that is, thus keeping
> that outside the plugin API.)
The ideal is asyncronous variables that can be updated RT, once per lopp
simply through a list of "address", "datum" pairs.
>
>
> > For example a slider GUI element want to display the current volume value of
> > an audio track, which gets automation data from another source.
> >
> > In this case I want that the GUI slider gets all new volume values from the
> > audio app, to reflect the "motorized-fader" effect.
Perhaps a list of "address", "message name" pairs so that the RT process
could grab each of these values of pack it up and add it to the lock-free
buffer as a message to be forwared asyncronously to the reciveing [GUI]
element.
> >
> > With a lock-free fifo this would be ideal since even if the GUI element is non
> > RT, it does not loose any data opposed to the model where you are simply using
> > a shared variable.
>
> Nice chance of implementing some sensible filtering algo, that is.
>
>
> > (Plus the shared variable will not work with datasets > 32bits because of the
> > atomicity problems)
>
> (Not even 32 bits are safe on all archs, as we have seen.)
This solves the problem for single datum out/input, but I don't know how
it would work with arrays... the output from a Graphic-Eq par exampla.
>
> > The same must work in reverse order (a slider controlling a volume value):
> > use the same fifo approach.
> >
> > If I want network transparency, I simply write a "proxy-element" which
> > has as input the lock-free fifo and as output a network connection or vice
> > versa.
>
> Exactly.
The key to all this is the asyncronous (non-RT) messageing server that
runs and communicates, passing all the messages around.
>
> > I admit that this asynchronous communication method is not easy to deal with
> > (even on the same machine) since receivers need to be woken up or do active
> > polling.
We just need to provide a LADPA-GUI support library that will contact the
local messageing server, and block until the reply (hopefully) arrives,
then reurn to the call with the value requested.
> >
> > Having investigated the speed of IPC methods (I got some advices from kernel
> > guys too), the only thing you can tolerate is to write a byte to a unix pipe,
> > where the receiver gets woken up after sleeping in a blocking read.
>
> Hiding this (or some other solution, like a kernel module) in an API
> library for clients to use should work. (Note: This does not apply to
> MAIA plugins even though the "talk" the same kind of events; see
> above.)
>
>
> > And in order to avoid zillions of calls to write() within the RT audio thread,
> > we should delegate the wakeup of receivers to a butler thread which gets woken
> > up by the RT thread and gets informations about whom to wake up using shared
> > mem.
> > That way the RT thread can send data to a huge amount of receivers without
> > fearing a dropout.
>
> Something like that, yes... (A kernel module would be nicer in away,
> but unless it really provides better performance, it's probably not
> motivated. Unless of course, we start talking about security on
> multiuser systems...)
Hum getting vibes of non-elegance if we're talking about kernel modules
just to implament your messageing API...
I think the future lies in having as much of the setup/processing done in
non-RT space as possible, The maximum that the RT stuff should have to do
is to talk to the sound card, and talk to a pair of lock-free FIFOs.
> > Handling the sending of data from non-RT elements to RT-elements is easier,
> > since all the non-RT element has to do is to write to the lockfree-fifo.
> > The RT-element will poll for new data at every processing iteration.
> > (in the case of audio each time a fragment is played).
>
> Yep.
>
> > Now my question are MCOP or libsigc++ capable of such things ?
> > (I mean: lock-free async communication plus VERY lightweight functions to
> > to connect/disconnect senders with receivers).
> >
> > If these APIs are not suitable for this, then we have either to change them or
> > write one from scratch.
> >
> > David O. how would MAIA compare with MCOP/libsigc++ in that area?
>
> I'm not sure I know enough about MCOP or libsigc++ to comment on
> their implementations, but I think the above covers my version. As to
> the API, MAIA differs basically in two ways from RPC-like systems:
>
> 1) You simply send an event; you don't wait for the
> destination to get it, deal with it, and return a result.
>
> 2) You timestamp events to be able to control exactly when
> the destination will get the event.
>
I'm not sure we need this, the delays we are talking about are stupidly
small anywhere, and just think abou the delay that the mouse-interuppt
handler is going to cause operating at "only" 1200baud.. I this particular
moment can't think of anything that needs more than just "update this as
fast as you can if you could... and keep me posted". I'm obviously not
thinking about the right things.. if someone could give me an example to
enable me to start thinking around this one.
>
> > Without these characteristics, my "virtual studio" idea cannot be implemented.
> >
> > (You know: all "applications" runs as "plugins" of a main host, which
> > never never never blocks (*) (mark the last 4 words .... ) )
>
> The MAIA model assumes that it's ok to send events from plugins as
> well as from applications outside the engine, or from plugins in
> other engines on other machines, so no problem there.
Starting to add (perhaps unnessecary?) complexity. Think about debugging
these type of asyncronous multi-threaded realtime. I recall RMS having
talking bits about GNU Herd. I'm not sure that we want to go quite this
far... nor have the ten years in which to get it working. just my
0.02ukp...
> I don't know about MCOP or libsigc++ in that respect though; is it
> possible to do "decoupled" (non-blocking) operations, even if the
> target happens to be in the other end of countless meters or cable?
>
>
> > (*) audio I/O is the exception but we all know that this provides a good
> > determinism. (see low latency tests))
>
> Well, that's the time base that drives the host. (*Truly*
> non-blocking hosts always do nasty things to my machines... ;-)
The only thing that I can see that should ever block is the plugin reading
from the audio card, and awaiting for the next "packet" to come through.
Paul
-- e: paul_AT_sladen.org t: 0115 922 7162
This archive was generated by hypermail 2b28 : Sun Nov 19 2000 - 18:48:59 EET