Subject: Re: [linux-audio-dev] plugin ideas based on Guenter's mix plugin API
From: David Olofson (audiality_AT_swipnet.se)
Date: pe elo 27 1999 - 19:08:57 EDT
On Fri, 27 Aug 1999, est_AT_hyperreal.org wrote:
> > > * The only global defined by a plugin should be a uniquely named
> > > function that returns a plugin class. The idea here is to make it
> > > possible to use static as well as dynamic linking.
> > >
> >
> > .. hmmm seems to make sense ..
>
> I'm not particular about the exact interface, as long as static
> linking isn't precluded.
And this is about as far from the main execution paths as we'll ever get, so it
doesn't matter much how it's done, as long as it's nice and does the job well.
> > > * Change properties to a * int32_t to avoid binary compatibility concerns.
> > >
> > > * Change sparebytes to a void * for the same reason.
> >
> > Yep, dynamic allocation, but then, if we want to access (some later added)
> > properties we always have to check if the plugin has allocated memory
> > for them.
>
> You've got that covered by numproperties.
With out-of-band events (that is, events with no time stamps - handled by the
reciever ASAP) for requesting and setting properties, this is reduced to a
matter of plug-in implementation. The host just asks for a list of properties,
and can then access them safely through the event system. Bonus: Full
automation for free; just record/playback the interesting event streams...
> > > * The number of channels per input and output signal should be a
> > > plugin property vector. Counts of frames and samples should be
> > > clearly distinguished throughout.
> > >
> >
> > Basically I'd like to have each channel on a seperate input.
> > We have to set some restrictions, as through flexibility we
> > will make it harder to write applications for the plugins,
> > and introduce too much incompatibility.
>
> I want multi-channel support pretty badly. I certainly appreciate the
> flexibilty of a mono-channel paradigm, but I think there are
> situations where multi-channel makes sense. For example, scope
> plugins will often want stereo input. Kai has also pointed out some
> of his needs in this area. Further, I've found that stereo data is a
> sitting duck for hefty 3dnow acceleration.
Yeah, doing SIMD in that direction (as opposed to processing multiple
sequential samples at once) eliminates the dependency problems...
> It doesn't seem reasonable
> to make plugins that want multi-channel input do their own merging
> when merging/splitting can be trivial standard plugins.
Point taken. With the bus and memory speeds of modern workstations, the
conversion is pretty low cost. It does mean extra process() function call
overhead, which starts to get significant with very small buffers, but then
again, you could probably use inline code in the engine instead... (Macros in
the host side headers?)
> One restriction that no one might find objectionable is that the input
> signals and output signals of a plugin both be homogeneous with
> respect to the number of channels. Then you only need inchannels and
> outchannels parameters for the plugin. If you want mono, just make
> sure these are both 1 (which could be the default).
I can't fully agree here. What about a stereo compressor with a mono
side-chain? And when considering modular synthesizers, there are control
signals as well, and even though most "modules" would be mono + mono control
signals, the control signals should probably be allowed to have lower sample
rates. That is, signal_fs = audio_fs/n, where n is a power of 2. (Nothing else,
as that would only mean slow conversion code in the plug-ins.)
So, I think there might be a point in allowing mixed signal formats, Rather
than forcing inherent stereo plugs to dual mono just because the want some
independent control signals as well.
> > > * The sample rate should be optionally settable at instance creation
> > > time and the input and output buffer sizes should be optionally
> > > settable per call to the process method. I usually code in a way that
> > > can accomodate this, but I agree that it's important to support less
> > > flexible plugins as well.
> >
> > It's possible to set these things whenever you like.
> > The plugin programmer can decide which properties can be changed,
> > and which not. That's what the virtual verify function is all about.
>
> Hmm..I'm wondering if there should be a harder distinction between
> properties and parameters re mutability.
I think there should be. Some things just can't be changed in a sane way without
destructing and re-creating the plug-in instance.
However, I do think the sample rate is one property that should be possible to
change while processing. Plug-ins aren't required to handle it perfectly, but
they should accept it, and recalculate filters and such accordingly.
For this and other similarly brutal events to work properly in a low latency
environment, there should be a way of switching the plug-in out of real time
context until it's done with the recalculations. If you have a plug-in that
needs to do some nice work on a 2048 sample internal buffer, while the
process() buffer size is 16 samples... BANG! Drop-out!
The plug-in should have a way of telling the engine
"I can't sort this request out in the time it takes to
process a buffer! Please call function offline_recalculate()
for me in your lower priority thread, and send me a message
with it's return code when it returns."
The plug-in should do something sensible in subsequent process() calls, until
it recieves the message after offline_recalculate()'s return.
BTW, do you know how Cakewalk handles this? Well I do: It doesn't. Any time a
plug-in does something like that, or you insert or remove a plug-in, the entire
engine STALLS until the change has been performed! Useless... Stand-alone units
don't do that to the mixer. And games don't do that when enemies appear or die.
> > > * To handle resampling, variable output buffer sizes are important.
> > > It's also important that the process method can report how many output
> > > frames it generated (is this the function of the return value of the
> > > process method?). It may even be important that the plugin can
> > > be queried as to how many output frames *will* be generated to avoid
> > > overruns due to disagreements about fencepost issues.
> >
> > Isn't it enough to have different sizes for input and output buffers ?
>
> I take back most of this and I think the issue will need some careful
> thought, especially considering David Olofson's comments.
I think specifying the number of extra samples required in the input buffers
cover most situations, but it's still limited in that you can't just decide to
change those figures at any time. The engine may need to catch and recalibrate
for inherent plug-in latencies through large parts of the processing net before
the plug-in in question can get those extra samples. Specifying a maximum
look-ahead value, and then not change it seems reasonable in most cases. But it
may add extra latency for no obvious reason in cases where a plug-in may get
very long internal delay to compensate for with some parameter settings.
> > > * The popup method should probably be expanded to a vector of generic
> > > gui methods.
> > >
> >
> > As few as possible, but right, we may need more .....
>
> Maybe config_dialog(), show(), hide() and a gui type string?
And here I come again with chainsaw running and Doom-style expression on face!
;-)
Put this stuff in a parallel API. Why do you want it in the same file as the
processing code? You're asking for sync problems...
Why? The processing code will most probably not be called from the same thread
as the user interface code, which means any data access *must* be both thread
and SMP safe! That's not nice code to have inside plug-ins... It belongs in OS
kernels and/or IPC libraries. Or what are you expecting to do from within these
GUI functions, actually?
I'd suggest hacking another API for native code GUIs. The GUI should be aware
of plug-in instances, and communicate with the processing code using this event
system I'm raving about all the time.
Cool bonus: You can have a GUI module that's aware of all instances of the
plug-in class, so you could hack a nice VU-meter bridge, or a multichannel
analyze/control tool that can pick up and affect signals all over the
processing net without the user getting lots of little silly windows all over
the screen.
My GUI system will probably be based on HTML-like markup files (possibly
compiled as an alternative for those nervous closed source guys), with custom
widget plug-ins where needed. That is, you get to use custom plug-ins like the
normal components, telling them where to show up from the markup files. Custom
widgets will basically be a platform independent abstraction for a basic canvas
(drawing surface) widget, so that you can do the usual stuff without digging
deep into the current platform's native graphics subsystem. Any suggestions for
a fast and nice canvas widget to rip out and use for this? (Could use X
directly, but there are those other OSes... Although I don't care much for
most of them.)
Finally, separation is a Good Thing (TM). Even when you don't have to face
sync problems or split environment systems, it has many advantages. Why do
everybody seem to want to go elsewhere?
//David
·A·U·D·I·A·L·I·T·Y· P r o f e s s i o n a l L i n u x A u d i o
- - ------------------------------------------------------------- - -
·Rock Solid David Olofson:
·Low Latency www.angelfire.com/or/audiality ·Audio Hacker
·Plug-Ins audiality_AT_swipnet.se ·Linux Advocate
·Open Source ·Singer/Composer
This archive was generated by hypermail 2b28 : pe maalis 10 2000 - 07:25:53 EST