Subject: Re: [linux-audio-dev] plugin ideas based on Guenter's mix plugin API
From: David Olofson (audiality_AT_swipnet.se)
Date: pe elo 27 1999 - 20:50:03 EDT
On Fri, 27 Aug 1999, est_AT_hyperreal.org wrote:
> David Olofson discourseth:
> >
> > > * The only global defined by a plugin should be a uniquely named
> > > function that returns a plugin class. The idea here is to make it
> > > possible to use static as well as dynamic linking.
> >
> > Nope. For kernel modules, it only makes sense to export init_module()
> > and cleanup_module(), and init_module() has to
> > register_plugin(&audioeff_class) to tell the engine about the new class.
> > This eliminates name space conflicts, as no symbols are exported. Also,
> > a module/library can register multiple plug-in classes this way.
>
> Ah..but that still precludes static linking.
Ok... I didn't think much, obviously.
It is possible to exclude init/cleanup_module(), so that the object files
don't export anything that would cause name space conflicts. (Not sure if
that's really allowed, though...)
But then the kernel space engine will have to find this uniquely named
function in some way... That is, searching the kernel symbols after the module
has been loaded. Not nice, but possible. A kernel hack shouldn't be needed,
AFAIK.
Just one problem with this unique name... How do we keep track of reserved
names? Or what do you do when you want to link in two proprietary binary-only
plugs, and there's a name conflict? Hmm... Hack a file that you link with one
of those .o files to export the function under a different name? Oh, well. As
long as it's possible to work around, it's ok, as long as it doesn't happen
frequently.
> > > * String parameters should be provided for.
> >
> > On the contrary, as much as possible of the not directly signal
> > processing related stuff should be kept *out* of the DSP part of the
> > plug-on spec.
>
> I think this is totally true for many plugins. However, some plugins
> will *be* gui elements. Some will even use (shhh) files. :O These are
> inappropriate for use in rtlinux, but they represent important audio
> processing that many on this list are involved in.
Uh oh... Starts to sound like my GUI plans. Perhaps we should define two APIs
that can be used separately, or together, depending on the nature of the
processing code?
> > 1) Only one interface + subsystem for automation, MIDI-style
> > events and system control events.
> >
> > 2) Events and parameter changes are independent of buffer sizes.
> >
> > 3) *Everything* can be defined in relation to real time.
> > (A way for plug-ins to tell the engine which events they
> > support, and their real time characteristics is needed.)
>
> Hmm..this sounds interesting but radically different from the approach
> Guenter is taking. Perhaps you should write up a competing proposal?
I will, but I have to get this CD project song ready first... (Kind of feel more
like hacking now, but I have to get this over with.)
> > The basic rule is that the engine gets to decides what sample rate to
> > use. Complex processing nets with plug-ins that have various smart ideas
> > about what sample rates to use just becomes a mess of lost quality and
> > CPU power...
>
> Sounds good to me. Does anyone really want sr-specific plugins?
> Perhaps for i/o purposes?
Yes, but that really belongs in the engine, and in the drivers. However, I've
had ideas about using an internal, extended version of the plug-in API for that
kind of things. For example, the multichannel disk streamer could be
implemented as a plug-in that communicates with a thread doing the disk access.
That way, you get the interface to the engine for free. :-)
> > > * To handle resampling, variable output buffer sizes are important.
> > > It's also important that the process method can report how many output
> > > frames it generated (is this the function of the return value of the
> > > process method?). It may even be important that the plugin can
> > > be queried as to how many output frames *will* be generated to avoid
> > > overruns due to disagreements about fencepost issues.
> >
> > DO NOT adapt output buffer size to the inputs!!! It's doing it all
> > backwards.
>
> OK, I was on crack when I wrote that. Honestly, I've always coded to
> the give-me-this-many-output-samples paradigm so I have no idea what I
> was thinking.
Well, I actually thought it would be useful for some things upon a time, but
the recursive engine implementations I thought of to sort it out kind of
made me nervous... Why solve a problem when you can avoid getting there in the
first place? *hehe*
> > The algorithm is: "I need X samples. How many input samples do you need
> > for each channel?"
> >
> > That's what this section of my little draft is for:
> >
> > int look_ahead; /* Number of extra frames needed
> > * after the end of input buffers.
> > * Can be negative.
> > */
> > int skip_behind; /* Number of frames skipped
> > * at the start of input buffers.
> > * (That is, inputs[n]+=look_behind
> > * before process_XXX() is called.)
> > * Can be negative.
> > */
>
> Can you expand on this? I'm slow. :)
I don't blame you. It wasn't the first solution to cross my mind... :-)
Let's say we have a simple mono FIR filter plug-in. The FIR filter needs a
window, and will have an inherent latency that depends on the size and shape of
that window.
With VST, we would have to use our own internal buffer to provide the input
data it needs to calculate the samples near the start and end of the buffer.
This complicates the code, and adds unwanted latency that the engine doesn't
know of, and can't compensate for.
|<--- buffer_size -->|
|====================|====================|====================|
| | |
->|_-"-_|<--FIR window |
| | |
|==========================|<-(internal buffer)
Now, let's do it my way... :-)
|<--- output_size -->|
->| |<-- -skip_behind ->| |<--look_ahead
|=================|==|====================|==|=================|
| | |
->|_-"-_|<--FIR_size |
| | |
|<----- input buffer ----->|
Here we use a skip_behind of -(FIR_size/2) and a look_ahead of
(FIR_size/2). The result is that we can simply run our filter accross the
input buffer (still output_buffer_size frames!), and the result will
automatically be a complete output buffer. An extra bonus is that we get rid of
the filter phase error as well. :-)
So as long as the input and output signal are of the same sample rate,
skip_behind = output_pos - input_pos
and
look_ahead = input_end_pos - output_end_pos
> > Yes, indeed. Also, it has to be strictly defined *when* you can allocate
> > and free memory, as it's not real time operations.
>
> Hmm..what kind of predicates would be useful? Perhaps
> allocs_only_at_init() and alloc_is_param_independent() might help? I
> think those would be true of many, many pure dsp plugins. They could
> default to false for plugin authors who don't think about such things.
Sounds reasonable. However, consider that offline_recalculate() thing I
mentioned in another post. It could be used for this too... Actually, two
levels would be nice, one that can still run at hard RT priority in order to
get the recalculations over with ASAP, and one that runs on non RT context, so
that you can reallocate buffers and that kind of stuff. Just return from that
function when you're done, and the process() function will get a message with
your return code. The event should make the plug-in start using the new
buffers (or whatever), and request another offline_recalculate() to dealocate
anything not needed any more.
> > It's possible to
> > design a fast, non fragmenting real time memory manager with a locked
> > memory pool for those who really need it, but dynamic allocation is
> > generally considered a no-no in hard real time systems.
>
> I think many pure dsp plugins would work fine using a no-op free().
> Just sequentially allocate all their init stuff in a statically
> allocated arena.
I'm not thinking about static stuff... free() is *needed*, or you'll run out of
memory in a short while.
> A tangential question re fragmentation: isn't there always some in a
> malloc()/free() paradigm (assuming your free() does something)? For
> example, what's the minimal safe-for-space factor for a buddy-block
> system?
The solution is to use a pool of fixed sized blocks that are never split.
Take a look at the kmalloc() (in the Kernel) for an example of the kind of MM
I have in mind. IIRC, it's using buffers of size 128k, 64k, 32k a s o, and you
always get a full buffer. If the manager is out of buffers of the "right" size,
it'll check if there's a bigger buffer free.
Oh, I have one written in C++ lying around at work, BTW. (I'm hacking C++ in
the kernel, that is. Kind of cool when the compatibility stuff is sorted, but
I wouldn't recommend it for new projects.)
> Thanks for being involved, :)
Well, thanks, and the same... :-)
//David
·A·U·D·I·A·L·I·T·Y· P r o f e s s i o n a l L i n u x A u d i o
- - ------------------------------------------------------------- - -
·Rock Solid David Olofson:
·Low Latency www.angelfire.com/or/audiality ·Audio Hacker
·Plug-Ins audiality_AT_swipnet.se ·Linux Advocate
·Open Source ·Singer/Composer
This archive was generated by hypermail 2b28 : pe maalis 10 2000 - 07:25:53 EST