Subject: Re: [linux-audio-dev] Plug-in API progress?
From: David Olofson (audiality_AT_swipnet.se)
Date: ke syys 22 1999 - 19:09:04 EDT
On Wed, 22 Sep 1999, Juhana Sadeharju wrote:
> >From: David Olofson <audiality_AT_swipnet.se>
> >
> >Conditional code is complex and expensive. I think the low level data flow
> >should consist of only two data types: raw data and events. What actual data
> >formats we're really dealing with will be hardcoded in the plug-ins anyway
>
> I were thinking that some might want keep the gain (say) constant,
> changing time to time, changing constantly. If the plug-in fixes the
> type, then should we have N+1 different version of the same plug-in?
> Or how?
Events... They are decoded with optimized case statements (any decent compiler
generates jump tables), and as you *have* to check for some stuff anyway,
supporting extra events comes for free.
If you don't want to change anything, just don't send any events...
> >Static data? Isn't a single out-of-band event enough, or are we thinking
> >about different things?
>
> Anything the particular instantiation of the plug-in might use.
Ok.
> >> #define PLUGIN_INOUTTYPE_NONE 0
> >
> >For padding or what...?
>
> The plug-in will use the default value if the gain parameters has no input.
> I think NULL as a function call argument does the same.
Conditionals that can't be optimized away... With an internal copy of the
parameter, and events to change it, no extra tests are needed.
> >> #define PLUGIN_INOUTTYPE_BUFFER 1
> >> #define PLUGIN_INOUTTYPE_EVENTLIST 2
> >
> >Another way to handle Event Ports (as I call them for now) is like something
> >you create during the instantiation of a plug-in (create any number you need).
> >Don't know if there's really much point in having multiple event inputs on a
> >single plug-in...
>
> If a plug-in has both the frequency and boost parameter in the function
> call, you need two eventlists (if events are used on those parameters).
> The event actually means "change the gain to value <event value>".
>
> There is no any target info on the events so that the events such as
> "gain please! change yourself to <event value>" cannot be sent (to that
> unique port).
Ok...
> If we later add such a target system, then the targets can be parsed
> at building time and be hardcoded the above way at run-time.
Hmm... I kind of like the hardcoding idea. But I doubt that it'll pay off:
Have a look at the engine side... And what if you have say 15 parameters of
which you may automate any few? That's a veeeeery long stack frame, most of
which will be unused most of the time. (Or am I missing the point?)
> >Do you put multiple channels in the same buffer here?
>
> Good question. I planned to provide the number of channels, samplerates,
> etc. as one static input. It is the matter of building time and not
> of run-time.
>
> The samples are interleaved in the buffer. Event should also have
> simultaneous data for all channels if the parameter affect to all
> channels.
Sounds a bit inflexible to me...
> >How about
> > int time;
> > int code; /* What is this? */
>
> Leave this out because it is hardcoded at network building time.
> We know it already.
Ok. (See above - "many parameters".)
How do we access parameters (that are used less frequently) if they're not
"registered" as an event input in the current net? (And BTW, if the code is
supposed to be recompiled to get rid of the conditionals, we're in serious
trouble. Such a restriction can never be removed without completely redesigning
the system, which I think defeats the whole point with this "prerelease" API.)
> > int from; /* Who is this from? */
>
> Hardcoded. Or no need to know.
Not if all code is always present, and the work is supposed to be done at run
time. Unfortunately, there's probably no way to avoid the fact that plug-ins
must be possible to deliver as closed source binaries...
> > int size; /* Number of data bytes */
> > /* dynamic size data follows... */
>
> Hardcoded. Basically the time and arbitrary data (which the plug-in knows
> how to cast) are needed.
Same thing... BUT, the event size *can* be hardcoded in the meaning that only
the sender and the reciever of the event needs to know the actual size. Sending
event buffers as arrays of pointers to events eliminates the need for the
engine to know about event sizes.
> >so we might be able to have dynamic event size from the start.
>
> Yep.
>
> >> f = (filter *)malloc(sizeof(filter));
> >> and default values:
> >> filter_setsamplerate(f,44100);
> >
> >One of my latest wild ideas it to replace this entirely with the
> >event system.
>
> Above calls are not called by the engine but the network builder.
And that's the way it should be. The point is just getting rid of one of two
similar interfaces...
> >No struct, just a callback function that the engine asks about properties
> >when instantiating a plug-in, or just wants some general info anout the
> >plug-in class.
>
> I would layer the data this way:
> -the things needed when the plug-in flows the audio;
> -the things needed to add the plug-in to the network.
Yes.
> The simple API I were thinking would let the second part to the
> programmer of the application. He would know early all the plug-ins.
He can just send some request events to the plug-ins init function. The event
system code is there anyway, and the ability to generate answers at run time is
interesting. (Remember the buffer size and sample rate restrictions
discussion...?)
> Because we are including only the audio flow data structures to simple
> API, the more complex system can be build over the simple API.
Well, my idea was to make the basic system even simpler, so we have less
functionality that will be removed or changed. Perhaps doing nearly everything
with the event system isn't optimal, but once the system is there, it's very
simple to implement stuff on top of it. I'd rather start with that, and then
move things out to the structures later on, if required for performance reasons
or otherwise.
> I think even the random access wave editor plug-ins would use the flow API
> for the actual processing when and if possible.
Yes, as long as the headers make everything look nice on both sides (and not
only on the plug-in side, as with most existing SDKs...), there's not really
any excuse not to use it. Preferably, you should only need to include one or
two header files, hack a few lines and then fill in your code to get started.
(The default settings should be well defined and reasonable, so that most
plug-ins won't have to answer most of the standard init questions from the
engine or net builder. I think this beast a struct where you can use default
values for some of the fields...)
> >> The runtime call would be
> >> filter_do(f,inout *input, inout *output, inout *freq, inout *boost);
> >> where input and output would get buffers and where freq and boost would
> >> get events.
> >
> >I think
> > filter_do(f, void **inputs, void **outputs, event **events);
> >will do.
>
> No if we hardcode the event paths.
True, but I think that kind of stuff belongs in a control signal system - not
an event system. (Control signals can be implemented as audio streams.)
> >Also, no output buffer for events.
>
> They are in my system.
That is, an event output buffer is always sent directly to an input of some
other plug-in?
> >Yeah, but there's a slight timing problem here; how do you know where you're
> >headed before you've recieved the next point?
>
> If you need to know the curve very well, it can be edited in the editor
> and given to plug-in as static data. If sent as events or buffers, we
> face the same latency problems as with audio or MIDI data.
I don't like the distinction. Too many subsystems and data types...
> >Well, even if my event system might be more complex than what you had
> >in mind
>
> I would like to go the simple hardcoded way and only later look at the
> general event system.
I'm afraid that will result in two interfaces/subsystems that overlap too
much... And is your event system really that much simpler to implement?
> I think we will use the closures as used in Csound and in Quasimodo. They
> are quite the same what I proposed but less parameters at function calls.
Kind of like a way of making better use of the plug-in instance struct. And the
less data that needs to be moved to the stack for each call (ie explicitly
regarded as "data that will be used anyway" - which may not always be true),
the more chances of optimizing the plug-in code.
//David
·A·U·D·I·A·L·I·T·Y· P r o f e s s i o n a l L i n u x A u d i o
- - ------------------------------------------------------------- - -
·Rock Solid David Olofson:
·Low Latency www.angelfire.com/or/audiality ·Audio Hacker
·Plug-Ins audiality_AT_swipnet.se ·Linux Advocate
·Open Source ·Singer/Composer
This archive was generated by hypermail 2b28 : pe maalis 10 2000 - 07:27:12 EST