Subject: Re: [linux-audio-dev] Plug-in API progress?
From: David Olofson (audiality_AT_swipnet.se)
Date: ti syys 21 1999 - 16:56:19 EDT
On Tue, 21 Sep 1999, Juhana Sadeharju wrote:
> >From: David Olofson <audiality_AT_swipnet.se>
> >
> >Back from Malta, and it's friday. :-)
>
> Welcome back!
Thanks!
> I have updated my GASP pages to have a list of commercial
> API resources (see the section "APIs" in the list of commercial softwares;
> http://www.funet.fi/~kouhia/gasp/).
I'll have a look. :-)
> I have an idea: the discussion goes to quite deep matters but we could
> in the first place make a quite simplified plug-in API. So that, it
> doesn't take two years yet again before we have the plug-in API.
...and when we have finally finished this ultimate API, we realize it's too
complex, too inefficient, and just not good! *hehe*
Yes, I think developing a basic API to get started is a good idea. We'll get
some actual code to test our ideas on, and we'll also get a chance to get more
real life facts rather than making important design decisions through qualified
guessing...
> We basically need input and output connections. For audio data, they
> will be buffers of the given length -- processing is done from input
> buffer to output buffer. For control data, they will be either buffers,
> event lists, or static data.
Conditional code is complex and expensive. I think the low level data flow
should consist of only two data types: raw data and events. What actual data
formats we're really dealing with will be hardcoded in the plug-ins anyway, and
there's not really any other way to do it without wasting performance, and
complicating the code. And the engine only needs to keep track of buffer sizes
and events, as it's not supposed to mess with the data directly.
Static data? Isn't a single out-of-band event enough, or are we thinking about
different things?
(Anyway, we're perhaps thinking along the same lines - I just want to make sure
we reduce the API to the "narrowest cut", if you know what I mean... Few
but useful features.)
> Some commercial plug-ins indeed have a GUI for drawing curves and so on.
> We could agree type of those curves and curves would be given to the
> plug-in (as static data or as event at run-time) instead of plug-in
> allowing the user to draw them.
Basically, GUI is a different part of the system. I think defining the GUI API
as a subset of the plug-in API (most importantly featuring the event system)
would be a pretty flexible way to do it, as that means the developers only need
to learn one API. The GUI engine would be very similar to an in-application
plug-in host. It would be possible to hack a standard GUI plug-in that works
somewhat like the simple string based user interface stuff of VST, so that
plug-ins can instantiate, initialize and control basic GUIs via the event
system. Whether you want to hack that GUI control code directly into the
processing code, or into a separate module is up to you.
The point is that this way, it can all be done without extendning the plug-in
API. All you have to do is allowing certain kinds of plug-ins to use
functionality outside the plug-in API. GUI plug-ins can use X, while processing
plug-ins aren't allowed to touch *anything* but the stuff defined in the
plug-in API. First of all good programming style, but it also happens to fit
well with real time programming in general, and RTLinux programming in
particular. (That is, plug-ins will compile and run under RTL right away if
that rule is followed.)
> We could assume the plug-ins are included at compile time. So, no GUI
> or names are needed. The programmer looks the inputs and outputs from
> the plug-in reference book.
Saves some work perhaps, but I'm not sure. (Ok, there is a problem with .so
linking and name space...)
> This would be easy to do and the plug-in development could go on
> immediately. At least Quasimodo needs GPLed opcodes to replace the
> original opcodes and this would be a good place to make the opcodes
> available for any audio software.
Yes, anything to make reuse of code easier is very good, even if it's just on
the source level for now.
> Here is my rough try on this. This looks so simple that you should look
> if I just have goofed. This is *not* the final version we have to think
> about good structure and field names and their final types.
>
>
> #define PLUGIN_INOUTTYPE_NONE 0
For padding or what...?
> #define PLUGIN_INOUTTYPE_BUFFER 1
> #define PLUGIN_INOUTTYPE_EVENTLIST 2
Another way to handle Event Ports (as I call them for now) is like something
you create during the instantiation of a plug-in (create any number you need).
Don't know if there's really much point in having multiple event inputs on a
single plug-in... (With my system, you can see where events come from anyway,
if your plug-in needs to keep track of multiple event sources.)
> #define PLUGIN_INOUTTYPE_STATIC 3
> struct {
> int begintimeoftheplugincall;
> int inouttype;
> char *buffer;
Do you put multiple channels in the same buffer here?
> int numberofsamplesinbuffer;
> event **eventlist; /* buffer of pointers to events */
> int numberofevents;
> void *static;
> } inout;
>
> struct {
> int time;
> what to put here?? int idata, float fdata, char *anydata?
> } event;
How about
int time;
int code; /* What is this? */
int from; /* Who is this from? */
int size; /* Number of data bytes */
/* dynamic size data follows... */
?
Or we could just leave out the dynamic size part for now and throw in a couple
of ints or something. But I think I'll get something useful together this week,
so we might be able to have dynamic event size from the start. (Less to change
in the plug-in code, but it is a little more complex, of course. I'll hack some
nice macros to encapsulate it, though, so it might not make much of a
difference in the end.)
(Benno: I'm starting to think one time stamp/event is pretty reasonable in most
real life situations...)
> The effect has a data structure:
> f = (filter *)malloc(sizeof(filter));
> and default values:
> filter_setsamplerate(f,44100);
> filter_setfreq(f,400.0);
> filter_setboost(f,-20.0);
> and if plug-in is general:
> filter_setchannels(f,2);
> filter_sampletype(f,SAMPLETYPE_FLOAT);
One of my latest wild ideas it to replace this entirely with the event system.
No struct, just a callback function that the engine asks about properties when
instantiating a plug-in, or just wants some general info anout the plug-in
class. Very flexible, and no structure to extend as new things must be
supported... (And of course, it's yet another ++usecount for my Universal
Solution; the Event System. ;-)
> The runtime call would be
> filter_do(f,inout *input, inout *output, inout *freq, inout *boost);
> where input and output would get buffers and where freq and boost would
> get events.
I think
filter_do(f, void **inputs, void **outputs, event **events);
will do.
The plug-in won't have to check and decode the contents of a structure for
each buffer and call - the engine will send events to tell the plug-in about
any buffer size changes and that kind of things.
Also, no output buffer for events. You'll just have to keep track of the Event
Ports you want to send to occasionally, unless you're just going to reply to
events you recieve.
> We could agree the static and event curve data is a polyline, and that
> the plug-in decides if the curve is actually polyline or spline. So that
> it is safe for a plug-in to send polylines to parameter port of the next
> plug-in.
Yeah, but there's a slight timing problem here; how do you know where you're
headed before you've recieved the next point? Staying one or two events behind
isn't exactly good timing... It'll have to be events with destination values
and times, and prediction for true real time. How the curves are rendered is of
course up to the plug-in.
> By anyway, if this discussion made you think a better simple plug-in API,
> will it be only good because we need fast suggestions for this quickly
> written simple plug-in API.
Well, even if my event system might be more complex than what you had in mind,
I think we'll save time if we can use it as the single way of communication
except for raw data. And, perhaps there's a better chance we'll come closer to
the final API that way, as we won't have structures with lots of fields
describing the details about things that are likely to change.
But I'll hack on, and see how it turns out...
> I propose that we *end the discussion* on this simple plug-in API
> at **October 11**. Then I will write my current set of filters to conform
> this simple API and will call plug-in writers.
>
> October 11, that is it!
Sounds reasonable, I think.
//David
·A·U·D·I·A·L·I·T·Y· P r o f e s s i o n a l L i n u x A u d i o
- - ------------------------------------------------------------- - -
·Rock Solid David Olofson:
·Low Latency www.angelfire.com/or/audiality ·Audio Hacker
·Plug-Ins audiality_AT_swipnet.se ·Linux Advocate
·Open Source ·Singer/Composer
This archive was generated by hypermail 2b28 : pe maalis 10 2000 - 07:27:12 EST