Subject: Re: [linux-audio-dev] big picture, for a moment
From: David Olofson (audiality_AT_swipnet.se)
Date: ti loka 12 1999 - 19:44:42 EDT
On Tue, 12 Oct 1999, Paul Barton-Davis wrote:
> Getting closer ...
>
> >...while I view a plug-in as a black box with a number of inputs and
> >outputs, in the form of audio ports and event ports. The average
> >plug-in would have >=0 audio inputs, >=0 audio outputs, one event
> >input, and possibly one event output. During initialization, the
> >plug-in will tell the host what events it understands (little more
> >than info for the UI), and if it has an output port; what events it
> >might send through it.
>
> =======================================================================
> > The plug-in doesn't care (or know) who listens
> >to it's output, nor does it know where the events it recieves come
> >from.
> =======================================================================
>
> we both absolutely agree on this key point.
Yes. I just want to go one step further and get the subscribe() call
out of the plug-ins.
> i think that our difference is small here. i look at the address of a
> port as being a statement about the events the port will "send".
Ok. That makes a very big difference.
> if
> you make this concession for a moment, i think you'll find we're
> saying something very similar. the plugin tells the engine what it
> understands. you say that it says "i understand float #5", whereas in
> my view, its says "i understand midi cc 34". the only difference here
> is what the semantics of an address are, for perhaps float 5 *is* midi
> cc 34 ! in both cases, the engine will subscribe the plugin to a
> particular port, right ?
Yes, but does that mean that one event source == one event? (As
opposed to my "send all through the same output port" mess.)
And what happens if you have multiple ports sending MIDI CC 34, or
whatever? And how do you route events so that it doesn't become
hardcoded in the plug-ins which outputs can be connected to wich
inputs? Where's the remapping done?
[...]
> essentially, i want *more* flexibility than you in that i want event
> sources to be able to *restrict* the engine's view of the kinds of
> events that will be posted to their port(s).
"be able to *restrict*"? What if they don't? If it's *required* that
event sources send only one event, or possibly a few related event on
each port, OK. Doing the event dispatching before the events even get
a chance to be mixed up sounds reasonable to me.
> most plugins might only declare a single port, but others might
> declare many ports. consider a plugin that provided a GUI for a mixing
> console (its on my mind:). instead of mux-ing all the events for
> parameters for every strip into a single port, it would declare N
> ports, one for each strip. Then the plugins that were implementing
> strip-level FX/EQ would just be subscribed to that port, not a port
> that would contain a bunch of events they didn't care about at all.
Ok. (However, in the system I proposed, the engine would de-MUX the
events *before* they reached the subscribers - and it would do
remapping at that point as well.)
> in essence, i imagine a heterarchy of ports, some completely generic,
> some totally specific: "this port posts byte events describing the
> state of the I-Cube tap sensor".
>
> any "input port" (really, just a pointer to a port, since there is no
> data copying)
There *is* still some data shuffling, even if it's possibly only
building a list. If you don't build a new list before the listening
plug-in is called, how would you subscribe to multiple ports?
> that could handle byte data could be connected to this
> port, but in addition, a plugin that listed a set of possible sources
> for its displays could show "I-Cube tap sensor", and connect directly
> to that port.
>
> and imagine being able to dynamically describe all the ports in the
> system:
>
> Port Description Data Type Source
> --------------------------------------------------------
> #13 generic float pluginNN
> I-Cube tap sensor byte pluginNM
> #14 generic string pluginXX
> #15 pluginNN_t struct pluginNN
> #16 audio audio sensor4
> MIDI CC 34 byte sensor5
>
> and so on. I regard this as equally generic as your system, but also
> potentially much more efficient. What do you think ?
It looks fine, but there's still the remapping issue, and the
sorting/merging of event lists.
> A last comment on this. None of the above is supposed to change the
> basic notion that event sources have no idea who is listening, and
> event receivers have no idea of the event source. From a plugin's
> perspective all that matters is the *address* of a port, which also
> tells us something about its usefulness to the plugin. The plugin, in
> subscribing to port XXXXX has no idea where events on that port come
> from.
Ok, but it *will* get all events in the *single* input buffer, right?
>
> >Well, compare that to the overhead involved when handling the number
> >of event ports needed to get the same flexibility.
> >
> >Input from multiple senders:
> >
> > * The engine has to merge the events into one list,
> > and they must be sorted according to the timestamps.
>
> yes, but what is the problem with that ?
It's expensive... Hopefully not *too* expensive, as it cannot be
avoided with any system - only made harder or simpler to do.
> the plugin process() call looks like:
>
> /* build a Lisp-like sorted list of all events on each
> subscribed port that occured before the current deadline.
>
> *NOTE* NO DATA COPYING: the list is just a set of pointers
> to the event_t's still in the ports event lists.
> */
>
> mcs_build_event_list (&plugin->ev_list, &plugin->ports);
> plugin->process (..., event_list_t *events, ...)
>
> >Output to multiple destinations:
> >
> > * _ALL_ events will have to be copied to _ALL_
> > destinaiton ports by the engine, as the sending
> > plug-in has no idea who's listening to what. And
> > neither does the engine...
>
> no. events are never copied at all. they exist on a port's event
> list. things that listen to the port will find them there.
...but pointers will have to be moved around when building the
listeners' lists. However, as long os it's only for events that are
actually used, it's probably not that bad. The average even isn't all
that much bigger than a pointer, though...
[...]
> >What I'm thinking about is that it's not necessarily true that
> >there are no events after the last one to be processed during
> >the current cycle. There may be events to be processed later
> >on... This happens when interfacing between sub nets with
> >different buffer sizes, and when communicating accross threads.
>
> leaving aside the subnets with different buffer sizes, i thought that
> the whole point of the design here was something like this:
>
> events arrive during one pass through the plugins' process()
> functions.
Yes.
> they are timestamped with the sample on which they
> occured.
...and the time stamps must in that case be adjusted by adding the
fixed latency. Timestamps on input events should tell exactly when
they occured, while timestamps on output events should indicate when
to perform an action.
However, I think "cheating" here i pretty OK; if you record
something, you probably want the playback to sound identical to the
monitor mix while recording. That means the events that go into the
sequencer should have the same timestamps as the evets that reached
the plug-ins while recording, *not* the exact timestamps of the input
events!
> during the next pass through the plugins, the plugins get to
> process the events, because the sample timestamps will correspond to
> samples they will generate/process during that or a later call to
> process().
Yes...? That's what I was trying to say; "or a later call to
process()", in particular.
> >As long as the GUI is also in its own *file*, OK. I can load ELF into
> >kernel space and similar weird places, but I don't feel like cleaning
> >out "irrelevant" code on the fly...
>
> sounds fine.
Makes it possible to run the GUI as a separate task, to keep it from
taking the (user srace) engine with it as well, and also enforces
better programming style.
> >Why? That's *exactly* where the intermediate FIFO hidden behind the
> >shadow ports come in handy. If sensor threads can do it, so can
> >clients, *and*, it makes them a lot more similar to plug-ins. A
> >client is to MCS what a plug-in is to an engine... To some extent at
> >least; clients can do lots of cool stuff that plug-ins cannot.
>
> fine. its more complex, and it will involve more overhead on both
> sides of the communication link, but fine.
Nothing comes for free... But the cost in this case is very low
considering the flexibility that the event system brings, IMO. And
BTW, counting per event, how many IPC systems are more efficient
than this event system? We can allocate memory for an event, fill it
in and post it without *one single* function call. A thread can send
hundreds of events doing only a few calls to qm_new_buffer().
> i don't buy any of this
> shadow port stuff - i think its a totally unnecessary concept. i think
> that ports should just be event FIFO's: posted to by event sources,
> read from by port subscribers, and cleaned out by the engine.
As long as the event ports stay as simple as they are now, you're
right; shadow ports are pointless. The "events" field becomes the
FIFO write pointer, and the reader will only change the FIFO read
pointer. The heap will only be used by the writer. Unless there's
anything more a port should be able to do, that's about all there is
to it...
//David
·A·U·D·I·A·L·I·T·Y· P r o f e s s i o n a l L i n u x A u d i o
- - ------------------------------------------------------------- - -
·Rock Solid David Olofson:
·Low Latency www.angelfire.com/or/audiality ·Audio Hacker
·Plug-Ins audiality_AT_swipnet.se ·Linux Advocate
·Open Source ·Singer/Composer
This archive was generated by hypermail 2b28 : pe maalis 10 2000 - 07:27:59 EST