Re: [linux-audio-dev] XAP spec - early scribbles

New Message Reply About this list Date view Thread view Subject view Author view Other groups

Subject: Re: [linux-audio-dev] XAP spec - early scribbles
From: David Olofson (david@olofson.net)
Date: Thu Feb 06 2003 - 20:09:38 EET


On Thursday 06 February 2003 06.11, Tim Hockin wrote:
> > The host struct is the plugin's interface to various host
> > provided resources. Unfortunately, making these resources
> > thread-safe would result in a major performance hit. (Consider
> > the event system, for example.)
> >
> > The easy way to deal with this is to just give plugins another
> > host struct when you want to mess with them in another thread.
> > The plugins won't be able to tell the difference, unless you want
> > them to.
>
> At what time do we need to change out the Host struct? I'm not
> seeing where or why we need to do that. Concrete example, please?

Changing any control that isn't RT safe. A delay time change that
requires buffers to be reallocated, for example.

However, IMHO, this turns into just another motivation for worker
callbacks, and even suggests that use of them should be encouraged.
In fact, I think worker callbacks for non RT safe operations should
be *required* for plugins that claim to be RT safe at all.

Problem is that the host cannot know when a plugin has received an
event that will trigger a non-RT safe operation. Besides, you can't
take a processing plugin out of the net anyway, as you'd disrupt
anything that passes through it. Only the plugin can deal with this
properly.

Taking the delay time change as an example, the plugin would do this:

        When the offending event is received:

                * Fire a worker callback to allocate and
                  initialize new buffers.

(Processing goes on as if nothing happened.)

        When the worker callback finishes:

                * Throw in the new buffers, change params etc.

                * Free the old buffers. (Worker call or
                  host provided RT safe free() replacement.)

I think this applies to most stuff that isn't RT safe, and it's the
only way to have plugins with non RT safe operations work correctly
in RT hosts. I also think implementations can be really rather simple
most of the time.

> > > We can track silence per Port, or just per Plugin (simpler, I
> > > think).
> >
> >_
> > Simple, but insufficient for any but the simplest plugins. If you
> > have_
>
> Again, I think we are speaking of slightly different things. I am
> talking about the time when (for example) the synth at the head of
> the chain has stopped playing notes. A reverb with this as it's
> input would be told 'your input is now silent'. It has a tail, of
> course. It can be marked silent when it's tail is done. Once it
> is silent, it does not need to be processed._

I see. I also see a big difference between synths and "effects" here.
It's hard to tell when you need to start processing a synth again,
after it's been silent. You can *guess* by looking at the input of an
effect plugin, but you can't even be sure in that case with some
effects. Such effects would just have to ignore this feature
altogether and be processed at all times.

> I don't know what use it is to mark individual buffers/Ports in
> this case.

The point is that the effects that process the output from each
channel of the synth can take great advantage of knowing whether
input is silent or not.

> What is the point of having 'Silent' or NULL buffers? I mean, I
> guess it can be a minor optimization.

It's a rather *major* optimization when it means you can return
without doing any processing at all... That's what Audiality's old
"reverb" does when it realizes the tail is out and there's no input.

> Maybe silence is a
> per-channel thing? I don't know that finer grain flags buy
> anything at all..

Why channels? If it's not per audio port, it might as well be
anything... I think this depends entirely on plugin internals.

[...]
> > > Then the
> > > host can hot the bypass switch (is bypass optional?) or remove
> > > it from the net, or just stop processing it.
> >
> > No, that won't work. A plugin being silent doesn't mean it's
> > passive, or even that it won't start generating output again in a
> > moment, even if there's no new input. A long feed-back delay fed
> > with a short sound would intermittently generate non-silent
> > buffers for a good while, and plugins following it could still
> > have use for the "silence flag".
>
> In my view this is not silence. A silent plugin is one that is
> totally passive.

A silent plugin is also one that you cannot know when to start
processing again... Unless the host eavesdrop on all event queues of
silent plugins. Plugins could provide a callback to check all Queues,
but then what's the point? Plugins could just do this in run()
instead. Then they could even chose to ignore some events, if
appropriate.

> > Besides, you cannot safely stop calling a connected plugin. If it
> > has inputs directly connected to outputs of other plugins,
> > there's no way you can know when to start calling the plugin
> > again, since you won't even know about the events sent to it's
> > Queues. What you get is a dead plugin draining your event pool.
> > :-)
>
> Agagin, hrrm. I always envision the host snooping events. I'll
> think some more about this.

I think our direct plugin->plugin design is much more efficient than
anything that requires events to be sent through the host. And as
I've pointed out before, if that works, you automatically have an API
that allows the host to really be just a *host*, with sequencers and
whatnot running as plugins; real or host integrated.

> > > * EventTarget:
> > > A tuple of an EventQueue and a cookie. Plugins have an
> > > EventTarget for each Control, as well as a master and
> > > per-Channel EventTarget.
> >
> > Not quite. Targets aren't really "input objects" in the API, but
> > rather two other things:
>
> I'm not following you - what did I say wrong?

Well, first of all, I've been thinking some, and I'm going to propose
the complete elimination of the event "action code" as a required
argument. I want to use EventTargets for *everything*, as it makes
the API cleaner, faster and simpler.

There isn't really much point in having the physical ability to send
events without a connection, especially not when it requires an extra
level of decoding in all plugins, as well as an exception in the API
and semantics.

That is, there has to be one EventTarget for every *action* on every
object in a plugin. Some EventTargets may use an argument as an
action code, but there's no reason to force action code decoding for
events that are fully qualified through their cookies.

Anyway, what I initially reacted against was that it sounds like these
EventTargets are visible objects that you can just get and connect
to. However, this is not the case physically, since the objects would
be generally useless to plugins internally.

When you ask for an EventTarget to make a connection, the plugin just
figures out which EventQueue to return, and encodes the address
fields in some suitable way. (So internal tables, switch()es and
stuff can be used for quick decoding.) This is basically just an
implementational issue, but I don't feel good about making it sounds
as if EventTargets as physical objects exist inside plugins. (They
*are* physical objects on the API level and usually also internally
in the output handling of plugins.)

Now the interesting part for the API level: A plugin might want to
know whether it's inputs are connected or not. If so, the plugin will
have to rely on the host using the get/release (open/close) calls
properly, so you can't just assume calling get_input_target() doesn't
mean anything to the plugin.

In that light, stating that the EventTarget is a physical object, and
calling the function to get it get_*() is really rather confusing.
(To me at least. :-)

> > * The physical struct that you get filled in or
> > returned when you ask a plugin for one of it's
> > input controls.
> >
> > * An abstract object inside a plugin, representing
> > a control output.
>
> right - did I say something different?

Well, I *interpteted* it differently. :-) See above.

[..]
> I don't mind the model. The names I object to. Targets aren't
> just control. Controls are the biggest user, but there needs to be
> a master target and a per-channel target. Targets are, in fact,
> event targets.

Yes, and this is in total agreement with the idea of eliminating
action codes, as outlined above.

> {open,close}_event_input()
> {connect,disconnect}_event_output() or {open,close}_event_output().

Yes. I think I prefer {connect,disconnect}_event_output(), although
{open,close} is shorter and suggests that it's related to the input
counterpart... {connect,disconnect} is more accurate, though, as it
actually *is* the output that makes the physical connection.

> > > Process all events up to and including 'now'. For a 0 frame
> > > run(), all timestamps are 'now'.
> >
> > No, timestamps are still valid, so there's no way to guarantee
> > this. All events in the queue that match the current time will be
> > processed, but any future events will be left in the queue.
>
> Sorry, I wasn't clear. If you want to change a control
> immediately, set the change event to the current time, and run() 0
> frames. It will process events for 'now' which will get the
> change. It will then see that there are 0 frames to process.

Fine.

> Alternatively, 0-frame runs are kind of special. nframes=0,
> timestamps can be 0, too.

No, this has nasty implications. You'll either need a special event
decoding loop in plugins, or you'd have to temporarily set the
running time field in the host struct to 0, and *require* that event
timestamps are 0.

//David Olofson - Programmer, Composer, Open Source Advocate

.- The Return of Audiality! --------------------------------.
| Free/Open Source Audio Engine for use in Games or Studio. |
| RT and off-line synth. Scripting. Sample accurate timing. |
`---------------------------> http://olofson.net/audiality -'
   --- http://olofson.net --- http://www.reologica.se ---


New Message Reply About this list Date view Thread view Subject view Author view Other groups

This archive was generated by hypermail 2b28 : Thu Feb 06 2003 - 20:12:02 EET