Re: [linux-audio-dev] XAP spec - early scribbles

New Message Reply About this list Date view Thread view Subject view Author view Other groups

Subject: Re: [linux-audio-dev] XAP spec - early scribbles
From: David Olofson (david@olofson.net)
Date: Thu Feb 06 2003 - 02:59:22 EET


On Wednesday 05 February 2003 23.41, Tim Hockin wrote:
> > Well, you know my opinion about what controls plugins.... ;-)
>
> how about:
>
> * Host
[...]
> * Control:
[...]
> * EventQueue

Good.

[...]
> > Yep. The effect of this might be cool enough to deserve a
> > comment;
>
> it is commented in the details section :)

Yeah, I noticed... :-)

> > There will never be "output EventQueues", since you're always
> > sending directly to the connected inputs, as far as plugins are
> > concerned. Whether you're actually sending directly to another
> > plugin, or to a host managed sort/merge proxy queue is pretty
> > irrelevant to the API and plugins, since it's a host
> > implementation thing.
>
> Well, there is an output-queue, it just might be someone else's
> input queue.

Yes, but I think it's more confusing than helpful to drag that in.
What you *own* is your input Queues. When it comes to outputs, the
only objects you own are these "XAP_target" structs.

[...normalized and audio...]
> I've always heard it called normailzed, and it jives with what I
> know.
[...]

Yeah, I know - but it's rather confusing when "normalized" in relation
to *controls* means that out of range values are illegal.

[...]
> > And connecting an audio port means handing it a buffer pointer,
> > among other things? That is, buffer pointers aren't changed
> > frequently, and never silently.
> >
> > I *think* this should do, since it does not conflict with reuse
> > of buffers. Hosts can just figure out optimal buffer usage
> > patters when building nets. Right?
>
> That is how I have envisioned it - does this need special
> annotation?

Not really. Since we're using function calls (rather than shared
variables), it's rather obvious. I was thinking out loud, and only
came to the conclusion that this should work. :-)

[...]
> > I'm afraid this won't work for all hosts. There are situations
> > where you might want to run a plugin under a fake copy of the
> > host, in another thread, to perform non RRT safe operations
> > without disrupting the rest of the network.
[...]
>
> How does this apply to the host struct, though? Do we need to
> change the host struct? hmmm

The host struct is the plugin's interface to various host provided
resources. Unfortunately, making these resources thread-safe would
result in a major performance hit. (Consider the event system, for
example.)

The easy way to deal with this is to just give plugins another host
struct when you want to mess with them in another thread. The plugins
won't be able to tell the difference, unless you want them to.

Of course, you'll have to disconnect any inputs before you start
messing with that plugin, and if you're going to do something that
makes the plugin send events, you have to make sure they're not sent
directly to live Queues in the RT thread.

In short, just detach the plugin from any RT resources, and if you
need to, hook it up with fakes. Do what you need, and then throw the
plugin back into the RT network. (Or destroy it, or whatever you're
up to.)

[...silence<...]
> We can track silence per Port, or just per Plugin (simpler, I
> think).

Simple, but insufficient for any but the simplest plugins. If you have
multiple outputs, they can be completely independent. Your average
multichannel/multitimbral synth without integrated mixer will have
one or more outputs per channel, and you will most likely not have
output from all of them at all times.

> Tail size is a good indicator - if we know that the plugin before
> the current plugin output silence, then we need only wait for the
> tail on this plugin to end and then it will be silent. This relies
> on plugins getting their tail-sizes correct. Tail can be 0, finite,
> infinite. Infinite tails always need processing

Yeah... Although I'm getting the feeling that it's more work for both
hosts and plugins, than basically just "marking" buffers.

Speaking of which, the new Audiality mixer just has a flag to go with
each bus accumulation buffer. Instead of clearing the buffers, I just
set the flag to "silent". The first "unit" (mixer stage or synth
voice) that hits a silent buffer gets to initialize it and set the
flag to "in use". Whenever a mixer stage reads from a buffer, it
checks the flag first, and then decides what to do - which in many
cases means return instantly from process(). If you're doing
something that results in no output (ie you're mixing silence into a
buffer), just don't touch your output buffer. If it's marked as
silent, it just stays that way.

The disadvantage is that everyone with an output has to be aware of
this, because a buffer marked as silent is intended to be *ignored*.
That is, it may (and generally will) contain garbage, which you'll
have to clear out first, unless you have a replacing version of your
inner loop.

> We can also do a call and answer style - the host tells the plugin
> that it's feed is silent. When the plugin has died down and is
> also silent, it responds to the host that it is silent.

I like to think of "silent" as a property of the data. A boolean per
buffer, basically. (Wherever we'd put that...) For inputs as well as
outputs.

The FX plugins of Audiality get NULL pointers for silent input, but
they use their state (multiple RUNNING states... *heh*) to tell the
host whether or not they generated output for each block. The NULL
pointers for inputs is nice and easy, and easy enough for plugins to
fake themselves: Just point any NULL inputs at the host provided
silent buffer. The output part "works", but only because FX plugins
can only have one stereo in and one stereo out.

> Then the
> host can hot the bypass switch (is bypass optional?) or remove it
> from the net, or just stop processing it.

No, that won't work. A plugin being silent doesn't mean it's passive,
or even that it won't start generating output again in a moment, even
if there's no new input. A long feed-back delay fed with a short
sound would intermittently generate non-silent buffers for a good
while, and plugins following it could still have use for the "silence
flag".

Besides, you cannot safely stop calling a connected plugin. If it has
inputs directly connected to outputs of other plugins, there's no way
you can know when to start calling the plugin again, since you won't
even know about the events sent to it's Queues. What you get is a
dead plugin draining your event pool. :-)

[...master and channel "queues"...]
> I didn't mean that they are separate queues - the host will ask the
> plugin for an event target for the master and each channel. It may
> be the same Queue.

Ok.

> Should we define Event Target as a term and use
> it?

Well, it's an actual struct in Audiality. :-)

> * EventTarget:
> A tuple of an EventQueue and a cookie. Plugins have an
> EventTarget for each Control, as well as a master and per-Channel
> EventTarget.

Not quite. Targets aren't really "input objects" in the API, but
rather two other things:

        * The physical struct that you get filled in or
          returned when you ask a plugin for one of it's
          input controls.

        * An abstract object inside a plugin, representing
          a control output.

In practical terms, something like:

        /* Get target for plug2 input control X */
        XAP_target target = plug2->get_control_input(X);

        /* Connect plug1 output control Y to that. */
        plug1->connect_control_output(Y, target);

and perhaps:

        /* Notify plug2 that it's input X is connected. */
        plug2->connect_control_input(X);

Plugins will most probably keep these in arrays or other structures
internally, for their output controls, but the get_control_input()
call will nearly always construct targets on the fly. All it does is
pick the right Queue and transform the "full adress" of the control
into something handy for internal decoding; ie a cookie.

> > > The Host queries the Plugin for EventQueues via the
> > > get_input_queue() method. In order to allow sharing of an
> >
> > I would prefer calling it get_control_target() or something,
> > given that what you actually get is not a Queue, but rather an
> > "atom" containing a Queue pointer and a cookie. Each of the two
> > fields of this "atom" (XAP_target) is completely useless on it's
> > own, but together, they form the complete address of an input
> > Control.
> >
> > > Controls may also output Events. The Host will set up output
> > > Controls with the set_output_queue() method.
> >
> > How about connect_control_output() or similar? Again because
> > queues have very little relevance on this level, and because
> > "connect" is
>
> Adjusted to get/set_event_target(). Good?

Still feels wrong... I have another idea, inspired by what's actually
going on, as well as this "do we notify inputs after connection?"
thing:

typedef struct
{
        XAP_queue *queue;
        unsigned int cookie;
} XAP_target;

typedef struct
{
        unsigned class;
        unsigned bay;
        unsigned channel;
        unsigned index;
} XAP_address;

Inputs:
        int open_control_input(XAP_address *a, XAP_target);
        void close_control_input(XAP_address *a);

Outputs:
        int connect_control_output(XAP_address *a, XAP_target *t);
        void disconnect_control_output(XAP_address *a);

Usage:
        XAP_address output = {...};
        XAP_address input = {...};
        XAP_target target;

        /* Connect */
        if(p2->open_control_input(&input, &target) < 0)
                return ERROR;

        if(p1->connect_control_output(&output, &target) < 0)
        {
                p2->close_control_input(&input);
                return ERROR;
        }
        return OK;

        /* Disconnect */
        p1->disconnect_control_output(&output);
        p2->close_control_input(&input);

("this" pointers left out for brevity.)

Looong names those, though... I don't want "event" in them, because
events are not equivalent with controls; they're just used to operate
them. Leaving out "control" might cause confusion with audio stuff,
and dropping the "input" and "output" part would be generally
confusing.

> > Another thought: Would plugins want to know when their inputs get
> > connected and disconnected?
>
> That is why these are methods - a guaranteed trap for connecting
> ins and outs, just like Ports..

Yeah, but I was thinking about the difference between asking for a
target from an input and actually connecting it. As you can see
above, this design makes things quite asymetric, but after throwing
the close_control_input() counter part in, it looks rather nice,
IMHO. The point is that both plugins now know when connections are
made and broken. Without the close_control_input() call, receivers
would be left in limbo if a connection fails in
connect_control_output().

[...]
> > More importantly, when do you stop processing events? There are
> > two alternatives:
> >
> > 1) Process only events that affect audio frames that
> > you're supposed to process during this block.
>
> Which (not obviously) means 0 frames = no work.

Yes - but from a practical POV, why call run() in that case?

> > 2) Process as many events you can, without skipping
> > into the next block.
>
> Process all events up to and including 'now'. For a 0 frame run(),
> all timestamps are 'now'.

No, timestamps are still valid, so there's no way to guarantee this.
All events in the queue that match the current time will be
processed, but any future events will be left in the queue.

Note that you get this behavior for free with a normal event/audio
compound loop; it doesn't require any special cases or anything. (In
fact, *guaranteeing* that there always is a "tail" event beyond the
end of the block would eliminate the test for ev->next == NULL when
readin events.)

> The Plugin must process all events for
> now.

Yes.

> > > Plugins can assume that VVIDs are at least global per-Plugin.
> > > The host will not activate the same VVID in different Channels
> > > at the same time.
> >
> > Well, as usual, I'd just like to whine some about it being
> > assumed that the sequencer is an integral part of the host. ;-)
> >
> > It could be an event processor or a phrase sequencer you're
> > listening to. Not that you'd care when you're a plugin in the
> > receiving end, but things get weird if you're about to implement
> > a plugin that outputs synth control events, while in the API
> > docs, it appears that only hosts can do such things...
>
> ok, ok - I'll reword it. How should it be worded? Assume an
> arpegiator or something that outputs VOICE. Where does it get
> VVIDs?

Host call, just like allocating events and whatnot.

> Does it have to ask the host to reserve a bunch for it?
> How?

From the VVID prototype for Audiality:

/*--------------------------------------------------------------------
        VVID API
--------------------------------------------------------------------*/

typedef Uint32 ADY_vvid; /* The reference */
typedef Uint32 ADY_vvid_entry; /* The object (for receivers only!) */

/*
 * Allocate 'count' Virtual Voice IDs.
 * The return value is the index of the first VVID in the host's
 * array, or a negative value if an error occurred.
 */
ADY_vvid ady_vvid_alloc(int count);

/*
 * Free a range of VVIDs previously allocated with ady_vvid_alloc().
 */
void ady_vvid_free(ADY_vvid first);

/*
 * Get the base address of the VVID table.
 *
 * IMPORTANT:
 * This may *change* between engine cycles! It is therefore
 * very important that plugins do not keep their own copies
 * of the VVID table base address across process() calls.
 */
ADY_vvid_entry *ady_vvid_table(void);

Of course, in XAP they'd be function pointers in the host struct
instead. (Audiality's "host struct" is still an implicit singleton -
or "global stuff spread all over the place", if you like. :-)

ady_vvid_table() would probably be better off as a pointer in the
struct. (Why have a function call? Can't remember why I have one
here...)

> Good feedback - looking for more - specifically, help me flush out
> the Table of Contents so all the details get in there...

Well, I'm too tired to deal with structured things like that now, so
I'll have to have another look at it tomorrow... ;-)

//David Olofson - Programmer, Composer, Open Source Advocate

.- The Return of Audiality! --------------------------------.
| Free/Open Source Audio Engine for use in Games or Studio. |
| RT and off-line synth. Scripting. Sample accurate timing. |
`---------------------------> http://olofson.net/audiality -'
   --- http://olofson.net --- http://www.reologica.se ---


New Message Reply About this list Date view Thread view Subject view Author view Other groups

This archive was generated by hypermail 2b28 : Thu Feb 06 2003 - 02:58:32 EET