Re: [linux-audio-dev] XAP spec - early scribbles

New Message Reply About this list Date view Thread view Subject view Author view Other groups

Subject: Re: [linux-audio-dev] XAP spec - early scribbles
From: torbenh_AT_gmx.de
Date: Thu Feb 27 2003 - 08:47:22 EET


On Tue, Feb 25, 2003 at 06:20:40PM +0100, David Olofson wrote:
> On Monday 24 February 2003 11.24, torbenh_AT_gmx.de wrote:
> > On Fri, Feb 21, 2003 at 04:07:54PM +0100, David Olofson wrote:
> > > Hi, and wellcome! :-)
> > >
> > > BTW, do you know about the GMPI discussion? The XAP team is over
> > > there as well.
> > >
> > > http://www.freelists.org/cgi-bin/list?list_id=gmpi
> >
> > oh no... another mailing list i must read :)
>
> *hehe* I switched to full height folder tree a while ago for this
> reason... Way too many lists.
>
>
> [...]
> > My problem is now: How can i map something like the clock
> > which is a pure event generator to the XAP model ?
> >
> > it must get called somehow if the event delivery does not
> > provide for a callback.
>
> There's only one call; process(). (Or run(), as some prefer to call
> it.) This is called once for each block, whether or not the plugin
> processes audio. Might sound odd, but considering that the unit for
> event timestamps is audio frames, it turns out to be really rather
> logical - in theory as will as implementation.

Yeah youre right.
But then i have to implement graph ordering.
which galan does not do at the Moment.
I will also have to isolate the OpenGL stuff from
the audio processing... Well i have to do that
anyway...

>
> [...]

>
>
> [...compressor...]
> > revelation !
> > (ah... now i got your idea: this is very cool)
>
> Well, the idea is really rather simple: Just think of the event
> "streams" as somethig very similar to audiom streams. They're just
> structured and more flexible. Unit graphs can be constructed
> according to the exact same rules.

ok.

> > > 3) Doing anything like this just to *slightly* increase
> > > the chances of soft real time worker threads finishing
> > > within a single buffer cycle, while the audio thread
> > > is working, in a way that can only work on SMP machines,
> > > is rather pointless. If you fire a worker callback, you
> > > do it because you suspect it might take so long to
> > > finish that doing the work in the audio thread would
> > > cause a drop-out. That is, you're not expecting a
> > > result until one or more blocks later.
> >
> > The example i was thinking of, was a galan like clock
> > running at 10Hz or so sequencing a list of wav file names.
> > with the galan like clock the event is inserted into
> > the queues 100ms before it is due.
>
> That means it's not really a real time unit, since you know long ahead
> what it's going to say.
>
> Or do you?
>
> No. If you stop the clock, or change it's rate, the upcoming enqueued
> event becomes invalid, and must be modified or replaced for correct
> results.

right this is how its implemented.
hmm... ok... i will see how this can all be changed.
i need to get to work on the 0.3.0 branch again to get it out
of the testing state.

> If each plugin only ever worries about the current block for I/O, this
> is not an issue, because plugins would never generate events that
> they may need to take back later.

hmmm... this would make the event delay more complex but also more powerful...

>
>
> > if the event processing would track the event processing
> > queue through the wav file list to the sample loader.
> > it would know that in 100ms it must load sample x in 0seconds.
> > This is where i want the worker thread to be fired.
> > it has 100ms to get the wav loaded.
>
> Yeah, but what if the clock runs at 100 Hz...? How does 10 Hz
> automatically mean that the next event is known 100 ms ahead of time?
>
> I think you're looking in the wrong place for "look-ahead" data. If
> you need to know something X ms ahead of time, you simply need a
> protocol that allows the sequencer to send "prepare" events of some
> kind X ms before the real event.
>
> Obviously, this can only work with sequenced data. If you're
> generating events from like MIDI input or similar, there's just no
> way to do it.
>
> This suggests to me that "look-ahead" is a special case interface for
> some specific cases; not an idea to build a real time API upon.

yes...
a sequencer could have a clock and a prepare-clock
and then outs and prepare-outs....
this fits on top of XAP.

hmmm... but what if

an event gate:

if it gets an event on input
  if the last event on tocomp == the last event on comp
    output event from input.

PRIVATE void evt_input_handler(Generator *g, AEvent *event) {
  
  Data *data = (Data *) g->data;
  if( data->open ) {
      gen_send_events(g, EVT_OUTPUT, -1, event);
  }
}

PRIVATE void evt_tocomp_handler(Generator *g, AEvent *event) {
  
  Data *data = (Data *) g->data;
  if( event->d.number == data->comp )
      data->open = TRUE;
  else
      data->open = FALSE;
}

PRIVATE void evt_comp_handler(Generator *g, AEvent *event) {
  ((Data *) g->data)->comp = event->d.number;
}

                                        /--- trigger input
                                       / |
                                      / +---<----+
                                      | | |
                                      | evtdelay ^ The Loop is okey because of the delay
                                      | | |
                                      | | |
                                      | | |
                              dont do next trigger +----+ |
                                                \ | | |
                                                 \ | delay |
                                                evtgate |
                                                   | |
                                                   | |
                                                   | |
                                                   +--------+
                                                   |

this is not very excact but it is possible to build such
a mesh in galan. hope you get what i mean from the text....

Assuming there is only one event in the loop..
receipt of a new event on trigger input would
resync the clock but only on next trigger.

dont do next trigger closes the gate.
which opens again after it has blocked an
event.

in XAP this event feedback would already be illegal.
(but if the code was smart enough it could solve
 the cycle at the delay)

in galan this is legal but provides the user with
the possibility to build meshes which are not realtime
safe... you can also build for loops etc...

if there was a component which converted a string event
to a sample event, this could be connected before the
delay.

so the sample would already be loaded when it is time
for the delay to let the event pass...

the sample loading component would fire a worker thread,
which inserts the event with the same timestamp
as the incoming event... (at this point the timestamp is
in the past)
the evtdelay adjusts the timestamp to be in the future
again.

i take this approach for midi in also...
(not implemented yet)
the midi event has a timestamp from alsa, which corresponds
to the past. without the delay it would be processed now..
this would generate jitter.
with the delay some latency is imposed but the jitter is away.
the user can adjust the delay to his machine...

>
>
> > this seems to be not feasible, because the event peeker would be
> > fairly complex it must not follow the code path of the clock
> > posting events to itself because of the infinite loop etc... not
> > taking into account bigger event feedback loops....
>
> The "event peeker" has to be part of the clock and send events that
> are part of a special protocol for this task.
>
> We have something similar in XAP, meant for hard disk recorders and
> other plugins that need pre-buffering time before they can start
> playing from a random song position. A sequencer (or rather,
> timeline) can inform such plugins about loop and jump points
> beforehand, so that specific pre-buffering can be used to make it
> possible to jump to these points with zero latency.

ok.. it is time for me to look at the XAP specs...
where are they ?
as it seems you are already defining the higher levels...

>
>
> [...]
> > > > the synth could tell that it wants to be processed when the
> > > > event it just received becomes due.
> > >
> > > How? The host won't see any events, as they're normally just
> > > added directly to the synth's event queues by inline code in the
> > > sender... Besides, even if the host snooped every single control
> > > of the synth (ie all incoming events), it wouldn't know what's
> > > relevant and what isn't. Only the synth can know what
> > > combinations of values actually produce sound.
> >
> > yes... But due to the event peeking code it would get the event
> > 100ms before it is due. The event peeker is too complicated though.
>
> In fact, it's not even possible to implement in a generic way. (See
> above.)

i think i have found a method above...
but i am not sure if it fits onto XAP....

>
>
> > I have now understood the XAP model and wont be coming up
> > with stupid ideas.
>
> Well, supporting plugin "actions" that can't be performed with "zero
> latency" isn't a stupid idea - but it's rather tricky to get right.
> It's easy enough to just have plugins send off such jobs to worker
> callbacks, but that makes the responses soft real time.
>
> That said, hardware synths, and to even greater extent, samplers, have
> this problem as well, and it's no major issue in real life. It
> *could* be dealt with, but it's probably not worth the effort or
> added complexity for things like loading samples or rendering
> waveforms. Just don't load new sounds in the middle of the song. Do
> it before starting playback.

Yes... but galan has no stop. It is always running.
But a change sample will fire a worker and leave the old sample
until the new sample arrives.

>
>
> [...]
> > > > at the Moment it uses one global event queue which
> > > > makes the performance bad if you had 2 unrelated high frequency
> > > > clocks in the mesh. But the XAP model of individual event
> > > > queues would fix this with a not small coding effort involved.
> > >
> > > Probably. It also makes it possible to avoid explicit event
> > > dispatching inside plugins with more than one inner loop, and it
> > > eliminates the need for sorting events as they'le sent.
> >
> > yes right... at least the API in galan is clean enough to make the
> > changes...
> >
> > the problem is only with the event sideeffects.
> > A pure event plugin does not get REALTIME callbacks.
> > because the plugin gets the callback upon receiving the event.
>
> I see. But then, how is an event processor able to "generate timing"
> of it's own, as in generating events that are not direct and instant
> responses to incoming events?

it is simply not possible... all is driven by clocks...
very nice you point me at all shortcomings of the galan design.

>
> An example would be the Audiality MIDI player. You just start it, and
> then it generates events with sample accurate timing, until the song
> ends, or until you stop it. It's generating it's own timing
> internally, and it derives it from the audio time, through the
> process() call. It's as simple as "Do your thing for N frames!"

ok... this means for me: implement graph traverser.

>
> [...]
> > i will make my calls inline too...
> > But how do i remove a sorting send function ?
> > i dont get that yet.
>
> Took me a good while to figure that one out as well... :-)
>
> In Audiality, it's actually quite trivial. These are the rules that
> make it work automatically:
>
> 1) Senders must send events in timestamp order.
>
> 2) Each queue must receive events from only one sender.
>
> That's it.

yeah... that does not apply to galan...

>
>
> As to XAP (and most probably future versions of Audiality), it gets a
> little more complicated, since the second rule will be violated on a
> regular basis. The first step towards solving this is to look at how
> connections can be related:
>
> 1) One plugin sends to another plugin.
> No problem. Events are sent and received in
> order.
>
> 2) One plugin sends to multiple plugins.
> Still no problem. The fact that events are
> dispatched across multiple queues by the
> sender doesn't mean they get out of order.
>
> 3) Multiple plugins send to one plugin.
> Now we're in trouble... We'll have multiple
> ordered chains of events that must form a
> single chain on the same queue.
>
>
> (Here, we are assuming that each plugin has only one input event
> queue, and sends events from only one loop, thus guaranteeing that
> *all* events output from one plugin will be in timestamp order.)
>
> What we do about this is have the host connect all (or all but one) of
> the senders to one or more "shadow queues", so that the host can
> sort/merge as required, in between process() calls. Sort/merge is a
> really rather low cost operation compared to full sort or (even
> worse) per-event insertion sort, and it means plugins - and the event
> handling inlines - don't have to worry about this.

yeah... insertion sort like in galan :)

this is a clear path to optimization you have lined out...
very good...

do want to have audiality running in the kernel ?
this is a cool idea.

too bad i missed the XAP discussions...
but i had to waste my time with 1.Semester Computer Science
(at least i heard Math 3 which was kind of interresting

it gave me a different look (more algebraic than the engeneers)
on the Analysis

>
>
> There's another problem, though, specifically for XAP: Plugins may
> have multiple input event queues and/or multiple internal event
> sending loops. (This is mostly for "monolith" plugins like
> multitimbral synths and the like, which are most likely never going
> to be implemented with a single event/audio compound loop.)
>
> The solution is to stamp all control inputs and outputs with "context
> IDs", so that the host can treat these queues and loops as separate
> plugins WRT event routing. The rest is handled the way described
> above.
>
>
> [...]
> > Well no need to look at the code i think.
> > The XAP design is far superior than galan.
> > (considering performance at least)
> >
> > The API of galan is worth a look though.
> > And now that i have understood XAP i will have a try on changing
> > the implementation to be more optimal. I am tempted to think i can
> > do it without API change.
> >
> > shit another BIG todo item :)
>
> I have plenty of those... *hehe*
>
>
> > i will have a go on audiality also
> > why is there no ebuild ?
>
> One might as well ask, why ebuild?

why rpm ?
why dep ?

>
> I've hardly heard about ebould before, although I've been confronted
> with a few other make alternatives before. However, autotools are the
> defacto standard, and I know them well enough to get the job done. I
> just don't have the time and motivation to investigate all the
> alternatives...

ebuild is not an alternative to autotools.
it is a script telling my distro (gentoo)
where to download some source, which patches, which dependencies, how to compile etc...

generally doing configure; make; make install in a sandbox
and then installing onto the system....

-- 
torben Hohn
http://galan.sourceforge.net -- The graphical Audio language


New Message Reply About this list Date view Thread view Subject view Author view Other groups

This archive was generated by hypermail 2b28 : Thu Feb 27 2003 - 12:21:46 EET