Subject: Re: [linux-audio-dev] LAAGA API Proposal
From: Richard Guenther (rguenth_AT_tat.physik.uni-tuebingen.de)
Date: Tue Jun 12 2001 - 18:41:32 EEST
On Tue, 12 Jun 2001, Paul Davis wrote:
> >Just to speak with code, I've (quick and dirty) hacked something
> >up which is different from Pauls approach. Its in "commented
> >header" style, it helps if you know GLAME, but I'll try to
> >elaborate. Note that I dont explicitly show implementation details
> >(f.i. if using fds or not - I'm less convinced than before that
> >this would be a good idea, though I'd like to use poll/select).
>
> there is something appealing about this model, but i still have a
> major reservation about forcing clients to deal with the buffer
> model. if i understand the model correctly, this means that
Before answering the following questions I like to tell about my view
of the typical usage pattern from the UI point of view. Lets suppose
a simple setup with one recording app, one LADSPA app (i.e. just a
bunch of connected ladspa filters - I dont think we're going to have
a one-to-one LADSPA-plugin--LAAGA-app relation here), one playback app
(which may be the recording app itself) and one visualization app
(just a levelmeter f.i.).
The user would fire up all these apps and they would at some point
register their outputs as port, i.e. the recording app registers
two ports, "audioio:left" and "audioio:right" (remember: stereo),
the LADSPA app does the same (if the internal network does indeed
have two output ports), "ladspa:left", "ladspa:right". Now the user
would select "Connect to LAAGA app" inside the LADSPA app which
would fire up a list of available ports and let the user select.
The LADSPA app now connects to the selected ports (the audioio app).
The user repeats this procedure for the playback app (which may
be the audioio app) and the visualization app.
So connections are always instantiated by the consuming end.
Now for the questions...
> (a) they need to manage their own ConnectionHandles within
> their processing code
They need to "store" them somewhere, of course - but see below.
> (b) they need to understand the loop-until-no-more-buffers
> concept when reading buffers within their processing
> code (i.e. clients have to do their own port mixing, etc.)
I dont understand this sentence - so first for the bracketed stuff:
We dont have "automatic" mixing (I think the concept of automatic
mangling in any form is broken - the only useful think is buffer
replication (by reference or copying), but without change - this
allows to abstract the datatype from the LAAGA backend (remember
the guys speaking about midi...)).
loop-until-no-more-buffers... well - you usually (for mono operation,
i.e. independend processing of connections) would do this inside
a thread (one per connection). For processing multiple connections
at once (f.i. mixing) you need to be able to handle different sized
buffers from connections.
Now a general comment: You absolutely can wrap an API like yours
(with callbacks) around the above concept - but not the other way
around. And for the simple cases (with your API, an app with two
input ports (stereo) will receive buffers on those in sync? I.e.
inside one callback? I dont see how you handle this at all) its
nothing more complicated than a callback model (for which you need
a signalwaiting thread, too (its just not "explicit", but hidden
inside a lib) - if you want to have a GUI).
> for me, this is a significant disincentive to using an API like
> this. If its possible to do without it, I don't want the internal
> model of the library/engine to structure how my clients operate. hence
Have I mentioned that in my above example we dont have an engine at all?
(you could call the audioio app the engine, though)
> my preference for a model where once you get the address of the single
> memory region associated with the port, its just a chunk of memory as
> it would be in a very simple plugin system (e.g. LADSPA). that is, its
Hey wait - we dont want to do LADSPA2, do we? Interprocess communication
is nowhere like connecting LADSPA plugins (though its possible, but you
dont want that).
And you can always do an additional API layer on top of a general approach
to handle the very simple cases.
> not that i don't think this would work (GLAME shows that it does), but
> that its not ideal as an API intended to be dramatically simplify
> audio i/o at the same time as providing inter-application
> communication and cooperation.
I dont think explicit buffer handling is complex at all. It just makes
things like echo/delay/feedback possible without doing handcrafted
ringbuffers (and as such avoids unnecessary copies).
> >The following approach (handling Ports and Connections not symmetrically)
> >simply sidesteppes the issue of on-line re-connecting (the Graph is
> >always online, there is no thing like "start", all connections get
> >managed by the destination, the ports by the sources).
>
> although it seems ideal, i don't think we can sidestep graph
> reconfiguration. LAAGA is a synchronous system, driven by something
> like an audio interface (note the word "like"). its not like GLAME
> which allows totally async operation of all components. hence, the
> engine has to know how to drive the graph, which means knowing the
> graph order at all times. did i get this wrong?
Well, my proposed LAAGA _is_ actually async. by concept (but not
practically in real-world operation when driven by audio I/O) - you'll
notice that when thinking about sound rendering - you'll get automatic
pipelining/parallelization.
> if you move to an async model, then this is very nice, but i don't see
> how it would work for low latency/real-time demands. comments?
Err - first you say (>> is me, > is you)
>>Note that you cannot avoid "loosing" frames during such operation
>>(obviously - a worker reorder is not an atomic operation - making
>>it atomic would involve "stalling" execution and such probably
>>lead to other problems as well).
>
>agreed. i have long been an advocate of this position. sometimes, you
>may avoid an audio glitch due to good luck and excellent timing, but
>there's no guarantee of it.
now you tell me that having low latency/real-time demands are dependant
on the choice of sync./async. operation? If you do on-line reconnects
you are screwed anyway (you'll probably get dropouts at the end, but
not from the recording side - i.e latency does not suffer). If the graph
is async. driven - but by audio I/O - async. and sync. operation are
the _same_ - latency is then totally dominated by the number of context
switches and the implemenation of IPC (on which both you are dependant in
sync. operation, too).
Richard.
-- Richard Guenther <richard.guenther_AT_uni-tuebingen.de> WWW: http://www.tat.physik.uni-tuebingen.de/~rguenth/ The GLAME Project: http://www.glame.de/
This archive was generated by hypermail 2b28 : Tue Jun 12 2001 - 19:47:51 EEST