Subject: [linux-audio-dev] LADMEA revisited (was: LAAGA and supporting programs)
From: Richard W.E. Furse (richard_AT_muse.demon.co.uk)
Date: Sat Sep 29 2001 - 21:36:36 EEST
> -----Original Message-----
> From: owner-linux-audio-dev_AT_music.columbia.edu
> [mailto:owner-linux-audio-dev_AT_music.columbia.edu]On Behalf Of Paul Davis
> Sent: 28 September 2001 13:33
> To: linux-audio-dev_AT_music.columbia.edu
> Subject: Re: [linux-audio-dev] LAAGA and supporting programs
>
>
> >When can we expect LAAGA to be stable enough (API-wise) and be supported
> >by linux audio programs? It's the thing I've waited the most in Linux
> >related development, especially for the syncing capabilities it offers.
>
> * OK, can we start calling it JACK now, or has Karl's paper done us in ?
>
> its a good question. i'm sorry that i don't have an answer right
> now. i believe that the API as far as it currently exists will be
> stable. so far, several people on the list have been enthusiastic
> about it; conversely Richard Furse felt that the API wasn't adequate
> and offered his own much more, uhm, complex API. there wasn't much
> comment on that, but what there was didn't seem convinced by the need
> for more complexity. we need to be sure about this, because it would
> be crazy to proceed with the current delightfully straightforward API
> and then find out that Richard was right, and it can handle other things.
[...]
Hmmm, sorry I've not done a lot more on this - I'm very busy with other
things at the moment.
The original LADMEA prototype remains available at
http://www.ladspa.org/ladmea/ but I've not had time to build an SDK for it.
The API isn't long - I think if a people got their heads around it there
might be some debate.
The essential idea is to extend from approaches like LAAGA/JACK, GStreamer
or aRts where there is an all-encompassing audio infrastructure (an
"exchange") into which various client applications are slotted. In the
LADMEA world, "clients" and "exchanges" communicate through the lightweight
LADMEA interface. Clients can load exchanges, exchanges can load clients,
some other program can load both (when in library form).
This would mean that a newly written audio synth written using LADMEA could
immediately be slotted into LAAGA, GStreamer or aRts (assuming LADMEA
support) or use a LADMEA interface to ALSA. Correspondingly, if someone
writes a new framework for audio streaming over ATM (perhaps using
compression on some channels as bandwidth requires) then this can
immediately be used with the client applications such as recorders, players,
multitracks etc. Just about anything can be a client and it shouldn't be
hard to retrofit existing software such as Csound to read and write data to
LADMEA exchanges. And the choice of exchange can be tailored to the task:
some might allow crude and fast connections using RAM, some might allow
latent but high bandwidth connections across networks using compression,
some might be happy only with point/point connections, some may be happy
managing complex remote graphs. The way all this is managed is up to the
exchange - which might have a graph-based GUI, command line
connect/disconnect calls, a fixed patch-bay format or whatever.
I do not think that LADMEA is a particularly complex solution and hopefully
this is bourne out by a less than cursory investigation. The reason it
*appears* complex is that I've attempted to address a lot of issues in a
fairly confined space. Apart from the more obvious, somes of these include:
1. How can a client transmit a data type to another (potentially remote)
client across a exchange that has never heard of the data type?
2. How can a client know what existing channels it can use?
3. If clients offer variants of data types (e.g. 16bit unsigned
little-endian PCM, float l-e PCM, double l-e PCM, MP3), how can the exchange
persuade the clients to agree? If they cannot, how can the exchange go about
inserting a codec? Note again that the exchange may never have heard of the
data types involved.
4. How does a exchange know how much bandwidth it is likely to need to
communicate data (e.g. over a network)? How latent can data be? How much
jitter may be tolerated on a network? Note again...
5. How can an exchange know when a client has failed?
6. If sources go out of sync (e.g. audio streaming over an ISDN link across
the Atlantic drifts in sync relative to local overdubs) then how does the
exchange know this is happening and deal with it. (Ok, most exchanges won't,
but they could if they wanted.)
7. Consider a case where none of the clients in a graph require live
operation, e.g. a MIDI player is playing a 10min piece where the MIDI is
sent to a soft synth which generates PCM audio that is then passed to a
sound file writer. Say this graph can be run live and uses 20% CPU time on a
single CPU system. An intelligent exchange should be able to work out that
the graph instead can be run "offline" in 2mins as there is no requirement
to synthesise in real time. The same thing applies for subgraphs and
exchanges should be allowed access to enough information to cache output
from subgraphs. Again, many exchanges wouldn't wish to support such
facilities - but there ought to be a way to make these things options.
What worries me about LAAGA/JACK is that a very specific approach is taken
to exchange implementation that isn't right for some tasks (e.g. bandwidth
management, networking, handling of varied data types) although it is very
good at some other things (e.g. low latency PCM audio). What I'd like us to
investigate is a way forward in which clients can still make use of
LAAGA/JACK when these features are the user's priority, but don't restrict
the exchange to this form. In particularly, IMHO remote inter-application
communication for audio software is HUGELY important for the future and I
think the Linux audio community really needs to address it sooner rather
than later. This API should be perfectly adequate for a "rewire"-style
exchange. It also should be fine for a world in which a studio is made up of
a network of rackmount Linux boxes, MIDI controllers, SP/DIF connections and
ADSL connections to other studios.
There are a couple of issues that have been brought up with the LADMEA API,
both of which I consider fairly minor (let me know if I've forgotten
anything). These are:
a. There's no facility to perform a run_adding() type operation. Yep, this
has a performance cost but I think that's small in the scheme of things.
There are ways to bring this facility back if really required but I'd prefer
to leave this issue at least for the moment as the solutions seem
conceptually untidy to me.
b. Folk have argued that memory management for transmitted data structures
should be provided by the exchange. Yep, this makes very good sense,
particularly for exchanges that would want to use SHM as it allows us to
save a memcpy() when data is sent. I think this is pretty easy to add a
little later and I'd prefer to do it then rather than further clutter the
API now. It's not conceptually necessary, just a (very useful) speed
optimisation.
--Richard
This archive was generated by hypermail 2b28 : Sat Sep 29 2001 - 21:38:06 EEST