Subject: RE: [linux-audio-dev] LADMEA Prototype
From: Richard W.E. Furse (richard_AT_muse.demon.co.uk)
Date: Thu Aug 16 2001 - 22:48:59 EEST
The problem is isomorphic to disk I/O only in that I/O happens - some
exchange implementations might perform the lazy-writing for you. Whether
this happens or not, I don't see why LADMEA suffers from "related issues to
do with MIDI timing". What are these? LADMEA allows the client to output the
data the moment it has it (however it gets it). What could be better than
that?
If you don't allow asynchronous data you introduce unnecessary latency
averaging half your tick period into your signal chain. Asynchronous data
isn't that unusual once you go beyond PCM.
It sounds as if we're using "inactive" in two different senses. When we mean
the client is not switched on, the exchange will know this and either refuse
to run an incomplete graph or feed silence (or noise?) into the relevant
channel. When a client is broken or late, a exchange can know this and fail
or compensate using a variety of techniques. LAAGA's approach is one option.
When receiving data on a number of channels, you don't need a mutex because
of the way the client API is specified (unless I've made a mistake). This is
the exchange's problem. All data is timestamped already so the client has an
explicit of the "logical time" data relates to. Timestamps only need to be
checked when the client is interested in this data in its own right or when
the client has announced that it can tolerate latency on its inputs [Hmmm,
Richard now realises that the default latency requirement structure needs to
include a lower bound as well as an upper bound on latency so the client can
say how early data can be - easy to fix!].
Again I agree with your buffering comments but would prefer not to implement
at this stage to keep the API simple. The zero copy on 16channels of 44.1kHz
float audio is under 3Mb/sec which is a significant but not back-breaking
cost. For current test purposes I think this is tolerable.
--Richard
-----Original Message-----
From: owner-linux-audio-dev_AT_music.columbia.edu
[mailto:owner-linux-audio-dev_AT_music.columbia.edu]On Behalf Of Paul Davis
Sent: 16 August 2001 13:41
To: linux-audio-dev_AT_music.columbia.edu
Subject: Re: [linux-audio-dev] LADMEA Prototype
>Yep, I understand in LAAGA the client *must* do something when the clock
>ticks - this is fundamentally the way that LAAGA does things. It isn't
>necessarily useful - for instance, a client listening to a MIDI port will
>generate data asynchronously and will have no interest in the tick (its
only
>interest may be in timestamping the data). How would LAAGA stream such MIDI
>data?
Every time the client's process() callback is called, the client has a
chance to pass MIDI data to its ports. Nobody would sensibly write a
client that collected MIDI data in the same thread as the one in which
the callback was made - the problem is completely isomorphous to disk
i/o. you gather data in one thread (possibly timestamp it, which LAAGA
allows to occur from other threads), and pass it along when process()
is called.
Now, it just so happens that MIDI is problematic, along with any other
non-continuous protocol, since the tick system would force MIDI output
to occur only at (or around) ticks. However, I don't see a general
solution to this - the streaming model that LADMEA offers suffers from
related issues to do with MIDI timing. You really have to use an
independently scheduled thread that uses a constantly reset timer to
deliver MIDI properly. Integrating such protocols with continuously
streaming ones like audio and video is non-trivial.
> The tick mechanism more-or-less works for audio (because of its
>naturally fixed block size) but isn't even optimal there (e.g. when audio
>can be calculated ahead-of-time
again, the tick mechanism does not control when audio is generated. it
controls when audio or other data is *passed* to ports. because of the
design space for LAAGA, there has been a focus on real-time clients
that generate audio within their process() callback. but other clients
could easily be generating audio in another thread and get way
ahead. the purpose of the tick mechanism there is to ensure sample
sync between every client, in the sense that the audio delivered to
the audio interface from every client comes from the same "point in time".
or when block sizes vary or when the
>application is in more than one process/machine). LADMEA doesn't stop you a
>clock/tick when it's useful - it just doesn't force you to.
to repeat: LAAGA does not require that you generate audio or anything
else on the tick. it merely uses the tick to say "now i'm ready for
your data/i have data for you". you may not have the data ready to go
(a real-time synth client) or you may (a disk file player); LAAGA
doesn't care.
>I don't agree with your fail-over (and "inactive") logic. In your example
>where a sound processor fails, I don't see why LAAGA gives you an
advantage.
>If no audio is generated on a channel, there's not a lot that can be done
>about it.
Is this how gear in a studio works? If my Quadraverb isn't "generating
any audio", does my mixer stop because its waiting for the QV? No, of
course not. Designing a system in which the "inactive" state of a
component stalls the entire system until it is reactivated is
completely unacceptable to me.
> I suppose the LAAGA analogy is that the call to a client does not
>return fast enough.
No, thats an absolutely different error condition.
> This will bring LAAGA to a standstill or glitch.
The first tick in which it happens, yes it will glitch. It will then
remove the client from the processing graph, automatically breaking
all connections to and from it, and then continue smoothly.
>It is not true that "*every* kind of client has to do the same kind of
check
>as the hypothetical patchbay does on its
>ports-at-which-data-arrives".
> [ ... ]
>Checks *do* have to be made where there is more than one input
>required, e.g. in a patchbay, FX process with real-time controls or
>combined MIDI/audio sequencer or where there is some external binding
>to realtime (e.g. a realtime GUI component). These checks are hardly
>onerous however ("if (audiochanframes > 0 && controlchanframes > 0)
>{...}") given the flexibility provided.
Thats not an adequate check if data can arrive asynchronously. The
actual check requires (1) a mutex (possibly inter-process) (2)
timestamped data (3) checking the timestamps.
>The comments about buffer management are fair - I missed this out for
>simplicity. I think everything else present is necessary - adding buffer
>management isn't, although it could provide a small optimisation in some
>contexts. I'd prefer not to put anything in at this stage to keep the
weight
>of the API down - it's easy enough to add later. I don't actually think
this
>optimisation (or the run/run_adding kind) is especially significant in this
>context - LADMEA is intended primarily for inter-app communication, so
>zero-copy isn't nearly such a priority as in LADSPA IMHO.
Thats precisely where it *is* important, IMHO. Within LADSPA, we get
zero copy pretty much for free, since the host does buffer management
and everything is in the same address space. Moreover, most plugins
are mono or stereo at best. But between applications we may be
shovelling many channels of data from multiple applications into, say,
a mixer application. Thats precisely where zero copy is so valuable,
if for no other reason than it avoids trashing the cache lines that
reference data being moved from app to app, which LADMEA will do every
time.
If everything is in-process, this problem goes away, but the chances
of MusE or Ardour or SpiralLoops or any other significant application
being converted to in-process clients of any such protocol is more or
less zero, I think.
--p
This archive was generated by hypermail 2b28 : Thu Aug 16 2001 - 22:50:53 EEST