Subject: Re: [linux-audio-dev] questions about real-time for audio
From: David Olofson (david_AT_gardena.net)
Date: Thu Apr 13 2000 - 06:03:08 EEST
On Thu, 13 Apr 2000, John Regehr wrote:
> Hi all,
>
> I've been lurking on this list for a while in order to learn about audio
> applications, but the time has come for me to ask a few questions - I'd
> really appreciate it if someone could take time to answer them. Audio is
> one of the real-time application domains I'm learning about as part of a
> paper I'm writing.
>
> It looks like Linux dispatch latencies have gotten down into the small
> millisecond range. Is this low enough for just about every audio app, or
> is there still motivation to put some audio driver and/or DSP code into
> RTLinux tasks?
We're dealing with 2-5 ms latency on any decent, correctly configured
machine, which is pretty ok for any normal "full system" studio
setup. (Current dedicated hardware solutions, like Paris and ProTools,
have about the same latency, or worse.)
However, when dealing with "mixed systems", ie chains of digital (and
pure analog) machines, there's no such thing as too low latency.
Actually, there's no such thing as too low latency at all, but there
are practical limitations to what makes sense, and what can be done
without getting nothing but scheduling overhead.
I'd say the border to insanity is somewhere around .5 ms, which can
easily be achieved with RTLinux. It's *not*, however, easy to find
audio interfaces and converters that can deal with latencies in that
range, which pretty much invalidates efforts of cutting below 1 ms
latency...
> How much of the processing workload during real-time synthesis would be
> characterized as periodic? That is, would it be useful if Linux
> guaranteed your application that 2ms out of every 10ms of CPU time would
> be available to it (or 20ms out of 100ms, or whatever)? If audio
> processing is not periodic, what are the events that trigger processing
> and how often do they occur?
Most (*) time audio processing is by definition very periodic, as it
has to be done a little piece at a time, in order to actually be real
time. The whole point with real time processing is that there should
be both input and output, where the input affects the output with
sufficiently short delay. The processing can't be done before the
input data exists. :-)
(*) I say "most", because there are exceptions, such as systems
that may pre-process data in a higher latency engine. Some mp3
players with real time processing spring to mind.
> Do most of the latency requirements come from the fact that a person is in
> the loop, or do they come from the threat of losing data or creating sound
> glitches due to small hardware buffers?
Actually, the typical problems such as risk of buffer over/underruns
is a side effect (and indeed a nasty one) of the requirement to lower
the latencies.
The most obvious reason for wathing low latency is probably the
processing of live audio signals, ie virtual real time mixing. Unless
the latency is kept somewhere below 20 ms (at the very most!), it's
just plain impossible to use the processed sound for monitoring. That
is, you need to bypass the native processing system with either
analog or external DSP solutions, in order to get a useable monitor -
in which case you lose the ability to actually hear what you're doing
while recording.
-> This requirement is hard enough that is has kept native
processing workstations out of this business so far, but the
lowlatency kernels already perform well enough to change this.
Another example is soft synths. These require rather fast and
accurate response to MIDI input in order to be playable. The RT
requirements are not as hard as for the previous example, but still
hard enough to make most serious musicians (including myself) stick
with dedicated hardware solutions, such as rack synths and
wavetable/sampler soundcards.
-> The performance you can get from a lowlatency Linux system is
more than sufficient for real time MIDI controlled soft synths.
It actually beats all but the fastest hardware synths! (*)
(*) Many digital synths have at least two processors; one MCU for
MIDI decoding, envelope/LFO generation etc, and one or more for the
actual audio processing. Some more powerful synths have dropped the
MCU in favour for more generic DSPs/RISCs that do both the audio
processing and the MIDI decoding. This usually means significantly
lower latencies, as the (usually severely underpowered) MCU is
eliminated.
> When there is a glitch (audible or otherwise), is it usually because of OS
> latency or are processors not fast enough to do the DSP in time?
I have no statistics on this... Anyway, overloading your CPU is abuse,
and can easily be avoided by allowing for a sensible amount of
headroom. On the contrary, the non-hard real time scheduling of a
normal OS is a real problem which simply cannot be avoided without
fixing the OS, so I'd say *this* is the problem to worry about.
> How many glitches would you tolerate from your software sound system
> before you started looking at other solutions? For example, 1 per minute,
> 1 per hour, 1 per century...
Something like the last alternative... Monitor drop-outs, or even
worse; capture drop-outs, are not tolerable when recording live
stuff, and at the very least, very frustrating at all other times.
As a real time systems programmer/control engineer, I can't see a
technical excuse for these kind of failures, and as a
singer/songwriter, I can't tolerate time and inspiration being wasted
due to tools built on insufficient platforms. Restarting Windoze due
to an application crash once or twice a day is too much already, and
that doesn't even happen when recording.
Maybe I'm just a little oversensitive...? ;-)
//David
.- M u C o S --------------------------------. .- David Olofson ------.
| A Free/Open Multimedia | | Audio Hacker |
| Plugin and Integration Standard | | Linux Advocate |
`------------> http://www.linuxdj.com/mucos -' | Open Source Advocate |
.- A u d i a l i t y ------------------------. | Singer |
| Rock Solid Low Latency Signal Processing | | Songwriter |
`---> http://www.angelfire.com/or/audiality -' `-> david_AT_linuxdj.com -'
This archive was generated by hypermail 2b28 : Thu Apr 13 2000 - 07:06:56 EEST