Subject: Re: [linux-audio-dev] questions about real-time for audio
From: John Regehr (jdr8d_AT_cs.virginia.edu)
Date: Thu Apr 13 2000 - 20:22:03 EEST
David,
Thanks for the detailed response! Glad to hear that Linux latency is down
into the useful range for this application.
The paper I'm working on is called _Operating System Support for
Multimedia: The Programming Model Matters_. I'll post a pointer to this
list once it's ready.
John
On Thu, 13 Apr 2000, David Olofson wrote:
> On Thu, 13 Apr 2000, John Regehr wrote:
> > Hi all,
> >
> > I've been lurking on this list for a while in order to learn about audio
> > applications, but the time has come for me to ask a few questions - I'd
> > really appreciate it if someone could take time to answer them. Audio is
> > one of the real-time application domains I'm learning about as part of a
> > paper I'm writing.
> >
> > It looks like Linux dispatch latencies have gotten down into the small
> > millisecond range. Is this low enough for just about every audio app, or
> > is there still motivation to put some audio driver and/or DSP code into
> > RTLinux tasks?
>
> We're dealing with 2-5 ms latency on any decent, correctly configured
> machine, which is pretty ok for any normal "full system" studio
> setup. (Current dedicated hardware solutions, like Paris and ProTools,
> have about the same latency, or worse.)
>
> However, when dealing with "mixed systems", ie chains of digital (and
> pure analog) machines, there's no such thing as too low latency.
> Actually, there's no such thing as too low latency at all, but there
> are practical limitations to what makes sense, and what can be done
> without getting nothing but scheduling overhead.
>
> I'd say the border to insanity is somewhere around .5 ms, which can
> easily be achieved with RTLinux. It's *not*, however, easy to find
> audio interfaces and converters that can deal with latencies in that
> range, which pretty much invalidates efforts of cutting below 1 ms
> latency...
>
> > How much of the processing workload during real-time synthesis would be
> > characterized as periodic? That is, would it be useful if Linux
> > guaranteed your application that 2ms out of every 10ms of CPU time would
> > be available to it (or 20ms out of 100ms, or whatever)? If audio
> > processing is not periodic, what are the events that trigger processing
> > and how often do they occur?
>
> Most (*) time audio processing is by definition very periodic, as it
> has to be done a little piece at a time, in order to actually be real
> time. The whole point with real time processing is that there should
> be both input and output, where the input affects the output with
> sufficiently short delay. The processing can't be done before the
> input data exists. :-)
>
> (*) I say "most", because there are exceptions, such as systems
> that may pre-process data in a higher latency engine. Some mp3
> players with real time processing spring to mind.
>
> > Do most of the latency requirements come from the fact that a person is in
> > the loop, or do they come from the threat of losing data or creating sound
> > glitches due to small hardware buffers?
>
> Actually, the typical problems such as risk of buffer over/underruns
> is a side effect (and indeed a nasty one) of the requirement to lower
> the latencies.
>
> The most obvious reason for wathing low latency is probably the
> processing of live audio signals, ie virtual real time mixing. Unless
> the latency is kept somewhere below 20 ms (at the very most!), it's
> just plain impossible to use the processed sound for monitoring. That
> is, you need to bypass the native processing system with either
> analog or external DSP solutions, in order to get a useable monitor -
> in which case you lose the ability to actually hear what you're doing
> while recording.
>
> -> This requirement is hard enough that is has kept native
> processing workstations out of this business so far, but the
> lowlatency kernels already perform well enough to change this.
>
> Another example is soft synths. These require rather fast and
> accurate response to MIDI input in order to be playable. The RT
> requirements are not as hard as for the previous example, but still
> hard enough to make most serious musicians (including myself) stick
> with dedicated hardware solutions, such as rack synths and
> wavetable/sampler soundcards.
>
> -> The performance you can get from a lowlatency Linux system is
> more than sufficient for real time MIDI controlled soft synths.
> It actually beats all but the fastest hardware synths! (*)
>
> (*) Many digital synths have at least two processors; one MCU for
> MIDI decoding, envelope/LFO generation etc, and one or more for the
> actual audio processing. Some more powerful synths have dropped the
> MCU in favour for more generic DSPs/RISCs that do both the audio
> processing and the MIDI decoding. This usually means significantly
> lower latencies, as the (usually severely underpowered) MCU is
> eliminated.
>
> > When there is a glitch (audible or otherwise), is it usually because of OS
> > latency or are processors not fast enough to do the DSP in time?
>
> I have no statistics on this... Anyway, overloading your CPU is abuse,
> and can easily be avoided by allowing for a sensible amount of
> headroom. On the contrary, the non-hard real time scheduling of a
> normal OS is a real problem which simply cannot be avoided without
> fixing the OS, so I'd say *this* is the problem to worry about.
>
> > How many glitches would you tolerate from your software sound system
> > before you started looking at other solutions? For example, 1 per minute,
> > 1 per hour, 1 per century...
>
> Something like the last alternative... Monitor drop-outs, or even
> worse; capture drop-outs, are not tolerable when recording live
> stuff, and at the very least, very frustrating at all other times.
> As a real time systems programmer/control engineer, I can't see a
> technical excuse for these kind of failures, and as a
> singer/songwriter, I can't tolerate time and inspiration being wasted
> due to tools built on insufficient platforms. Restarting Windoze due
> to an application crash once or twice a day is too much already, and
> that doesn't even happen when recording.
>
> Maybe I'm just a little oversensitive...? ;-)
>
>
> //David
>
>
> .- M u C o S --------------------------------. .- David Olofson ------.
> | A Free/Open Multimedia | | Audio Hacker |
> | Plugin and Integration Standard | | Linux Advocate |
> `------------> http://www.linuxdj.com/mucos -' | Open Source Advocate |
> .- A u d i a l i t y ------------------------. | Singer |
> | Rock Solid Low Latency Signal Processing | | Songwriter |
> `---> http://www.angelfire.com/or/audiality -' `-> david_AT_linuxdj.com -'
>
This archive was generated by hypermail 2b28 : Thu Apr 13 2000 - 21:00:02 EEST