Subject: Re: [linux-audio-dev] LADSPA hard_rt_capable
From: Jack O'Quin (joq_AT_io.com)
Date: Wed Dec 13 2000 - 02:16:08 EET
I've been thinking about the requirements of realtime audio, including
the many interesting comments on this list. Clearly, you folks
understand it very well, but it's still easy to underestimate the
complexity of realtime programming. The main difficulty is that
"realtime" is a global attribute of the entire system, which any
component can mess up by being careless in subtle ways.
So, I've come to the unwelcome conclusion that in many cases, realtime
audio probably requires at least *two* different-priority threads.
Maybe three. And, that is not even considering SMP.
In support of this idea, I note that Csound implements three different
cyclical rates at which variables can change their values. Here is an
excerpt from the Csound manual:
csound> There are four possible rates:
csound>
csound> 1) once only, at orchestra setup time (effectively a
csound> permanent assignment);
csound> 2) once at the beginning of each note (at initialization
csound> (init) time: I-rate);
csound> 3) once every performance-time control loop (perf time
csound> control rate, or K-rate);
csound> 4) once each sound sample of every control loop (perf time
csound> audio rate, or A-rate).
Borrowing this terminology only for purposes of discourse, I observe
that most existing Linux realtime audio seems to run at either I-rate
(various MIDI control events, GUI commands), or A-rate (hard disk
recording, mixing). I'm speaking somewhat loosely, here. There may
be some plugins that would like to use something in between, like the
Csound K-rate. Perhaps MIDI continuous controllers are in this
category. I don't know. Perhaps not.
I have observed discussions between I-rate (not "irate") programmers
and A-rate programmers about whether MIDI is really a realtime
protocol. Clearly, it is. Responses at least in the low millisecond
range are essential in many cases for keyboard control input, for
example. Some would probably argue for even faster response times.
But, I believe that Ardour has to deal with much tighter response
requirements. Handling complex MIDI requests while simultaneously
reading or writing 24 (or more) channels of digital audio to disk is a
very difficult challenge.
In many sophisticated realtime systems there is a hierarchy of
priorities representing different guaranteed latencies. Code runs at
a high priority when it needs to respond with low latency. This, in
turn, places a requirement on every program running at or above that
priority to keep its worst-case pathlength within some constrained
limit. Other realtime programs, running at lower priorities, can be
allowed much longer pathlengths because their guaranteed response is
not so tight, and because their lower priority allows them to run
without affecting higher-priority activities.
I like to visualize this phenomenon as a "realtime response pyramid".
High-priority tasks at the top of the pyramid must execute in a short
time, symbolized by the pyramid's narrow width near its peak.
Low-priority tasks nearer the pyramid's base, can run much longer, if
necessary.
On Mon, 11 Dec 2000, Steve Harris wrote:
> > I've a few plugins that can't be marked as HARD_RT_CAPABLE because
> >thier cycle consumption varies too much when you change parameters
> >(ie. they use parameter watching and only rebuild tables if they need
> >to) otherwise they would be too slow with small chunk sizes. But this
> >means that they can't be flagged as RT_CAPABLE, even though they
> >don't use malloc or anything nasty like that.
> >
> > Is thier any advantage to allowing them to flag that they have
> >unpredictable CPU consumption, but are otherwise safe? I'm not sure
> >if that helps the host.
Using Csound terminology, the problem Steve describes is a case of
A-rate code being modulated due to I-rate (or maybe K-rate) events.
This makes the single HARD_RT_CAPABLE flag of LADSPA seem overly
simplified.
I realize there's probably nothing LADSPA, itself, can do about this.
But, it seems that relatively sophisticated hosts may wish to
implement the realtime response pyramid I've been describing. Perhaps
some already do.
For this, it would be helpful for plugins to describe their realtime
properties in terms of priority classifications. Maybe, Steve could
mark his plugin as A_RATE_CAPABLE, but with a parameter modification
routine that is only I_RATE_CAPABLE, for example. Perhaps that should
be handled by creating a separate, but related plugin. I don't know.
Benno Senoner <sbenno_AT_gardena.net> replied:
> Hi, yes these plugins are a bit a dilemma, especially when it comes to
> small block sizes. How does one know for example that a plug can work
> with a block size of X on a CPU with Y Mhz during worst case scenarios
> ? (user/audiomation software varies parameters like mad (and thus
> causing recalculation of tables during each run() cycle) In LADSPA it
> will probably not make sense, but on MAIA we could implement a system
> where these calculations are performed in a lower priority thread and
> where the results are delivered in an atomic way to the run()
> callback. Otherwise [snip...]
Benno seems to be thinking along similar lines. I don't know enough
to comment on his MAIA and LADSPA comparison, but I agree with his
idea of recalculating tables in a lower priority thread.
Essentially, I'm arguing that the extra complexity of describing a few
distinct realtime priority classifications is worthwhile, because they
actually simplify the truly difficult issue in realtime systems:
spelling out very clearly what the realtime characteristics must be
for every component.
This is not a fully thought out proposal, just some ideas I've been
kicking around. I'm open to comments and criticism.
-- Jack O'Quin Austin, Texas, USA
This archive was generated by hypermail 2b28 : Wed Dec 13 2000 - 03:59:04 EET