Subject: Re: my take on the "virtual studio" (monolith vs plugins) ... was Re: [linux-audio-dev] ardour, LADSPA, a marriage
From: Stefan Westerfeld (stefan_AT_space.twc.de)
Date: Sat Nov 25 2000 - 13:43:57 EET
Hi!
On Thu, Nov 23, 2000 at 10:11:26AM -0500, Paul Barton-Davis wrote:
> >> well, here we come to the crux of things. if MCOP can really be used
> >> in applications that seek to offer the same kind of performance as
> >> dedicated h/w boxes, then i'm all for learning it and using it. [...]
> >>
> >> Stefan is probably right that MCOP could be improved/extended to
> >> handle such systems. But I suspect there is real work there, work that
> >> is not guaranteed to result in a version of MCOP that can still be
> >> considered universal in its utility. if stefan really believes that
> >> the fixes for this are fairly trivial superficial details, i'm very
> >> interested to know more. [...]
> >
> >Okay, this could get really constructive here. We'll make a test. I will
> >give you three of the assumptions that I use, with an estimate how hard
> >it would be to undo (fix) them, then you can judge for yourself if it is
> >suitable for your purpose.
> >
> >1. cross server object connection is done blocking
>
> I mostly don't care about this, because in the instances I care about,
> such connections would never occur.
>
> I hope i correctly understand the aRts model to include a 1:1 mapping
> between a single audio driver API's "audio device" and a server (I
> made that complex, since ALSA now provides "multi" support which can
> make multiple soundcards appear to be a single device to the
> application).
You are right that if you have two different audio devices, you could and
probably should create two artsd processes for that (i.e. two sound servers).
But a server is more than that. A good definition of a server is: a process
which contains connectable MCOP objects. As such, for instance a GUI server,
on which you can run various controls (such as leds, buttons, scales, ...)
is an MCOP server.
Or many applications which are talking to artsd.
> [...]
> >2. audio data = floats
>
> No problem here. I would introduce one other limitation: audio data is
> range-limited to -1.0 to +1.0. For 24 bit samples, this can be done
> with 32 bit IEEE floats with no loss of resolution in a simple
> passthru situation.
I intentionally didn't limit it to -1.0 .. +1.0, because you can save
computation power if you don't. You simply can add, scale, filter,...
voices all over your network, without caring about clipping, out-of-range
situations and so on. Only in the end, when you really render it (i.e. to
a soundcard) you do clipping.
> >3. mallocs have reasonable costs
>
> >If you only run a fixed set of modules on one server, you don't need malloc.
> >All other operations, like connecting modules, sending events, calling
> >methods on other servers, connecting new servers,... do mallocs.
>
> the set of operations you describe are things that i simply don't
> think should be happening during the execution of the "interrupt
> handler" in the RT/SCHED_FIFO audio thread.
Well, aRts uses no threads at all. This means, there is no audio thread, and
no "interrupt handler". So it probably has quite a bit more impact than you
thought. I see two possible ways to go:
* stay with the single-threaded (never locking anything) model of
cooperative CPU sharing, then you need to do something like "reschedule
points", i.e.
...
if(audio_work_pending()) do_audio_work();
...
this is probably quite efficient, quite easy to understand and program,
and catch quite a few cases, but it will not ultimately avoid the case
where malloc takes 20ms due to swapping
* split up into threads - doing this intelligently is a real issue, because
if you for instance use locking to ensure that the flow system data
structures stay consistent during "connect", unimportant threads will
be able to "lock out" the audio thread
If it would be possible to more or less keep all features you have now, and
provide RT safe behaviour in any situation (even if you do connect things,
do network access,...), i.e. due to the use of lock-free data structures
combined with threads, this would be quite nice, though it would probably
at least break binary compatibility, if not large parts of the source
compatibility.
Anyway, for the answers below I'll assume you say: "well, then I don't need
to connect or create things while doing a realtime performance" ;).
> let me mention some quite different features that turn out to be very
> important in my audio software, and that i would not have guessed at
> before i started:
>
> * ability to handle non-interleaved data streams with no performance
> penalty
There are no interleaved data streams inside the aRts flow system, so this
is no problem.
> * ability to do per-channel operations with no performance
> penalty (more or less equivalent to the first point)
No problem either.
> * proper support for pro audio cards (sample clock settings,
> hardware monitoring settings, sync source settings,
> etc. etc.). These all need to be present in the
> abstraction offered to the low-level layers in the
> system.
There is currently no code for >2 channel hardware in aRts. I would imagine
that it would be something like adding one or two modules for "pro audio
input" and "pro audio output", and putting the complexity inside those. I
have seen that the code you use in ardour to control the hardware is in
fact quite long ;).
> * fragment wakeup count == 1 causes no problems
?
> * moving 9MB/sec of data to/from the audio interface is OK
I think moving data around should be computationally cheap compared to
applying the actual algorithms. So I would guess that if you can supply
9MB/sec from your algorithms (i.e. filters, hd reading, ...), you can
also move them to the audio device.
> * use of RTC as a timing source when needed
Well, there is no real timing architecture besides "system time" driven
timers. I plan to add "sample time" driven timers, too, so that you can
time on output samples rather than system time. Making "system time" use
RTC instead of gettimeofday should be possible.
> * 2.5msec - 5msec latency (given a patched kernel)
I tried the following:
* 3 fragments with 128 bytes each
* => effective latency = 2.1ms
* software synthesis (live midi)
* unpatched 2.2.x kernel
* realtime priority
* unloaded PII-350 system
With the current code, you'll observe the following behaviour: once you have
done something, like striking a chord, the sound is stable - when you add
or release notes, the sound gets choppy. This is because the aRts midi synth
code creates and removes new voices live as needed, and of course in turn
does lots of connect operations (which are non-realtimed, as you know),
mallocs and so on - all in the same thread.
You could solve this issue at least by rewriting it in a way that voices are
not dissolved or created dynamically but rather statically.
Anyway, the result also means, given a patched kernel, and an application
which uses no reconnects and no dynamic object creation while RT performance,
the code should run stable at that latency.
Cu... Stefan
--
-* Stefan Westerfeld, stefan_AT_space.twc.de (PGP!), Hamburg/Germany
KDE Developer, project infos at http://space.twc.de/~stefan/kde *-
This archive was generated by hypermail 2b28 : Sat Nov 25 2000 - 14:39:11 EET