Subject: Re: [linux-audio-dev] Realtime restrictions when running multiple audio apps .. Was: Re: disksampler ...
From: Stefan Westerfeld (stefan_AT_space.twc.de)
Date: Sat Jul 15 2000 - 23:30:19 EEST
Hi!
On Fri, Jul 14, 2000 at 10:45:56AM +0200, Benno Senoner wrote:
> It would be nice to hear comments from other list members
> (David Olofson , Paul , Stephan etc)
Okay, I'll try to briefly summarize my thoughts about realtime soundserving
generally, and your proposal and aRts specifically.
Generally, I think if linux is to be accepted as usable platform for audio,
we should get around the one-app-only restriction. Soundserving in the user
space is IMHO the way to go. This means, one app, the soundserver, gets
/dev/dsp, and all other apps do their sound output with the soundserver.
The soundserver should be "lossless" in a way for every app you could write
without the soundserver, there is also an equivalent implementation using
the soundserver. This rules out proposals like "we do a soundserver that
supports only IPC streaming", as this approach effectively kills all low-
latency apps.
Thus, a realtime soundserver must somehow allow in-process execution, so
that the realtime parts of apps can be gathered inside the soundserver.
About your proposal, which is basically a way of three-threaded merging of
apps, I can say that it looks good from a theoretical point of view. Using
multithreading and priorities to get dropout-free realtime performance
seems really be the way to go.
On the other hand, having a server that does *only* this will probably be
too much simplification (as having only IPC streaming is). You seriously
don't want to fully merge apps in the soundserver. Running the complete
Cubase with all plugins, X11 I/O, GUI etc. inside your three-threaded
server might be possible for one Cubase, but at least if you start the
second or another app, you'll run into serious trouble with duplicated
symbols, toolkit event loops, etc.
So you need both: Sophisticated IPC (more than streaming) and in-process
merging for a soundserver. This is also the general thought behind aRts.
You will want to have apps that run fully outside the soundserver. This is
why aRts supports plain streaming. You will also want to have apps with
realtime modules. Then, only the realtime modules of the apps run inside
the server, the rest of the app runs outside.
This will create a heavy communication requirement, as the apps are
practically split somewhere in the middle. Thus aRts supplies the programmer
with a CORBA-like middleware to do communication (called MCOP) with modules
either inprocess, on the local computer or over network, with the same
interfaces. It's kind-of RPC for object oriented systems.
aRts with this also supplies a powerful component model which allows
creating, connecting, querying components, specifying interfaces and data
types,... this is because when you finally have merged your five realtime
app parts, you will also want to connect and use them together. Besides
having smaller components is better, especially for maintainability and
development in an open source style.
Well, but finally let me talk a bit about what I see the right thing to do.
KDE will ship with aRts, so you'll have a realtime capable audio server on
a few million desktops, soon. On the other hand, with Gnome I am convinced
that there will be a way for Gnome users to use aRts as backend. Currently,
you can run
$ artsdsp esd
and have transparent aRts support for all Gnome apps, but Gnome-2.0 might
for instance have a modularization for sound i/o which allows using aRts
natively. And of course command line users can use aRts, too.
So I strongly encourage everybody to have a closer look at if what you want
to do can't be somehow done with aRts. I am open to every suggestion, and
I'd really like to see Linux being a multimedia platform soon. I think the
recent aRts developments will bring us a good step closer. Through
modularization, aRts will be able to be active in all of the following:
* synthesis
* hard disk recording
* disk based sampler (like disksampler)
* effect processing
* sound server
* editing
* midi stuff
* desktop media framework (mp3,wav,foobar player)
* plugin standard / object model
* communication standard
But let me get into some technical details with realtime optimization: the
aRts server itself can run with realtime priority, so it can deliver this
2.1ms latencies. However, the aRts daemon code itself is currently
unthreaded, which is obviously a major obstacle there. While you can do
synthesis with low low latencies, you will run into trouble with something
like disksampler, unless you implement threading for your module yourself.
So if every module writer plays fair (and uses threading), one should be
able to achieve those 2.1ms with aRts, too.
However it would of course be cool to make the daemon more threaded, to
help people with this. For instance, it would be great if the MCOP
communcation layer would run threaded. It would also be better if for
instance building new network structures would run in a seperate thread
than the execution, so if building requires time, it doesn't block
synthesis.
Basically, it is not too hard to do get something like this into the aRts
codebase, and the sooner it is done, the better, but I haven't too much
time to do it right now (all those KDE freeze stuff). So any help is greatly
appreciated. However I hacked together a threaded MCOP (which is
equivalent to having a multi-threaded ORB under CORBA) yesterday as a proof
of concept, so if you like, and have looked at aRts a bit already, you can
have a look at
http://space.twc.de/~stefan/kde/download/arts-mt.tar.gz
On the other hand, if you want to familiarize with aRts/MCOP, you're probably
better off using harmless stuff from
http://space.twc.de/~stefan/kde/arts-snapshot-doc.html
or even more convenient, a recent KDE2.0 beta/CVS version.
Cu... Stefan
--
-* Stefan Westerfeld, stefan_AT_space.twc.de (PGP!), Hamburg/Germany
KDE Developer, project infos at http://space.twc.de/~stefan/kde *-
This archive was generated by hypermail 2b28 : Sun Jul 16 2000 - 00:07:36 EEST