Subject: Re: [linux-audio-dev] an open letter to Linus re: low latency
From: Jay Ts (jay_AT_toltec.metran.cx)
Date: Fri Jun 23 2000 - 09:25:26 EEST
Paul wrote:
>
> Jay wrote:
> >> I wonder if UCB would have any interest in developing an advanced audio
> >> subsystem, in the same manner that MIT developed X Window?
>
> At the driver level, ASIO is emerging as an excellent de-facto
> standard from the Windows and MacOS world, and it has very clear
> parallels in ALSA
OK, thanks. ASIO is something that I'm still really dumb about.
> In user-space, there is a need for a multi-app-accessible interface
> (like esd), as well as networked audio services.
What I have in mind is a system that works very much like X in some
respects, and like NFS in others. Security is an issue, and hopefully
a better model than those used by X and NFS would be implemented. (We
don't want people from the net listening to us on our mic inputs, do we?)
I'm envisioning server daemons that run on each system. Instead of opening
the audio device through the kernel, apps would call a sound daemon library
routine. And there would be control to allow programs to shut off
the daemon's handling of the different devices, to allow one app to
take over that device when maximum efficiency is desired. Then the
daemon would treat all other open requests (or r/w requests on an already
opened device) as errors (resource in use).
The daemon would handle any number of inputs, outputs, synths, MIDI
devices, etc., on whatever sound hardware is available (no design limit).
It would allow multiple apps to access the MIDI device, and play and/or
record sounds in a cooperative manner. (Different apps could access the
MIDI I/O at the same time through different MIDI channels, and if an app
were already recording audio, another app could open the device to record,
and the daemon would send it a duplicate of the sound input, starting at the
time when it started reading from the input.)
How the daemon would handle the different apps using different numbers
of bits and sampling frequencies is something that is way beyond me right now!
For use over the network, an open Internet protocol would be developed.
One thing I really want right now is to be able to play a CD (or make sound
through some other means) on one system, and send it to the audio device on
another. Here I am logged into my server and writing this from the other
room, but if I run a wavplay command, the sound comes out in the room, where
the computer is, and where I can't hear it! Grr. Just as I have my telnet
session running in an X window on my screen, I want to have the sound come
out of this computer too. That would be a start at least. Another thing
that would result is that it would be easy to write an "Internet Phone"
app - connect to your friend's computer through its IP address, authorize
yourself on his system, and start talking. Why ever should it be much
more complicated than that?
The more advanced use would be for a geographically dispersed group of
musicians to be able to jam together, put on an Internet concert for millions
(billions?) of people in real time, and do multitrack recording and overdubs.
And maybe other things we haven't thought of yet. Why not?
> [...]
> it. X does try to hide this, but sometimes at a cost (though to be
> fair, recents tests of DRI-based XFree86 4.0 drivers for various video
> cards showed them to often be remarkably close to the fastest, newest
> Windows drivers, despite supporting the full multi-app, multi-user,
> networked model that X does).
I think that right there pretty much says it. X is an old design, and
was criticized a lot when it was new because it was "overblown and
inefficient". But it's held up remarkably well, with the help of
continued development (which it could always use a little more of,
of course! :).
> This distinction between "close to the h/w" and "i want my API" is
> even more true for audio. There is no way in hell that you would
> sensibly do real-time FX on a networked audio connection.
Not right now, but maybe later! X was developed by people who weren't
so much concerned with the level of performance of the hardware they
were developing on, but what would exist in upcoming years. That was,
if I may say so, rather brilliant of them. My latest issue of Fiber
Optic News has articles on running fiber into residences and 10Gbps ethernet
network speeds. Put that together with some fast CPUs and a network protocol
that allows for low-latency and priority for real-time data transfer for
multimedia (it's been talked about, is important for VoIP, and IIRC is included
in IPv6), and maybe we'll be doing real-time FX over the Internet (not to
mention LAN connections) in the not too far-off future.
One of my goals is to be able to dedicate one computer as a multitrack
recorder, and use another one or two for the synthesizer(s), another to
do pitch to MIDI conversion to MIDIfy my electric guitar, maybe another to
convert pulses from drum pads to MIDI, etc., and maybe a bunch of rack mount
Linux systems to replace the rack mount digital effects boxes in the studio,
and tie it all together with an ethernet LAN for the digital data flow.
(S/PDIF? Digital ADAT interfaces? Ancient, archaic stuff. Who needs them? :)
> But for many things (WM audio messages, for example), you really want a nice
> API that handles networking transparently.
Yeah, and you also want a sound system that allows a synthesizer app
to turn the WM audio messages off! One of the things that bugs me is
when I'm running Windows with a sound app, and I click on something on
the desktop and it mixes a (loud) "Doing!" sound in with my music. That's
really dumb.
- Jay
This archive was generated by hypermail 2b28 : Fri Jun 23 2000 - 09:50:53 EEST