Subject: Re: [linux-audio-dev] MuCoS, Glame, API Issues
From: Richard Guenther (richi_AT_student.uni-tuebingen.de)
Date: ke maalis 08 2000 - 04:58:37 EST
On Wed, 8 Mar 2000, David Olofson wrote:
> On Tue, 07 Mar 2000, Alexander Ehlert wrote:
> > Hi all,
> >
> > my second try to say hello to this list. I'm one of the developers of
> > glame(http://glame.sourceforge.net). I haven't been following all your
> > discussions here in detail, but from what I've seen, I think that our
> > so called "filter/filternetwork API" already addresses a lot of problems
> > that have been discussed in this mailing list.
>
> Yes, this is exactly why we are working on a new API. We'd like to
> come up with something that virtually *everyone* could use where
> plugins are needed. This means that we need a very flexible solution
> with minimum overhead and complexity. (Sounds like a nice and easy
> job, right? ;-)
Err, so you are proposing the merge of all lowlevel interfaces that exist
to control the data flow between "plugins" (effects, filters, etc.)??
This way you could just drop n-1 projects and concentrate on 1. The so
called "plugin interface" is the heart of each audio processing tool - at
least it is for glame now.
> > I try to summarize the most important features and explain the terms we
> > use:
> > - a single effect is called filter(ok, I now it's bad...), a filter is
> > some kind of plugin that basically send/receives data through ports
> > - ports can be used for any kind of protocol, if you need a new one,
> > define one :), currently we've got a only a protocol for transfering
> > sample buffers. All protocols are just extensions of a basic buffer
> > protocol
>
> * Do you have to skip through linked lists or the like to find ports?
No, they're hashed by name. Also theyre not allowed to change (i.e.
appear/disappear) during the time the plugin is registered.
> * When is a plugin allowed to add/remove a port?
Only at registration point (and only add). This may seem a limiting
factor, but note that the input/output ports are only hooks for the actual
connections and that one port can be connected multiple times - so usually
filters do have one input and one output port only.
> * Can this be done in a separate thread without stalling the engine
> thread until the whole operation has been performed?
Its thread safe as at the time you are allowed to add stuff the filter is
not allowed to be registered (be possible in use). What's this "engine"
thread?
> > - filters can be combined to filternetworks, every filter runs as a new
> > thread(we're using pthreads)
>
> Although this allows a great deal of flexibility, it's definitely a
> no-no for low latency audio in anything but RTOS environments like
> QNX or RTLinux. There is too much overhead involved, there is a risk
> of your engine being stalled by some kernel operation (even on
> lowlatency kernels!), since each switch involves the kernel directly,
> there are problems with scheduling multiple RT threads in low latency
> settings (no preemptive/timesharing switching among SCHED_FIFO
> threads), and some other things...
RT threads are bad. They're messy in case you get more than one of it.
As for the latency, on a P100 I get a latency caused by the thread/socket
overhead (on a modestly loaded system) of ~0.06ms using 3 threads, ~0.15ms
using 10 threads, with 2000 threads the latency (measured by "pinging"
through a linear loop of connected filters) goes up to ~10ms - this is not
to bad (its a P100!!). A quick test on a dual PIII-550 shows (it has a
load of 1 at the moment) ~5ms using 256 threads (the 3 threads case is
~0.05ms) - so the latency is not strictliy cpu bound. An IRIX box w 4 cpus
shows ~0.2ms for 3 threads and ~8ms for 80 threads (cannot go up with the
# of threads, seems to be limited per user).
> > - pointers to buffer heads are sent via unix domain sockets. To
> > avoid memcpy's we do reference counting on buffers. filters can
> > make buffers private, if this is done by two filters a copy of the
> > buffer is made. Otherwise zero copy read-modify-write buffer
> > modification is possible on private buffers, that means processing
> > is done in place. After the buffer is processed it can be queued into
> > the next pipe.
>
> This sounds like quite some overhead to me... Also, there shouldn't
> be any kernel calls (and preferably no function calls at all)
> involved with the execution of a single plugin.
no function calls??? you cannot do this without threads!? I think
you are vastly overestimating the overhead of syscalls, function calls
and thread switching time! For the overhead of this see the above
latency measures.
> BTW, how do you keep track of time? (I mean, sample accurate
> synchronization of plugins - plugin controlled buffering usually
> implies that one doesn't have much control over delays between
> plugins.)
As we are dealing with streams of audio samples synchronizing is easy -
just use the sample position (i.e. # of samples ^= time).
> > - automatic dynamic filter loading is not yet implemented but planned
>
> Hmm... Triggered by what? :-)
By the user trying to use a compound filter which in turn uses some yet
unloaded filter plugin.
>
> Anyway, sorry for the harch reply - and for being incomprehensible,
> if that's the case. That's just because we've been discussing all
> forms of plugin API issues for some months around here, and
> personally, I'm very interested in hard real time issues, as that is
^^^^^^^^^^^^^^
Linux is not hard real time. Its soft real time. And soft is relative
here, too. Audio is not hard real time stuff either - until you want
~1 sample latency from audio input to output. There is buffering!
> the main reason I got into this in the first place. (Windows and Mac
> systems are incapable of useful real time processing without
> dedicated hardware...)
>
> Your project seems very interesting, though. :-)
Thanx. The main concept of glame is to have very simple and generic APIs
where writing a special foo filter is easy (just 10-20 lines of code
overhead for a usual effect). Threads do allow this. If you dont like
threads you can "schedule" manually by using either MIT pthreads (software
thread library) or schedule via setjmp/longjmp (ugh!).
Richard.
This archive was generated by hypermail 2b28 : pe maalis 10 2000 - 07:23:28 EST