Subject: Re: [linux-audio-dev] MuCoS, Glame, API Issues
From: David Olofson (david_AT_gardena.net)
Date: ke maalis 08 2000 - 17:19:28 EST
On Wed, 08 Mar 2000, Richard Guenther wrote:
> On Wed, 8 Mar 2000, David Olofson wrote:
>
> > On Tue, 07 Mar 2000, Alexander Ehlert wrote:
> > > Hi all,
> > >
> > > my second try to say hello to this list. I'm one of the developers of
> > > glame(http://glame.sourceforge.net). I haven't been following all your
> > > discussions here in detail, but from what I've seen, I think that our
> > > so called "filter/filternetwork API" already addresses a lot of problems
> > > that have been discussed in this mailing list.
> >
> > Yes, this is exactly why we are working on a new API. We'd like to
> > come up with something that virtually *everyone* could use where
> > plugins are needed. This means that we need a very flexible solution
> > with minimum overhead and complexity. (Sounds like a nice and easy
> > job, right? ;-)
>
> Err, so you are proposing the merge of all lowlevel interfaces that exist
> to control the data flow between "plugins" (effects, filters, etc.)??
No, that's about as flawed a design approach as you can possibly use.
The only way that leads to a usable result is analyzing the
*requirements* of all these systems, and try to come up with an API
that can fulfill as many of these requirements as possible with as
few basic constructs as feasible.
> This way you could just drop n-1 projects and concentrate on 1. The so
> called "plugin interface" is the heart of each audio processing tool - at
> least it is for glame now.
Yes, it usually turns out that way if you design it that way.
Consequently, such APIs never fit into other projects, as other
situations than the ones designed for has been taken into account.
Again, I'd like to remind of this file system or IPC resemblance; an
API doesn't *have* to be capable of only one way of working to be
usable or efficient. I think the UN*X way of design is a pretty good
example of this. How many of the basic design concepts have actually
changed since the first real UN*X implementations?
Compare this to the Microsoft APIs, where you have to rewrite large
parts of your applications to take advantage of a minor improvement
of the "OS"... Just about *everything* was done the wrong way from
the beginning, obviously.
(There are other examples, of course, but I believe a lot of people
around here have seen and worked with these two environments to some
extent.)
> > > I try to summarize the most important features and explain the terms we
> > > use:
> > > - a single effect is called filter(ok, I now it's bad...), a filter is
> > > some kind of plugin that basically send/receives data through ports
> > > - ports can be used for any kind of protocol, if you need a new one,
> > > define one :), currently we've got a only a protocol for transfering
> > > sample buffers. All protocols are just extensions of a basic buffer
> > > protocol
> >
> > * Do you have to skip through linked lists or the like to find ports?
>
> No, they're hashed by name. Also theyre not allowed to change (i.e.
> appear/disappear) during the time the plugin is registered.
Ok; that is, the engine can keep direct pointers to ports to use as
long as the net doesn't change?
> > * When is a plugin allowed to add/remove a port?
>
> Only at registration point (and only add). This may seem a limiting
> factor, but note that the input/output ports are only hooks for the actual
> connections and that one port can be connected multiple times - so usually
> filters do have one input and one output port only.
Ok.
> > * Can this be done in a separate thread without stalling the engine
> > thread until the whole operation has been performed?
>
> Its thread safe as at the time you are allowed to add stuff the filter is
> not allowed to be registered (be possible in use). What's this "engine"
> thread?
In low latency real time systems, the only reliable (and simple
enough to actually realize) way it to run many plugins together in
one or a few threads. The most common approach is to have a single
engine thread that calls plugins one by one in a strictly controlled
order, as defined by the net description. This means less thread
switching overhead and no plugin/plugin CPU time interference.
> > > - filters can be combined to filternetworks, every filter runs as a new
> > > thread(we're using pthreads)
> >
> > Although this allows a great deal of flexibility, it's definitely a
> > no-no for low latency audio in anything but RTOS environments like
> > QNX or RTLinux. There is too much overhead involved, there is a risk
> > of your engine being stalled by some kernel operation (even on
> > lowlatency kernels!), since each switch involves the kernel directly,
> > there are problems with scheduling multiple RT threads in low latency
> > settings (no preemptive/timesharing switching among SCHED_FIFO
> > threads), and some other things...
>
> RT threads are bad.
No. They are *required* for certain tasks.
> They're messy in case you get more than one of it.
It's just a different kind of programming. Not always trivial, but
you can't avoid it and still get the job done, unfortunately.
> As for the latency, on a P100 I get a latency caused by the thread/socket
> overhead (on a modestly loaded system) of ~0.06ms using 3 threads, ~0.15ms
> using 10 threads, with 2000 threads the latency (measured by "pinging"
> through a linear loop of connected filters) goes up to ~10ms - this is not
> to bad (its a P100!!). A quick test on a dual PIII-550 shows (it has a
> load of 1 at the moment) ~5ms using 256 threads (the 3 threads case is
> ~0.05ms) - so the latency is not strictliy cpu bound. An IRIX box w 4 cpus
> shows ~0.2ms for 3 threads and ~8ms for 80 threads (cannot go up with the
> # of threads, seems to be limited per user).
We're talking about *peak* latency here. Drop-outs are not acceptable
in a real time audio system, and the worst case scheduling latency of
the kernel can appear as frequently once per kernel call, if you have
too much bad luck. (Which happens all the time in a real system.)
> > > - pointers to buffer heads are sent via unix domain sockets. To
> > > avoid memcpy's we do reference counting on buffers. filters can
> > > make buffers private, if this is done by two filters a copy of the
> > > buffer is made. Otherwise zero copy read-modify-write buffer
> > > modification is possible on private buffers, that means processing
> > > is done in place. After the buffer is processed it can be queued into
> > > the next pipe.
> >
> > This sounds like quite some overhead to me... Also, there shouldn't
> > be any kernel calls (and preferably no function calls at all)
> > involved with the execution of a single plugin.
>
> no function calls???
One function call to run the plugin. No function calls for getting
data or events. No function calls for transmitting data or events.
It's not really a big difference in the normal case, but obviously,
it can only work as long as you don't need to ping-pong between two
or more plugins within the time of a single buffer/call time frame.
> you cannot do this without threads!?
Well, one engine thread is quite handy ;-), but more is just extra
overhead and complexity. (Especially in the timing aspect.)
> I think
> you are vastly overestimating the overhead of syscalls, function calls
> and thread switching time! For the overhead of this see the above
> latency measures.
It's not the average overhead. It's the fact that you may be stalled
by some other part of the system (kernel) holding a spinlock - and
miss your deadline, even though there is unused CPU time.
> > Anyway, sorry for the harch reply - and for being incomprehensible,
> > if that's the case. That's just because we've been discussing all
> > forms of plugin API issues for some months around here, and
> > personally, I'm very interested in hard real time issues, as that is
> ^^^^^^^^^^^^^^
> Linux is not hard real time. Its soft real time.
Standard Linux, yes. With the lowlatency patch (and hopefully 2.4),
this is no longer the case.
> And soft is relative here, too.
Soft vs. hard can never be relative. A missed deadline is a missed
deadline, period. (Ok, allow for hardware failures and the like.
There is no such thing as a true guarantee in the real world.)
> Audio is not hard real time stuff either
It is indeed. Drop-outs during live concerts may cost you your job
and reputation. (That's why most live systems are still analog - not
even dedicated digital systems are reliable enough!)
> - until you want
> ~1 sample latency from audio input to output. There is buffering!
Buffering has nothing to do with it! The definition of a hard real
time system is that it can quarantee a correct response within a
certain maximum time frame. That time frame might well be 2 hours,
but it's still a hard real time system.
> > the main reason I got into this in the first place. (Windows and Mac
> > systems are incapable of useful real time processing without
> > dedicated hardware...)
> >
> > Your project seems very interesting, though. :-)
>
> Thanx. The main concept of glame is to have very simple and generic APIs
> where writing a special foo filter is easy (just 10-20 lines of code
> overhead for a usual effect). Threads do allow this. If you dont like
> threads you can "schedule" manually by using either MIT pthreads (software
> thread library) or schedule via setjmp/longjmp (ugh!).
The callback model isn't more complicated, it's just a slightly
different way of thinking. Usually, the inner loops end up
identical, for performance reasons...
//David
.- M u C o S --------------------------------. .- David Olofson ------.
| A Free/Open Multimedia | | Audio Hacker |
| Plugin and Integration Standard | | Linux Advocate |
`------------> http://www.linuxdj.com/mucos -' | Open Source Advocate |
.- A u d i a l i t y ------------------------. | Singer |
| Rock Solid Low Latency Signal Processing | | Songwriter |
`---> http://www.angelfire.com/or/audiality -' `-> david_AT_linuxdj.com -'
This archive was generated by hypermail 2b28 : su maalis 12 2000 - 09:14:06 EST