Subject: Re: [linux-audio-dev] more fundamental questions
From: Jim Peters (jim_AT_uazu.net)
Date: Tue May 22 2001 - 21:41:15 EEST
Now we are talking about moving to a multi-process approach, a whole
bunch of questions has opened up which I hadn't considered before
(including the ones raised by Paul below), and this needs some more
thought.
Basically, my position is to go along with this movement towards
multiple processes, as I don't have any strong reason to object to
this approach if it looks workable after all the analysis that has
been done on LAD over the last few days. This approach does give us
quite a few more problems to crack, though.
I'm assuming that we're working on the basis that each process will
have a real-time thread with SCHED_FIFO priviledges which is
exclusively used for the audio processing. We will also have a LAAGA
library which the application will use. There are two approaches to
the real-time thread in the application:
(1) A LAAGA library function controls the thread, calling the audio
processing code provided by the application whenever necessary,
and handling blocking and other stuff.
(2) The application code is in control of the thread, simply calling a
LAAGA wait() function between blocks.
Option (1) looks a lot more like the current setup, and would in fact
allow a single application to have more than one plugin inserted into
the LAAGA network, which might be useful in some cases. It also means
that our engine is effectively running in threads in all of those
processes.
This is interesting to me, because I think it would be good to have a
symmetric cooperation between processes, rather than having a master
process and slaves, because having a master and slaves causes roughly
twice the number of context switches, always referring back to the
master before a new slave can be executed.
If we are using option (1), then our `engine' could in fact be running
in a distributed and symmetric fashion over several processes, doing a
bit of work in whichever process is currently active before the
necessity comes to kick off another one and block the current one.
This is a bit like how the kernel executes in the memory context of
the current process until it becomes necessary to reschedule.
Using this method, we could have one LAAGA server which just handles
administrative stuff, such as setting up connections and so on,
communicating with the applications via some sort of RPC. All the
actual audio stuff would be handled by our little engine-threads
working away in all the application processes.
In fact, I guess the LAAGA server would have to have one real-time
thread to talk to the audio hardware, so the sequence of execution
each cycle would always start with the server real-time thread, and
also end with it.
How does this sound ? Too weird ? This could scale really really
well, you know.
Paul Davis wrote:
> 1) MP and shared memory
> -----------------------
> If we are going to support a multi-process model with LAAGA,
> there needs to be a way to create buffers and Ports and probably other
> objects in shared memory. This is difficult to do transparently,
> because the shared memory area will likely be mapped to a different
> base address in each process. I know that shmat can try to force the
> address, but there's no guarantee that it will work. That means that
> using straightforward pointers such as
>
> struct Port {
> sample_t *buffer;
> ...
> };
>
> won't work, since the value of the `buffer' pointer will be specific
> to the process that creates the buffer. The same will apply even to
> a pointer to a Port.
>
> Since supporting MP seems like an excellent idea, we need a solution
> to this. I can think of a few, but they seem a little ugly. Any ideas?
Storing an offset to the SHM base address is the only one I can think
of.
> 2) MP and shared memory, part II
> --------------------------------
> Its easy to define a "proxy" for the MP case so that the engine can
> treat both in-process and out-of-process clients in the same way when
> its executing the "process()" callback on each client.
>
> However, every other part of the API for a "client/plugin/whatever"
> would need to be represented by an RPC. For example, when a client
> requests/registers a new Port, the engine will need to make callbacks
> to the client. These callbacks will eventually need to result in some
> IPC to the actual process in which the client lives. This needs to be
> handled by the API we define, and not left up to each client.
In my proposal (way back), I suggested that there were two contexts -
the management thread, and the real-time thread. All management-
related stuff (connections, and so on) would have to be executed in
the management thread. In a multiple-process arrangement, things are
much the same, only extended.
All management stuff can only be performed by the LAAGA server, so
applications will have to make calls or requests to it via some kind
of RPC. The delay involved here is not critical as this is not a
strict real-time activity.
If an application's real-time thread wants to initiate a management
operation, it can either internally ask one of the other application
threads to do it for it, or maybe we can give it some way to queue
this request for the server. My idea of allowing call-backs to be
queued and run in the management thread can't be applied in the same
way with multiple processes, as only the LAAGA server process actually
has the management thread. If an application wants to emulate
something similar internally, I guess that is at its option.
> 3) Typed Ports
> --------------
> Although right now we are 100% focused on ports for handling PCM audio
> data, I want to keep our eyes on the idea that we need to offer easy
> ways to extend the Port types. My proposal for this is fairly simple.
>
> First, we use strings to name the Port types, not integer values. This
> way, people can invent arbitrary port types. Second, all such
> "non-builtin type" Ports can only be connected 1:1.
We can also allow 1:N without problem, which may be useful for certain
types of data - stuff that a lot of plugins might be interested in
seeing.
> Thirdly, output ports of such types provide their own sized buffer.
For data like MIDI (or SKINI, or whatever), this buffer needs to be
allocated and freed in shared memory dynamically, I would guess,
because the size is unpredictable. Perhaps for this kind of thing we
can offer two approaches:
(1) Allow plugins to allocate fixed-sized chunks when they setup their
ports which last until the ports are deleted.
(2) Have a scratch area of shared memory which acts as a heap, and
from which memory can be allocated by plugins on the fly. At the
start of each cycle this is cleared down to nothing. This gives
allocation with near zero time overhead, hopefully suitable for
real-time use.
These two methods should cover everything, although there is still the
possibility that our shared memory region will run out if there is a
big chunk of data to come through. Still it's better to have one big
limit that you can adjust, rather than lots of small hard-coded limits
all through the code (like a 128-byte limit for a MIDI buffer, or
whatever).
> Fourthly, and rather obviously, you cannot connect a Port of type A
> to a Port of type B.
Yes, we must resist all attempts to `extend' this nice and simple
method into something more complex and ghastly. The engine has no
idea what these types are or what they mean, and it never will. It
just passes them on.
> If we do this, then I think that allowing port types like "MIDI",
> "Ambisonic Data", "Positional 3D Audio", "Whatever" will be extremely
> easy to work into the proposal I made yesterday. The buffer's will be
> handled with zero-copy methods, and nobody but the two connected
> Ports (and their owners) will care what is in them.
I think this is perfect. This is completely open-ended and flexible -
just what we need to encourage people's creativity !
> 4) Ensuring correct scheduling
> ------------------------------
> In the MP case, each client must ensure that its audio thread is
> running SCHED_FIFO (or maybe just SCHED_RR, not sure about this
> yet). How do we do this easily? Do we care enough to try to enforce
> it? Presumably, its important that they run at the appropriate
> priority level within SCHED_FIFO as well ...
If our LAAGA library is in charge of the application's real-time
thread, it can check this, or set it up, or whatever.
> 5) Defining the MP communication method
> ---------------------------------------
>
> Abramo used read/write to a pipe; Steve used kill(2).
It would have to be a named pipe, I think, as the applications would
be starting out as separate processes.
> We need to define which method is used; I don't care. If we use a
> pipe, then we need to define the fd number that will be used by the
> client to listen for process() "requests" and notify that they are
> done. Its not quite clear to me how to set this up correctly. For that
> reason, the kill() method seems slightly easier, but it has a real
> cost: signal handlers can only safely use async functions. On the
> other hand, we can use sigwait() to catch the signal and then things
> are OK. This is slightly trickier to set up, but not much.
>
> Comments?
I'm not sure about kill versus pipe-writes, but sending a single byte
down a pipe seems to have less race conditions. With kill, there is a
chance that the kill is received just as we are entering the system
call to sleep on it, in which case we'd miss it. Bytes in a pipe will
always wait for us nicely.
Lots more thought is required on how to manage our interprocess
communication and cooperation between the real-time threads.
Any thoughts on my thoughts ?
Jim
--
Jim Peters (_)/=\~/_(_) Uazú
(_) /=\ ~/_ (_)
jim@ (_) /=\ ~/_ (_) www.
uazu.net (_) ____ /=\ ____ ~/_ ____ (_) uazu.net
This archive was generated by hypermail 2b28 : Tue May 22 2001 - 22:52:17 EEST