Subject: Re: [linux-audio-dev] API design again [MORE code]
From: David Olofson (audiality_AT_swipnet.se)
Date: su loka 10 1999 - 00:42:56 EDT
On Sun, 10 Oct 1999, Paul Barton-Davis wrote:
[...]
> >In the case of audio streaming, the natural sync point is the end of
> >the buffer cycle, and your thread will sleep there, or on the start
> >of the cycle, if the client gets data from the engine as well.
>
> absolutely not! one thread sleeping on another's thread activity is
> death for the kind of response we want. also, since neither select nor
> poll unify file descriptors with either msgsend and/or user-space
> condition variables, it makes programming much harder.
That's effectively what happens when a client communicates with a
server; it needs to sleep at some point, or it will hang the
system... :-) But the select/poll probleb is of course valid. The
system isn't really as flexible after all...
> but thats OK, we can still do this.
>
> So, lets try to clarify this. Since the MIDI input thread can't mess
> with the engine thread's data structures, what has to happen, as far
> as I can see, is that the MIDI input thread has to tell the engine
> thread to listen to its event port. So far, so good, since this is
> basically what you describe above.
Yes. That's what happens when the Communication System sets up the
shadow port.
> Next step: the engine finishes a control cycle, and sets about
> preparing events to deliver to plugins for the next cycle. Presumably
> it iterates over some list of event ports or FIFO or whatever that it
> has been told to listen to. So far, so good.
>
> But wait - how is the engine thread going to safely copy or otherwise
> use this data when the MIDI input thread (or any other h/w-driven
> thread) could modify the port's event list at any time ?
You don't do that. Once an event is "posted", it is considered
property of the destination context. And the event data is going
nowhere until the buffer is flushed. You allocate memory, fill in the
event_t, and add it to the list.
Actually, there's one more level; the transfer of the finished list
of events (for the cycle) to the destination port.
> this, it seems, is where we might be able to use a lock free queue. we
> have exactly one writer and one reader per queue, making them OK for
> the kind of logic that Eli Brandt outlined (which was thankfully not
> a Herlihy-like compare-and-swap system). as Eli pointed out, this
> does rely on pointer writes being atomic and surrounding writes not
> being reordered, which we can probably ensure.
Yep. I'd suggest that the "finish cycle" operation does something
like this:
1) Send the new events off through the lock free queue. From now
on, it is strictly forbidden to modify the events.
2) Flush the event port. (That is, *not* the heap! The buffer
that now holds the new events can safely be used for further
allocation, as that will only result in the life time of the
buffer being extended.)
The heap is still safe to use, but any new events we send to the
shadow port will be added to a new list, and won't affect the sent
events that happen to be in the buffer that still belongs to our
context.
> so, the MIDI input thread can *append* its own event queue (if tail is
> beyond head), but it cannot do anything to it the engine thread can
> *pop* items from the front of the queue if the head is behind the
> tail, but it cannot do anything else to it.
Well, that's the definition of a FIFO, I'd say...
> ---------------------------------------------------
>
> so here's some pseudo code to summarize what i understand so far:
>
> notice that the use of '<' and '>' to test for head/tail
> relations are nominal, not actual C code. if we could use fixed size
> arrays of event_t's, then we could actually use '<' and '>', but this
> seems unlikely to satisfy us.
>
> MIDI input thread:
> =================
>
> while (1) {
>
> ... set up fd_set ...
> select (...);
>
> timestamp = engine->timestamp ();
>
> if (ISSET(&readable_fds, midi_port_fd)) {
>
> read_midi_data_from (midi_port_fd);
> ... parse data ...
>
> foreach event to be generated from data {
> event_t *ev;
>
> ev = qm_alloc_event (my_event_port->heap, ...);
> ev->timestamp = timestamp;
> ... fill out rest of ev ...
> if (my_event_port->events_tail >
> my_event_port->events_head)
> my_event_port->events_tail->next = ev;
> } else {
> /* what ? */
> }
...or you can add all events to the list of the shadow event port, and
then just finish_cycle(my_event_port) to send the list off to the
engine. The engine will never see the shadow port at all - only the
code that gathers new events from the queues is aware of the fact
that the events actually come from a different thread.
What I proposed earlier was just to wake up the MIDI thread an extra
time to perform this operation once per engine cycle, rather than
doing finish_cycle() for every event, or every timestamp. With shared
memory and lock free queues it doesn't make much difference, but if
would in other settings...
[...engine thread...]
> and thats it.
Yep, pretty much what I had in mind.
> out of this come some expanded data structures:
>
> struct event_port_t {
> qm_heap_t *heap; /* how to allocate */
> event_port_t *engine_listen_next; /* link to engine listen list */
> event_port_t *subscriber_head; /* subscriber list */
> event_port_t *subscriber_tail; /* subscriber list */
> event_t *event_head;
> event_t *event_tail;
> }
Hmmm... Maybe at least some of these should be hidden in the private
parts of the implementation. (They will probably look different for
non-shared memory connections and other weird things...)
> Final note: we should come up with a new name for things like the MIDI
> input thread. They are not plugins, because they run independently of
> the engine's control cycle. But they are also candidates for "adding"
> to the engine, not necessarily at run time, but perhaps at startup,
> and certainly during compilation. One can imagine threads that handle
> h/w events from various strange devices (say, the serial port, or some
> A-D data acquisition card) etc.
It fits my idea of a client... The most significant difference from a
plug-in is that it has it's own thread, and runs whenever it wants.
Same data structures but a different communication scheme.
//David
·A·U·D·I·A·L·I·T·Y· P r o f e s s i o n a l L i n u x A u d i o
- - ------------------------------------------------------------- - -
·Rock Solid David Olofson:
·Low Latency www.angelfire.com/or/audiality ·Audio Hacker
·Plug-Ins audiality_AT_swipnet.se ·Linux Advocate
·Open Source ·Singer/Composer
This archive was generated by hypermail 2b28 : pe maalis 10 2000 - 07:27:13 EST