Subject: Re: [linux-audio-dev] introduction & ideas
From: Martijn Sipkema (msipkema_AT_sipkema-digital.com)
Date: Sat Mar 09 2002 - 18:37:15 EET
> > > Ok, I was of course thinking of timestamping MIDI events with the
> > > "audio clock". No problem when running a sequencer as a part of
> > > the audio network, but it does get messy to timestamp the MIDI
> > > events.
> >
> > I would prefer timestamping MIDI messages with some common clock.
> > Then the audio system can return the actual transmission time for a
> > buffer in this clock and MIDI and audio can be synced.
>
> Yes, but as Paul has been trying to explain for a while, that's not
> the problem. The problem is that the stoopid MIDI interfaces won't be
> of much help when it's time to actually *use* those timestamps during
> playback.
>
> One solution would be that we get something (a hardware IRQ) to wake
> us up at the right moment, so we can send some MIDI data, then check
> when the next chunk is to be played, and go to sleep until then.
>
> Another solution is doing it the audio way; just get (build...) a
> MIDI interface that pumps a constant rate stream of bits or bytes.
> Then, write data corresponding N "raw" bits at a time, where 31250/N
> makes a sensible "buffer rate" for your MIDI playback thread. The
> thread will block on writes, and provided you know how much total
> buffering there is between you and the physical MIDI output, you will
> know *exactly* (*) when each MIDI byte will be sent.
>
> If we have enough buffering to cover the worst case scheduling
> latency, there will be no timing jitter whatsoever on the MIDI output.
I think accurate MIDI timing eventually comes down on how well the
operating system performes. If Linux had a better performing nanosleep()
, i.e. if it would reprogram the clock chip to generate one shot interrupts
at the exact time the first ready thread will need to be woken up, then
the MIDI timing would be close to what the hardware is able to obtain.
I don't think a new clock would make much sense. You could just run
the kernel with a higher clock resolution I think.
> > > > and again, we can easily timestamp MIDI data on the way in -
> > > > after all, its clocked by an "external" source. the problem is
> > > > delivering it on time on the way out.
> > >
> > > Yeah... It's actually quite amazing that even "high end" MIDI
> > > interfaces are as stupid as they are...! *heh*
> >
> > Even with the stupid interfaces timing should be quite good on a
> > realtime operating system.
>
> Well, yes, *if* you have a way of finding out exactly how many bytes
> are in the output FIFO at any time. If you can't, and your output
> thread isn't "hardsync'ed" to the MIDI clock, you'll have to accept
> that you can only get accurate timing when the FIFO is empty as you
> start writing. As the FIFO fills up, your idea of the "current MIDI
> output latency" will gradually drift away from the actual figure.
>
> Now, you *could* probably abuse a relatively decent MIDI interface by
> padding with some harmless MIDI byte - but this buys us nothing
> unless we know 1) that the MIDI interface will wake sleepers
> up*immediately* when there is room in the output FIFO, and 2) the
> size of the FIFO.
>
> The sad reality is that many cards don't even have an IRQ for 1), and
> that 2) can't be found out by asking the driver.
For multiport interfaces there are internal FIFOs besides the serial port
FIFO. Also, this latency is inherent to MIDI and there is nothing that
can be done about it. Knowing all about the FIFO would perhaps enable
feedback to the application about when the MIDI command was actually
transmitted, but that information isn't of any practicall use. The
application
could have known it was flooding the MIDI interface by looking at the
number of events it is sending.
> > > Wonder if your average MIDI interface would handle the 0xfe abuse
> > > hack I suggested above... (Well, at least I know most
> > > SoundBlaster cards won't handle it!)
> >
> > I don't really see an advantage to this approach. It might work for
> > creating a clock, but it wouldn't work on multiport interfaces
> > where the accurate timing is most needed.
>
> Multiport interfaces *need* special features to manage timing, unless
> they're on something that's many times faster than MIDI. Nothing much
> to do about that.
The best is indeed to have an interface link much faster than MIDI
(firewire).
The above will not work for a multiport interface with a slow link anyway.
Both the Emagic and Steinberg units use a different approach.
> > I don't think timing
> > actually gets any better with this
> > approach,
>
> No, there's no point with it, provided you're using an RTL or RTAI
> MIDI output driver.
>
> If you aren't, you *will* need some form of buffering/queueing to
> keep scheduling jitter from leaking into the MIDI stream.
No interface I know of supports a buffered stream, except maybe the
new midiman interfaces, and they thus would have some delay. The
steinberg unit has a internal clock. The Emagic units use buffered commands
that can be triggered using a short message. They won't give details of
course.
> Not that a good Linux/lowlatency build causes all that much jitter,
> but it's still more than you'd get from any dedicated, professional
> MIDI device, such as a hardware sequencer. *That's* what this is all
> about, as I understand it; achieving dedicated h/w performance on
> workstation based hardware.
>
> We've already proved that this is possible with audio. Now, it turns
> out that MIDI is actually *harder*, at least if we want to achieve
> *true* dedicated h/w accuracy... *heh*
I hope (and think) that Linux will actually be able to provide good enough
scheduling for this sometime in the near future.
> > IMHO a MIDI driver should be a (real time) user space process able
> > to accurately transmit MIDI events on the hardware from
> > CLOCK_MONOTONIC or something similar.
>
> Well, that's fine if
> 1) there's a h/w timer to wake the thread up at the
> right times, and
I disagree. Just use POSIX realtime clocks on a good kernel. You just happen
to need a realtime kernel for MIDI. And then there will still be jitter in a
dense
MIDI stream, since a message takes about 1ms to transmit.
> 2) it's OK that the scheduling jitter is visible on
> the MIDI outputs.
This will not be significant with a good scheduler.
> Without "buffering" MIDI interfaces, a workstation is not going to
> deliver the same timing accuracy as say, a h/w sequencer - not
> without a hard real time kernel like RTL or RTAI.
That's a problem with the kernel then. BeOS could do this. Linux could
probably too without too much modification. The scheduling latency is
good enough with the latency/preemptive kernel patches. It is just the
clock resolution that is a little low.
When buffering, MIDI through performance will suffer.
--martijn
This archive was generated by hypermail 2b28 : Sat Mar 09 2002 - 18:27:47 EET