Subject: Re: [linux-audio-dev] introduction & ideas
From: Paul Davis (pbd_AT_Op.Net)
Date: Mon Feb 25 2002 - 19:06:11 EET
>OK. So say I have an application dealing with audio video and MIDI. Most
>of the time the audio will be used for the clock providing 'performance
>time'. Video and MIDI will have to be synced to this time.
in my experience (limited), its really hard to convince the video
people to buy this. they seem to think that they super-dooper SMPTE
timestamps on their video data is the master clock, and that audio has
to sync to that :(
> Scheduling video frames
>and MIDI messages on this time base is not practical however. So a common
>'wall clock' or 'system clock' is needed. This has to be an unadjusted
>clock.
>I don't know if CLOCK_MONOTONIC is supported yet within linux, but this
not via POSIX. its available either via the RTC that exists in almost
all computers on which Linux runs, or via the cycle counter on those
architectures that support such a thing.
>is what should be used I think to schedule MIDI messages and video
>frames. (You could call this UST).
sure. this is what MusE and SoftWerk and a few other Linux MIDI apps
do for MIDI sequencing (where the RTC is used for the UST source). i
don't know about video, but my impression is that video is much more
like audio: its a streaming data flow, and once you have a
"synchronous execution" model like JACK, the idea of "scheduling"
video frames pretty much vanishes into the ether. the application
still has to know when to play what, but the passage of time is marked
purely by it being told to handle N msecs worth of processing. thats
why in an SE system, i don't think you need an explicit clock for any
streaming data.
MIDI is different, because its not streamed - its an asynchronous data
flow that can go from saturating the wire to having no data at all for
extended periods of time and back again. video and audio are not like
this, AFAIK.
> So a mapping between
>'performance/media time' (MSC) and the system time
>(CLOCK_MONOTONIC/UST) is updated regularly. If all APIs supported
>this common clock than synchronisation between the different kinds of
>streams would be easier. For my MIDI API I don't want scheduling on a
>tempo clock in the API. This should be done by the application I
>think. So it will only support ordered scheduling on CLOCK_MONOTONIC
>and flush/drain functionality. If the audio API would give me the
>time in CLOCK_MONOTONIC on which a certain buffer (MSC or some other
>clock) was performed, than MIDI could be synced to that fairly easily
>without direct support in the MIDI API for the specific audio system.
I think this is much harder than you think :) Most MIDI devices cannot
tell you when MIDI data will be delivered to the wire. Even the at the
hardware level, there is a (small) FIFO that buffers between the
device driver and the UART chip. On decent hardware, this tends to be
8-16 bytes long, which represents 3-5 NoteOn/NoteOff messages and
about a few msecs of real time.
My own approach to this has tended to be to deliver data directly to
the device driver (i.e. do not use a "sequencer" layer) at my best
approximation to when it should be delivered to the wire, and assume
that the device driver and the hardware will work hard to do the best
they can.
Notions of time are best handled entirely in an application, as far as
i can tell. I don't want an external system to try to deliver my stuff
"when it is supposed to" unless "when" is "right now". The system's sense
of priorities and so forth might be very different to what I want.
>I also think that MIDI should be kept seperate from audio (ALSA) as much as
>possible. That a lot of soundcards have an onboard MIDI interface is more an
>implementation detail IMHO.
But ALSA is a MIDI API as well an audio API. it contains 2 different
MIDI APIs: the sequencer, which is more like the one you're talking
about, and the rawmidi API, which is what you'd probably base yours on
top of.
>> JACK has an interrupt-cycle-resolution UST source (at least when used
>> the existing ALSA PCM driver). It could easily add the cycle counter,
>> but that doesn't make any sense given the clock skew that I alluded to
>> above.
>
>Can it give a mapping of cycle counter and the JACK media clock?
it could do, but it doesn't. there is no portable way to read the
cycle counter. the RTC is considerably more portable, but you need
root or CAP_RESOURCE to set the frequency.
--p
This archive was generated by hypermail 2b28 : Mon Feb 25 2002 - 18:57:45 EET