Re: [linux-audio-dev] best option for audiovisual synchrony

From: Dominique Michel <dominique.michel@email-addr-hidden>
Date: Fri Oct 20 2006 - 17:55:49 EEST

Le Fri, 20 Oct 2006 14:39:46 +0200,
Simon Barthelmé <simon.barthelme@email-addr-hidden-paris5.fr> a écrit :

> Hi all,
>
> We're in the process of porting an open source Matlab toolbox to Linux
> called the Psychtoolbox. It's a piece of software that's widely used
> within neuroscience and psychophysics, to display graphics and play
> sounds in experiments. It uses OpenGl for its graphics backend, and
> right now the sound support is just matlab's, which is really poor.
>
> A lot of us researchers are interested in audiovisual experiments, where
> we study how the brain combines auditory and visual information (for
> example in speech perception).
>
> Concretely speaking, that means we need to get as precise a timing and
> synchrony as we can possibly get. A typical experiment will go
> something like this :
> - display a flash and, at the same time, play a beep
> - wait for response
>
> The tricky bit is of course getting a flash that's totally synchronous
> with the beep. Absolute synchrony is not achievable without dedicated
> hardware, but we need to get an approximation that's within the few ms
> range.
>
You will need a realtime kernel, which is basically a vanilla kernel with Ingo
Molnar's patches, and a way to fix the priorities. It can be realtime-lsm
security module (deprecated but simple and well tested) or rlimits (with or
without pam). Such a kernel is called realtime (-rt in short) or multimedia
because multimedia distributions as Demudi ou planet CCRMA use such a
kernel).

It is other implementations of realtime kernel, but this one is free, simple and
good enough to get a sound latency around the ms with a good hardware (even
with a live or audigy). The lowest achievable latency will depend not only of
the hardware but on the software too. Sound synthesis and filtering can use
too much processor power to be able to get the lowest achievable hardware
latency.

It is a few howto on the net on those rt kernels, among them:
http://proaudio.tuxfamily.org/wiki/index.php?title=Realtime_%28RT%29_Kernel
http://demudi.agnula.org/wiki/Low-latencyKernelBuildingHowto

 
> From what I've read it seems that Linux is a pretty good platform for
> that, since it has low latency/real time support.
> What I need to know is how I should go about implementing some kind of
> FlashAndBeep function, that guarantees that the sound starts playing as
> soon as the display buffer is flipped, with the shortest predictable
> delay possible.
>
> Is ALSA the way to go ? JACK maybe ? Anything else ?
>
I think at Jack will be better as alsa, because jack assure you at the
execution of all the clients is synchronous, will use all the realtime
possibilities of the kernel and have the possibility to set the latency.

Install jack-audio-connection-kit and qjackctl and you are done.

Best,

Dominique Michel
Received on Fri Oct 20 20:15:04 2006

This archive was generated by hypermail 2.1.8 : Fri Oct 20 2006 - 20:15:05 EEST