Subject: [linux-audio-dev] tricks wanted for buffer cache optimization
From: Paul Barton-Davis (pbd_AT_Op.Net)
Date: su tammi 30 2000 - 13:27:42 EST
ok, so ardour is working OK, but if i don't use huge latencies, i can
never read-ahead fast enough.
Why ? we have *26* tracks, and we can expect at least average seek
times. On current SCSI-2 drives thats either between 5 and 7ms which
will mean that each time we need to seek (which will almost certainly
be at the same time for each track), we've got on the order of
100-200ms *minimum* just to get the next chunk(s) of data into the
buffer cache. Recall that I store each track as a separate file to
avoid {de,re}-interleaving the data.
This is not a disaster: this is a tape recorder, and its OK to have
long latencies. When I run with about 180ms of buffering in the h/w,
and readahead about 3secs, things never go wrong. I haven't tried to
reduce this to see how low I can go - I just know that 20ms latency
isn't good enough without enormous amounts of buffering. Sometimes, it
can take upward of several seconds (!) to get the readahead data into
user space. E.g. we're in the middle of a 20 minute tape, we do an
instant rewind back to the start, and the tape butler thread (thanks
to est for this phrase) does read-ahead - we need to pull in several
megabytes of data that are not in the buffer cache, and potentially
flush some existing parts of the buffer cache back to disk. This is
*not* fast!
The Gigasampler approach isn't enough here: we allow random access to
various parts of many (20+) enormous (up to 100minutes of
32bit/96kHz!) audio files, not sequential access to a few (what, 4-10
max?) small samples ... reading in the first 5secs doesn't solve much
for an HDR system.
If necessary, I can start using the old watch cursor, and make people
wait while the tape butler does its work, but it would be really
*nice* to get true, "instant" (i.e. faster than typical human physical
response) tape positioning regardless of the number of tracks or file size.
OK, OK, this is all old stuff because video guys have to deal with it,
and so do some other apps, I would think. Does anyone have any good
pointers to documents on playing with the buffer cache (particularly
the Linux buffer cache) ? Tricks on data layout, periodic (f)sync,
metadata handling, etc, etc ? Basically, I want to get the buffer
cache to do my work for me, partly so that Nemesys can't possibly
complain about patent infringment :)
--thanks
--p
ps. i'm starting to wonder if a non-interleaved h/w is really the
right design. we would do much better with disk access if
the data was interleaved in both directions. OTOH, the infamous
4GB file size would soon kick in with multi-track files, so
there are other reasons why this a problem in the near term.
This archive was generated by hypermail 2b28 : pe maalis 10 2000 - 07:23:27 EST