Subject: Re: [linux-audio-dev] lowish-latency patch and toolchain
From: Khimenko Victor (khim_AT_sch57.msk.ru)
Date: Sun Jul 09 2000 - 00:25:30 EEST
In <20000708203335Z4895-532+18680_AT_nic.funet.fi> Juhana Sadeharju (kouhia_AT_nic.funet.fi) wrote:
>>From: "Khimenko Victor" <khim_AT_sch57.msk.ru>
>>>>doing silly things, the scheduling latency is reliably 4 milliseconds on
>>>>a 500MHz machine. Very occasionally reaches 7 millisecs. It has been
>>
>>> Lets talk about 7 ms latency only -- 4 ms is useless information in audio
>>> software which has to be 101% reliable.
>>
>>Sorry. It was said 1000 times already but I can repeat once more just for you:
>>101% reliability => HARD real time => RTLinux. Period. If you NEED 101%
> Forget that 101%.
> The most above quote tells me that scheduling latency is practically
> reliable with 10 ms latency (while taking account those not-to-do's).
> If only 7 ms peaks are observed, it looks 101% reliable to me.
It's still not 101% reliable and you cut very important part of message:
-- cut --
... Very occasionally reaches 7 millisecs. It has been
tested during kernel builds, netperf, lmbench, mmap001, mmap002,
x11perf, general ext2 and NFS hammering, general usage.
-- cut --
There were not stated how often 7ms was observed.
> It would be very silly to talk that Linux would have 1 ms latency
> if it reaches it only once a week.
Hmm. Why not ?
> Get the point?
No. We CAN NOT talk about ANY latency without probability factor (if we can
it's RTOS). And then if 1 ms latency is reached for 0.001% of time it's still
1 ms latency reached 0.001% of time. What IS important (and what was NOT
stated in Andrew's letter, unfortunatelly) is simple question "how often
latency is worse then 4 ms ?".
> Lets not talk about 4 ms latency but 7 ms from practical point.
Why ?
> Development point is that 4 ms is reached at most of the time and somehow
> we should get those 7 ms occasional latencies away.
Heh. If it was THAT simple :-) The more pre-emption points you adding the
more slim is chance to get approval from Linus...
> Remember, this was the situation before Ingo's patch, and I'm sure we reach
> the reliable 4 ms latency.
In mainstream kernel ? Who knows. I'm not sure if this six kludges are
acceptable from Linus's viewpoint, let alone additional ones needed for
more reliable 4 ms latency.
>>> That is poor compared to earlier results 2.5 ms.
>>
>>It can be expected. There NO WAY IN HELL to include Ingo's original patch
>>in kernel, so stop talking about it already. It REALLY starting to look like
> I was not suggesting that. We should look at a possible good way to solve
> this problem.
So far I've not seen any "good way". There are proposed few long-term solutions
but without working code so we can not be sure if they will actually work.
And all short-term solutions are kludges of some sort :-/
> I'm not happy with 4 ms audio-I/O latency, but it could be
> reached. I'm willing to dig in to this mess just in case more people means
> more suggestions, at least in this case.
May be. But again may be not.
>>-- cut --
>>My guess is you'll be digging yourself in deeper and deeper,
>>sprinkling random hacks in random places, as Linus put it.
>>-- cut --
> My own audio code design is a far from a hack.
It's every programmer's belief, right ?
> Everything is very well organized, well thought.
Or so you think :-)
> I would like to see if I'm able to say my opinion in kernel's low-latency
> matters from non-hacker's point of view.
>>Huh ? What's "the most important points of how kernel works" ???
> How about telling what are those 6 points where the patch additionally
> schedules? A good start. You don't need to give me a full lecture.
Have you read patch ? It's short and it's clearly stated there:
sys_unlink, sys_sync, sys_read, sys_msync (2 points) and sys_write
>>Basically Ingo added extra scheduling everywhere where it looked right
>>on first glance without deep investigation - with just few benchmarks
>>(AFAIK anyway).
> Yep. I would like to see more scientific analysis, and would like to help
> on that.
There were some scientific anylysis. Just EVERY place in kernel where work
was long was fixed with addition of pre-emption points without deep thought
if it can be fixed in some other sane way. This patch is still basicelly the
same as I can see: few functions where kernel will work for long time was
spotted with kernel patch and pre-emption points was injected there. Can not
see deep difference :-(( Just this time kludge it less intrusive and thus
looks more acceptable but that's all.
>>Since there are MUCH less pre-emption points.
> Were they choosed the same way than Ingo choosed his points?
More or less :-/
> Empirically based on a program analyser whatever...?
As it was said by Andrew. There were included explanation of "sources of
latency spotting process".
> If yes, we at least have a way to improve if this patch is ignored
> as well.
It this patch will be rejected it will be rejected by the same reason as
original one: it's a kludge. Smaller (MUCH smaller) kludge but still a kludge.
> Yes, I think Ingo's patch is kludge, and wonder if this new one is
> another kludge.
Yes, it is. It's in the same league. Just much smaller and much less intrusive.
>>I'm not sure if it's possible for Linux kernel at all (it's really easy
>>do draw principal diagram of dataflow in linux but what it'll buy you?
> Hey, I just wanted a short cut for understanding the issues related to
> low-latency audio, not the full lecture on kernel design.
Huh. I'm pretty sure this issues are well-known for you already. Linux kernel
in NON-PREEMPTIVE kernel. Thus if some operation is long in ANY part of kernel
you chance to get low-latency audio is void. Thus you can not investigate just
some "important parts" of kernel. You MUST track ALL kernel functions and make
sure kernel threads will can schedule() often enough. Basically kernel SHOULD
NOT do a lot of work in one batch and so shedule will be called when atomic
function will be ended. Sometimes it's inavoidable. If there are few such
places we can add pre-emption points to them. If there are A LOT OF such points
then the only way is to redesign kernel (make it preemptive or may be just
redesign parts of kernel where it was designed wrong in first place and thus
casing big delays). Have you ever written ANY program for Windows 3.x ? It's
basically the same problem. May be non-preeemptive linux kernel is bad thing
for current world and may be it can be even fixed. Just not this close to
2.4 release: it's HUGE change.
This archive was generated by hypermail 2b28 : Sun Jul 09 2000 - 00:59:35 EEST