Subject: Re: [linux-audio-dev] huge latency peaks with 2.4.10-smp kernel
From: D. Stimits (stimits_AT_idcomm.com)
Date: Sat Oct 20 2001 - 04:00:02 EEST
It seems like a major reason for not getting latency bug reports is
because it is so hard to measure or quantify latency for the average
user. They know something is wrong when they get an OOPS and locked up
machine, but knowing their latency is now 25 ms instead of 15 ms isn't
always obvious. Even if it is obvious, tracking down why is still much
more difficult than an OOPS pointing to a particular line in
fs/block_dev.c (and a corrupted partition).
Some API's (OpenGL comes to mind) often require not just certain things
be present to qualify as that API, but also require some regression
tests be passed. This is beyond me to write what I am about to suggest
(I'm not a kernel hacker, I'm partial to C++), but could probably be
done, in a way similar to regression tests: Create a module that is
capable of characterizing latency, and breaking it down in a way that is
useful for kernel maintainers. Have it generate a log message each
bootup. Maybe generate a message about kernels that are roughly "low
latency", "normal latency", or "high latency" category. Place the
results in /proc/ each time the module is called, offer a tool to
trigger this.
I suggest this oddball idea because without something like it only fatal
or obvious bugs will get reported...this would offer a chance for anyone
configuring the module to run a kind of regression and report bad
changes.
D. Stimits, stimits_AT_idcomm.com
Roger Larsson wrote:
>
> Kai and others,
>
> Suppose a patch gets into the kernel that lowers throughput by 10%
> How many bug reports will there be on linux-kernel within 24 hours?
>
> Suppose another patch doubles maximum latency.
> How many bug reports will then be on linux-kernel?
>
> (There will be reports on both but many for the throughput case and
> only a few for latency, people are nowdays running mp3 players
> while benchmarking throughput - intentionally :-)
>
> On Friday 19 October 2001 19:56, you wrote:
> > >I don't like the way it's done. Mainstream goes digging latency holes all
> > >the time and small group of people is running after and trying to fill
> > > those holes. I'd like to see more co-operation and less one-eyed way of
> > > doing things.
>
> If latency is this important for you Kai then why didn't you send a BUG
> report to linux-kernel?
>
> YES, a bug report !
>
> >
> > agreed. but this is happening less and less. Ingo, Andrew and now
> > Richard's work is making many more people think about scheduling
> > latency issues. this would never have happened without their
> > demonstrating the amazing effect of their patches.
>
> 32 processor computers can not afford long locked regions with several
> processors spinning on that lock, that is why the preemptive kernel work
> is so nice. Both lowlatency folks and multiprocessor folks aim for the same
> target :-)
>
> >
> > >That's why he should be thinking about every feature "how this affects
> > >throughput and latency". If it increases throughput by 10% but causes
> > >latencies to raise by order of magnitude that "improvement" should be
> > >dropped.
> >
> > there are many, many users of the linux kernel who would not agree with you
> > at all about this. probably even the majority.
> >
>
> Linus is one - but someone has to indicate that there is a problem...
>
> > [- - -]
> >
> > >Kernel could also disable badly behaving driver. Once too long in driver
> > > and *bang* driver is unloaded and device disabled.
> >
> > Ouch. You mean instrument every place where we block and unblock
> > interrupts? Linus and 99.9% of the rest of the world are going to
> > really love that!
>
> There is some work by Ingo Molnar that measures how often an interrupt
> happens when already servicing interrupts - it then disables those
> who behave bad.
>
> There is a new (2.4.12) proc parameter in /proc/irc/max_rate that I think is
> slightly related - if a driver is in interrupt service to much. The interrupt
> will be disabled and it will be polled instead...
> (this is really for GigaBit ethernet drivers, they can generate so many
> interrupts that no user space work gets done...)
>
> /RogerL
>
> --
> Roger Larsson
> Skellefteċ
> Sweden
This archive was generated by hypermail 2b28 : Sat Oct 20 2001 - 03:57:20 EEST