On 5/9/06, Wolfgang Hoffmann <woho@email-addr-hidden> wrote:
> On Tuesday 09 May 2006 01:08, Lee Revell wrote:
> > On Tue, 2006-05-09 at 00:47 +0200, Florian Paul Schmidt wrote:
> > > On Mon, 8 May 2006 21:54:53 +0200
> > >
> > > Wolfgang Hoffmann <woho@email-addr-hidden> wrote:
> > > > Apropos: on your page on -rt setup (excellent page, btw., many thanks!
> > > > :-), you suggest raising "softirq-timer/0" to prio 99, to make sleep()
> > > > function right (http://tapas.affenbande.org/?page_id=40 sleep()
> > > > based/system timer).
> > > >
> > > > I did so, and got strange latencies (> 40 ms) exactly once every 10
> > > > minutes, caused by some routing-related action (rt_secret_rebuild)
> > > > being run by the softirq-timer/0 thread. Don't you get bit by that,
> > > > too? Kernel is 2.6.16-rt16
> > >
> > > I haven't been bitten by that. Do you also get xruns with [i suppose so,
> > > just asking to make sure]? I haven't had as much time as before to play
> > > around and test things, so maybe it has crept into the kernel recently
> > > or maybe i just always had high-res timers enabled.
>
> To be more precise, I get xruns, not scheduling latencies. Scheduling
> latency
> is low (~50us), but softirq-timer/0 preempts all other threads when boosted
> to prio 99, leading to xruns in jackd.
>
> It's not a scheduler or -rt latency problem, it's a priority problem, since
> softirq-timer/0 runs different jobs that would require different priorities:
> wakeup sleep() at highest prio, rt_secret_rebuild et. al. at low prio.
>
> With CONFIG_HIGH_RES_TIMERS=y it seems thatt softirq-timer/0 is no longer
> needed for waking up sleep()ing threads, so it can be left at it's default
> low prio (or even demoted to SCHED_OTHER?).
>
> > > > My solution is to configure with CONFIG_HIGH_RES_TIMERS=y. Then,
> > > > sleep() wakes up correctly even with softirq-timer/0 being
> low-priority
> > > > (SCHED_FIFO 1 or even SCHED_OTHER).
> > > >
> > > > In general I find adjusting priorities of the various softirq kernel
> > > > threads a bit of secret art. I can't find much documentation about
> > > > "what kernel thread runs which job" that would help making some proper
> > > > decisions here. I found my desktop "feels" most responsive when
> > > > demoting all softirq thread to SCHED_OTHER. I did so after seeing that
> > > > with a non-rt kernel, bottom-half handler don't run SCHED_FIFO/_RR at
> > > > all. So -rt now gives me robust low latencies for jackd, and still
> > > > proper desktop feeling.
> > > >
> > > > Well, maybe this is getting off-topic for this list. But it seems to
> me
> > > > trimming priorities between kernel and userland threads is a bit like
> > > > no man's land.
> > >
> > > I agree. Maybe Lee Revell knows more [CC'ing him]. Lee, you know
> > > something about all these softirq threads? What exactly do they do?
> >
> > Not really. The problem with making the softirq timer thread high
> > priority is that lots of random code gets run from softirq timer
> > context, including rt_secret_rebuild() which is a well known latency
> > killer.
>
> Ah, ok. Now well known to me, too ;-)
>
> > Lately I've been focused on getting mainline usable for audio - I have
> > not run -rt in a while.
> >
> > It is a bit of a black art because you need lots of knowledge of the
> > kernel to know how to set the priorities. But that's the nature of hard
> > realtime - you really need to know a lot about the system to get the
> > priorities right.
> >
> > I've never had to alter the priority of the softirq timer thread myself
> > - does it work OK if you leave the softirq thread alone?
>
> With CONFIG_HIGH_RES_TIMERS=y, yes.
>
> With CONFIG_HIGH_RES_TIMERS=n, no. Then, high-prio theads don't wake up from
> sleep() if a mid-prio CPU hog is running. Typical scenario is a high-prio
> watchdog like Florians rt_watchdog
> (http://tapas.affenbande.org/?page_id=38),
> which fails to work unless softirq-timer/0 runs with higher prio than the
> CPU
> hog. It seems that softirq-timer/0 has to be scheduled for any thread to
> wake
> up. It's a bit like priority inversion.
>
> > If not try setting the softirq thread to a lower RT priority than your
> > soundcard, JACK, and all other interrupt threads.
>
> Yes, that should cure the xruns. But I wonder if jackd's watchdog can do
> it's
> job when a jack client is running wild? Probably not. softirq-timer/0 would
> not get to run then, so the watchdog wouldn't wake up.
>
> > I've added Ingo to the cc: list, maybe he has some input.
>
> That would be helpful. Also adding Steve to cc: as he mentioned writing a
> section in an O'Reilly book about how to use the -rt patch in
> http://www.ussg.iu.edu/hypermail/linux/kernel/0604.2/0340.html
>
> Wolfgang
>
Has anyone got a working x86_64 RT system? I can't boot rt kernels
here, hangs after loading the schedulers.
Loki
Received on Tue May 9 20:15:03 2006
This archive was generated by hypermail 2.1.8 : Tue May 09 2006 - 20:15:03 EEST