Subject: [linux-audio-dev] Re: low-latency news: preemptive kernel, preemptive-2.3.52-B7
From: Roger Larsson (roger.larsson_AT_norran.net)
Date: Wed Mar 15 2000 - 17:28:38 EST
Benno Senoner wrote:
>
> forwarding news from Roger Larsson,
>
> Roger that stuff looks interesting,
> can you please enlighten us on the linux-audio-dev list,
> which advantages the new approach has ?
I give it a try (Ingo please correct me if I completely get lost...)
Standard kernel:
* To get low latency from process wake up (done by IRQ routine) to
the actual process running.
A)The waken process has to have higher priority than the currently
running, the currently running will be marked with (need_resched).
B)When the IRQ routine returns; the (need_resched) flag will be checked
it it is true and the current thread is not running in kernel space
it will reschedule.
C)BUT if it is running in kernel space it won't be rescheduled until
it exits back to user space (end of executing system call). This can
take a while... (worst case more than 100 ms)
Low latency kernel:
A) same
B) same
C) at certain _spots_ in the kernel where it is known to stay for a
while(looping throu buffer info...) a test if the process needs
to be rescheduled (need_resched) and a schedule call is done if
needed.
This code 'conditional_schedule()' gives a problem too. You leave
the current spot in the kernel, the higher priority process starts.
What if it makes a system call and ends up in the same kernel
routine and modify the same structures as the original process?
This gives you have to be extremely careful where you place those
conditional_schedule() and the code has to be reentrant.
"fully" preemtive kernel:
Nowadays the kernel has to be reentrant, since the other
processor in a SMP system may enter the same code before the first
will leave - code segments that can not be reentrant are protected
by spinlocks/semaphores ...
[There is more to this - a rescheduled process might be passed in
code by the higher priority one. This is unlikely to happen to
SMP processes rarely runs at different speed, then they would
not be called SMP (S = Symetrical). This situation would more
resemble running a kernel with CPUs of different speed...
Nice is it not? [more... this could happen with the same process
speed too, an interrupt gets served on one of the processors and
the other passes, ouch...]]
Why not use this? It would allow rescheduling of any process
executing in kernel space that is not holding a lock of any kind?
When releasing the last lock it should also check if rescheduling
is wanted to be able to switch as soon as possible. This part can
be skipped - you will soon get another IRQ at least with 100 HZ
but then it would not give you the low latencies you want...
- It would add extra code in lock handling, and since it is executed
very often in critical patch... = not good
A) the same
B) after handling an IRQ do not reschedule if the current process that
(need_resched) is executing in kernel with a lock. Only then it can
not reschedule.
=> can reschedule directly after an IRQ if we were executing in
user space or in kernel space without holding a lock. And locks are
held for as little time as possible.
C) We will reschedule from most places in the kernel but with a big
penalty when it is not needed.
Ingo preemtive kernel:
It is a mix. Instead of making everything but places where locks are
held preemtive. It marks portions that ARE preemtive.
preempt_on();
memclear_highpage_flush(page, partial, PAGE_CACHE_SIZE-partial);
preempt_off();
First at preempt_on() it checks for pending reschedules and anytime
during the enclosed part it may reschedule immediately when needed.
Note:
preempt_on();
preempt_off();
Is semantically equal to:
conditional_schedule();
But with more overhead, a
preempt_point();
can be introduced.
A) the same
B) as with preemtive kernel
C) is a mix of lowlatency kernel and preemtive kernel
You know where there are problematic code. Now you will not need
to make a periodic check inside a loop, like in memclear_...
When a patch with all spots from lowlatency kernel is prepared it
would be a kernel with even lower latencies but with slightly less
performance...
/RogerL
PS
Linus objects to the patches since (I think) the BEST way would
be not to have the loops in the first place, by changing data
structures - but there are certain situations where it is necessary
to loop a lot (with bad cache performance).
Like when copying data.
The patches are in a way ugly - but they will also help in exposing
SMP races that are extremely hard to find.
DS
>
> Anyway I will benchmark this 2.3.52-B7 kernel to see how it compares
> to the old stuff.
>
> Roger, can you post the URL of the 2.2.15 lowlatency patch ?
(se other mail)
>
> Benno.
>
> On Wed, 15 Mar 2000, Roger Larsson wrote:
> > Hi again Benno,
> >
> > Have you seen the newest stuff from Ingo.
> > It is a new approach!
> >
> > Where you mark kernel code as preemtive, like:
> > unsigned int csum_partial_copy( const char *src, char *dst, int len,
> > int sum);
> > preempt_on();
> > ret = csum_partial_copy_generic ( src, dst, len, sum, err_ptr, NULL);
> > preempt_off();
> > return ret;
> > }
> > (The 'preempt_on' does a conditional_reshedule too!)
> >
> > /RogerL
> >
> > PS
> > Ingo has released a lowlatency for 2.2.15 recently.
> > DS
> >
> > Ingo Molnar wrote:
> > >
> > > This is the latest preemptive-kernel patch:
> > >
> > > http://www.redhat.com/~mingo/preemptive-kernel/preemptive-2.3.52-B7
> > >
> > > (also attached) Changes since A7:
> > >
> > > - wrappers: preempt_on() and preempt_off().
> > >
> > > - more places marked preemptable:
> > >
> > > - all the user-copy functions in usercopy.c
> > > - user-checksumming functions in checksum.S
> > > - some highmem places which are safe
> > > - those buffer.c 'buffer clearing' places which can be
> > > rescheduled
> > >
> > > - fixed a bug in random.c: it was doing a copy_user with
> > > TASK_INTERRUPTIBLE - this can cause stuck processes.
> > >
> > > - similar bug in n_tty.c: memcpy_from_user done with
> > > TASK_INTERRUPTIBLE.
> > >
> > > - (added a temporary assert to filter out bugs like those above)
> > >
> > > - (removed checksum-old.c and old references from x86, nothing was
> > > using it anymore)
> > >
> > > - patch against pre3-2.3.52
> > >
> > > The kernel is rock solid here under load (SMP).
> > >
> > > Ingo
> > >
> > > ------------------------------------------------------------------------
> > > Name: preemptive-2.3.52-B7
> > > preemptive-2.3.52-B7 Type: Plain Text (TEXT/PLAIN)
> > > Encoding: BASE64
> >
> > --
> > Home page:
> > http://www.norran.net/nra02596/
-- Home page: http://www.norran.net/nra02596/
This archive was generated by hypermail 2b28 : Thu Mar 16 2000 - 01:28:19 EST