Subject: [linux-audio-dev] linus speaks out on mmap
From: Paul Barton-Davis (pbd_AT_Op.Net)
Date: Thu Apr 06 2000 - 04:54:58 EEST
Over on linux-kernel, Linus replied to a message from me in which I
said:
I was very disheartened to find that on my system the mmap/mlock
approach took *3 TIMES* as long as the read solution. It seemed to me
that mmap/mlock should be at least as fast as read. Comments are
invited.
He wrote:
>People love mmap() and other ways to play with the page tables to
>optimize away a copy operation, and sometimes it is worth it.
>
>HOWEVER, playing games with the virtual memory mapping is very expensive
>in itself. It has a number of quite real disadvantages that people tend
>to ignore because memory copying is seen as something very slow, and
>sometimes optimizing that copy away is seen as an obvious improvment.
>
>Downsides to mmap:
> - quite noticeable setup and teardown costs. And I mean _noticeable_.
> It's things like following the page tables to unmap everything
> cleanly. It's the book-keeping for maintaining a list of all the
> mappings. It's The TLB flush needed after unmapping stuff.
> - page faulting is expensive. That's how the mapping gets populated,
> and it's quite slow.
>
>Upsides of mmap:
> - if the data gets re-used over and over again (within a single map
> operation), or if you can avoid a lot of other logic by just mapping
> something in, mmap() is just the greatest thing since sliced bread.
>
> This may be a file that you go over many times (the binary image of
> an executable is the obvious case here - the code jumps all around
> the place), or a setup where it's just so convenient to map the whole
> thing in without regard of the actual usage patterns that mmap() just
> wins. You may have random access patterns, and use mmap() as a way
> of keeping track of what data you actually needed.
>
> - if the data is large, mmap() is a great way to let the system know
> what it can do with the data-set. The kernel can forget pages as
> memory pressure forces the system to page stuff out, and then just
> automatically re-fetch them again.
>
> And the automatic sharing is obviously a case of this..
>
>But your test-suite (just copying the data once) is probably pessimal
>for mmap().
>
> Linus
The test program (for mmap/mlock) looks like this. Don't remeber if I
posted this before.
I am likely to avoid mmap for this kind of access now. It works
fabulously on small data sets (typical soundfiles that will fit into
memory easily), but its clearly not the right approach for an
application doing sequential reading of large/huge datasets.
--p
#include <stdio.h>
#include <string.h>
#include <errno.h>
#include <stdlib.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/mman.h>
#include <sys/stat.h>
#include <asm/page.h>
#include <asm/msr.h>
#define CYCLES_PER_MSEC 450000.0f
#define LOCKDOWN_BYTES PAGE_SIZE * 64
#define NTRACKS 24
#define DIRECTORY "/tape/24.ardour"
#define WITH_MLOCK
typedef struct {
int fd;
size_t file_size;
volatile char *mmap_buffer;
size_t mlock_start;
size_t mlock_end;
} track_t;
int
track_setup (track_t *t, char *path)
{
struct stat statbuf;
if ((t->fd = open (path, O_RDWR)) < 0) {
fprintf (stderr, "can't open file %s\n", path);
return -1;
}
fstat (t->fd, &statbuf);
t->file_size = statbuf.st_size;
t->mmap_buffer = (volatile char *) mmap (0, t->file_size,
PROT_READ|PROT_WRITE, MAP_SHARED,
t->fd, 0);
if (t->mmap_buffer == (char *) MAP_FAILED) {
printf ("cannot map (%s)\n", strerror (errno));
return -1;
}
t->mlock_start = 0;
t->mlock_end = LOCKDOWN_BYTES * 2;
if (mlock (t->mmap_buffer, t->mlock_end)) {
fprintf (stderr, "%d: cannot do initial mlock (%s)\n",
t->fd, strerror (errno));
return -1;
}
return 0;
}
int
track_shift (track_t *t)
{
unsigned long then, now;
if (munlock ((char *) t->mmap_buffer + t->mlock_start,
LOCKDOWN_BYTES)) {
fprintf (stderr, "%d: cannot unlock (%s)\n",
t->fd, strerror (errno));
return -1;
}
if (munmap ((char *) t->mmap_buffer + t->mlock_start,
LOCKDOWN_BYTES)) {
fprintf (stderr, "%d: cannot unmap (%s)\n",
t->fd, strerror (errno));
return -1;
}
rdtscl (then);
if (mlock (t->mmap_buffer + t->mlock_end,
LOCKDOWN_BYTES)) {
fprintf (stderr, "%d: cannot mlock (%s)\n",
t->fd, strerror (errno));
return -1;
}
rdtscl (now);
printf ("%d: %u .. %u took %.3f msecs\n",
t->fd,
t->mlock_end,
t->mlock_end+LOCKDOWN_BYTES-1,
(float) (now - then) / CYCLES_PER_MSEC);
t->mlock_start += LOCKDOWN_BYTES;
t->mlock_end += LOCKDOWN_BYTES;
return 0;
}
int
main (int argc, char *argv[])
{
unsigned long then, now;
track_t tracks[NTRACKS];
int i;
int ok_to_continue;
for (i = 0; i < NTRACKS; i++) {
char pathbuf[32];
sprintf (pathbuf, "%s/%d.0", DIRECTORY, i+1);
if (track_setup (&tracks[i], pathbuf)) {
fprintf (stderr, "cannot set up track %d\n", i);
return 1;
}
}
ok_to_continue = 1;
while (ok_to_continue) {
rdtscl (then);
for (i = 0; i < NTRACKS; i++) {
if (track_shift (&tracks[i])) {
ok_to_continue = 0;
break;
}
}
rdtscl (now);
printf ("***** all tracks took %.3f msecs\n",
(float) (now - then) / CYCLES_PER_MSEC);
}
return 0;
}
This archive was generated by hypermail 2b28 : Thu Apr 06 2000 - 05:45:59 EEST