Subject: [linux-audio-dev] "simple" way to handle multi-GB files :-)
From: Benno Senoner (sbenno_AT_gardena.net)
Date: ke helmi 09 2000 - 06:41:52 EST
Hi,
the multi-GB files discussion seems interesting
:-)
here my thoughts:
on most OSes actually we have two kinds of linmitations:
- virtual memory limits ( 1-4GB on linux )
- file system limits ( 2 GB on ext2)
But with the right trick this is not a problem:
the key is still to use mmap() , but avoiding to mmap()
very large areas into mem.
Let's say we want to handle 100 audio "files" of 12GB each.
In this case we need some "middleware" which does the disk I/O
for us.
My idea is to:
- determine the max filesize usable by the filesystem (2GB in the ext2 case)
- handle a large audio "file" in chunks of 2GB.
- provide functions to open , read , write to these virtual files, which
use mmap() / munmap() , mapping and unmapping only relatively small blocks in
memory ( I'd say 0.5-2MB at time) in order to avoid running of out virtual
mem.
It would be really cool to develop a library which does the virtual file
handling,since when you switch architecture you can tune the libs to suit your
needs best.
For example if we were running XFS there would be no need to split the virtual
file in multiple real files.
Or if we run on a box which has 100GB of RAM , we could mmap() larger chunks
into mem in order to reduce some overhead.
I think that such a lib would be another nice goal for MuCos,
since if such a lib existed, programmers would use it.
comments ?
Benno.
This archive was generated by hypermail 2b28 : pe maalis 10 2000 - 07:23:27 EST