Subject: [linux-audio-dev] Jussi's Was: info point on linux hdr
From: Benno Senoner (sbenno_AT_gardena.net)
Date: Fri Apr 21 2000 - 21:37:37 EEST
I tested Jussi's preallocation scheme:
( lseeking in 4096 increments + writing 4 bytes at the beginning of each block)
30secs von 80secs when doing plain write()s even when writing 1MB at time.
that means more than twice as fast.
But I still have my doubts that all blocks gets really allocated:
for example I can do
ftruncate(filesize)
lseek(filesize-4,SEEK_SET)
write(value,4);
and get an empty file of filesize len,
but these blocks aren't really allocated since it takes a fraction of
second to do this.
when you do a cat of this files, the disk doesn't almost flash up,
and it returns very fast ( create on write algorithm ?).
Although st_blksize reports 4096, fragmentsize seems to be 1024,
is this related to the preallocation issues.
If the FS always does it's IO in 4096 byte blocks,
why should the lseek method be much faster, since basically both methods
would touch the same amount of data.
If the granules were 1024 bytes, then the above figures would make more sense:
since the first method lseeks in 4096 byte steps, it would allocate only
1 block every 4 blocks (of 1024byte size)
And my thesis seems to be right !
I tried various increments 4096,2048,1024,512 on Jussi's method:
results:
with 4096: 30secs
with 2048: 51secs
with 1024: 82secs
with 512: 83secs
(plain writes of 256KB data blocks take about 80sec)
BINGO !
That means the allocation "granule" seems to be 1024 byte long ,
since as you can see there is no difference between 1024 and 512 byte
increments, which let me assume that in these case ALL blocks are referenced,
where the 4096 case references only 25% of the blocks.
This proves Stephen's claims, that the only method to preallocate ALL blocks
of a file is to write() the data.
Jussi: IMHO you are using too small filesizes in order to get reliable results:
I tried with 300-600MB , and all numbers are consistent with my above findings.
comments ?
PS: at this point I am not really interested in preallocation since it degrades
the write performance in my multitrack testcases.
Benno.
On Fri, 21 Apr 2000, Jussi Laako wrote:
> Paul Barton-Davis wrote:
> >
> > also, in the code you show, the ftruncate is redundant, because it
> > doesn't allocate any blocks, and the point of preallocation (if there
> > is one) is to get the blocks allocated in a certain way.
>
> It depends on where in the block we write the data. If we write data to
> start of the block and without ftruncate() we get file size
>
> filesize = requested_file_size - blocksize + write_data_size
>
> Allocated block count is same...
>
> I got little better preallocation performance with
> ftruncate()/lseek()/write() compared to write() of full blocks.
>
> Test source follows
>
> --- 8< ---
>
> #include <stdio.h>
> #include <fcntl.h>
> #include <string.h>
> #include <unistd.h>
> #include <errno.h>
> #include <time.h>
> #include <sys/types.h>
> #include <sys/stat.h>
> #include <sys/time.h>
>
> #define TEST_FILE_SIZE 33554432l
> #define TEST_BLOCK_SIZE 4096l
>
>
> double ConvTime(const struct timeval *spTime)
> {
> double dTime;
>
> dTime = (double) spTime->tv_sec + (double) spTime->tv_usec / 1000000.0;
> return dTime;
> }
>
>
> int main()
> {
> int iHandle;
> long lFilePos;
> double dStartTime;
> #ifdef TEST_WRITE_ONLY
> long lWriteRes;
> char cpData[TEST_BLOCK_SIZE];
> #endif
> struct timeval sCurrentTime;
>
> iHandle = open("test.file", O_WRONLY|O_CREAT|O_TRUNC, S_IRUSR|S_IWUSR);
> if (iHandle < 0)
> {
> printf("open() failed: %s\n", strerror(errno));
> return 1;
> }
> gettimeofday(&sCurrentTime, NULL);
> dStartTime = ConvTime(&sCurrentTime);
> #ifndef TEST_WRITE_ONLY
> if (ftruncate(iHandle, TEST_FILE_SIZE) < 0)
> {
> printf("ftruncate() failed: %s\n", strerror(errno));
> return 1;
> }
> for (lFilePos = 0l;
> lFilePos < TEST_FILE_SIZE;
> lFilePos += TEST_BLOCK_SIZE)
> {
> if (lseek(iHandle, lFilePos, SEEK_SET) < lFilePos)
> {
> printf("lseek() failed: %s\n", strerror(errno));
> return 1;
> }
> if (write(iHandle, &lFilePos, sizeof(long)) < sizeof(long))
> {
> printf("write() failed: %s\n", strerror(errno));
> return 1;
> }
> }
> #else
> lFilePos = 0l;
> while (lFilePos < TEST_FILE_SIZE)
> {
> lWriteRes = write(iHandle, cpData, TEST_BLOCK_SIZE);
> if (lWriteRes < TEST_BLOCK_SIZE)
> {
> printf("write() failed: %s\n", strerror(errno));
> return 1;
> }
> lFilePos += lWriteRes;
> }
> #endif
> fsync(iHandle);
> close(iHandle);
> gettimeofday(&sCurrentTime, NULL);
> printf("Operation took %.2f seconds\n",
> ConvTime(&sCurrentTime) - dStartTime);
> return 0;
> }
>
> --- 8< ---
>
> - Jussi Laako
>
> --
> PGP key fingerprint: 161D 6FED 6A92 39E2 EB5B 39DD A4DE 63EB C216 1E4B
> Available at: ldap://certserver.pgp.com, http://keys.pgp.com:11371
This archive was generated by hypermail 2b28 : Fri Apr 21 2000 - 22:58:33 EEST