Yearly Archives: 2007

libeatmydata

Posted on by
1

Following my successful linux.conf.au talk “Eat My Data: How Everybody Gets POSIX File I/O Wrong“, I started to feel the need to easily be able to have my data eaten.

Okay, not quite. However, when you’ve written your software properly, so it uses fsync() correctly, opening files with O_SYNC or whatever – tests take longer as you’re having to wait for things to hit the rust.

So….. LD_PRELOAD=libeatmydata.so to the rescue! With a POSIX compliant fsync() (that does nothing) and filtering on open(2), it can take your test run times down dramatically.

The only time you shouldn’t use it for your tests is when you end up crashing the machine to test durability (i.e. when the OS doesn’t have the opportunity to cleanly write out the data to disk).

See the libeatmydata project page: http://www.flamingspork.com/projects/libeatmydata/

and the bazaar repository: http://www.flamingspork.com/src/libeatmydata

(it’s seemed to have saved somewhere between 20 and 30% of the time for innodb/ndb tests in mysql-test-run).

Better disk allocation with MythTV and XFS

Posted on by
Reply

Running MythTV on XFS? Noticed that all your recordings end up rather fragmented? (use xfs_bmap to find out) Well, the culprit is MythTV not being too nice to the file system. Good news is, it’s rather fixable.

From the MythTV source code, edit libs/libmythtv/ThreadedFileWrite.cpp and look for the following:

void ThreadedFileWriter::Sync(void)
{
if (fd >= 0)
{
#ifdef HAVE_FDATASYNC
fdatasync(fd);
#else
fsync(fd);
#endif
}
}
You then want to, after the first squiggly bracket (a { ) put in a “return;” so that it looks like this:

void ThreadedFileWriter::Sync(void)
{
return;
if (fd >= 0)
{
#ifdef HAVE_FDATASYNC
fdatasync(fd);
#else
fsync(fd);
#endif
}
}

Recompile MythTV and go!

(Before anybody says, yes – this patch could be neater and all that… I just haven’t had time yet).