libeatmydata

libeatmydata is a small LD_PRELOAD library designed to (transparently) disable fsync (and friends, like open(O_SYNC)). This has two side-effects: making software that writes data safely to disk a lot quicker and making this software no longer crash safe.

DO NOT use libeatmydata on software where you care about what it stores. It's called libEAT-MY-DATA for a reason.

Source

There's a Bazaar repository up at http://www.flamingspork.com/src/libeatmydata.

Download libeatmydata-5.tar.bz2 which is bzr revno 5 (i.e. "version" 5)

Using

Use libeatmydata on program 'foo' like so:

  1. bzr branch http://www.flamingspork.com/src/libeatmydata
  2. cd libeatmydata
  3. make
  4. make test
  5. LD_PRELOAD=./libeatmydata.so foo

Performance Improvements

Running mysql-test-run --do-test=innodb without/with libeatmydata:

TEST                           RESULT         TIME (ms)   TIME (with libeatmydata)
----------------------------------------------------------------------------------

main.innodb-lock               [ pass ]           4523     4323
main.innodb-replace            [ pass ]            102       56
main.innodb-ucs2               [ pass ]           5786     1084
main.innodb                    [ pass ]          78306    24900
main.innodb_gis                [ pass ]           2647     1544
main.innodb_mysql              [ pass ]          86810    68579
main.innodb_notembedded        [ pass ]            198      150
main.innodb_timeout_rollback   [ pass ]           2990     2750
main.innodb_trx_weight         [ pass ]           1608      841
---------------------------------------------------------------
Stopping All Servers
All 9 tests were successful.
The servers were restarted 7 times

WITHOUT: Spent 182.97 seconds actually executing testcases
WITH   : Spent 104.227 seconds actually executing testcases

WITHOUT:               WITH:
real    3m36.053s      real    2m10.610s
user    0m42.323s      user    0m41.939s
sys     0m2.844s       sys     0m2.356s

This saves us about a third of the total runtime!

Use with Firefox 3 fsync too often issue

The current (May 2008) Linux builds of Firefox in some distributions have a problem where they call fsync() way too frequently, which on commonly used file systems (ext3) while performing other IO (e.g. a compile) leads to very poor performance. libeatmydata has been mentioned as a workaround for this issue as it disables fsync(). NOTE: this will mean if your computer crashes at the wrong point, you may loose some data such as saved passwords, cookies, history, bookmarks etc

You may want to use the tool provided over at https://bugzilla.mozilla.org/show_bug.cgi?id=435712 to recover from any potential loss (it's a backup utility).


FlamingSpork Home

Stewart Smith (stewart@flamingspork.com)