Author Archives: Stewart Smith

Running for President (of Linux Australia)

Posted on by
Reply

The other day, I accepted the nomination for President of Linux Australia in the upcoming election. The following, is the spiel I’ve put up to help make my case for electing me:

Hi, I’m Stewart Smith and I want to be the president of Linux Australia in 2008.

I have previously served on the Linux Australia committee both as Vice President and as an Ordinary Committee Member.

This year, I’m running on the following platform:
– Keep the organisation solvent (if we loose money, we go broke and don’t have an organisation any more… which likely means no LCA, which would be a disaster).
– Ensure an awesome LCA is organised for 2009
– Improve processes, especially in finances and book keeping
– Use LA’s resources for more things directly benefiting members (including investigating member services, talking to government to ensure Australian law is friendly to the needs of our members, supporting LUGs)
– Be inclusive and in touch with members, and what’s holding back other people becoming members.

In addition to being on the LA committee previously, I’ve also been on the LUV committee, am currently helping with LCA08 things and was rather active back in the (not too distant) past at my university. I’ve co-authored two books related to free software (Practical MythTV and the MySQL 5.1 Cluster Certification Study Guide), currently work for MySQL as a Senior Software Engineer on MySQL Cluster and live in Melbourne.

I’m also the author of this membership and voting software, MemberDB – and no, it’s not rigged but yes, it is free software.

A few weeks with new stereo system

Posted on by
Reply

Just before Christmas, I bought myself a new stereo. This is something I’ve been thinking about for a while and one of those things that takes some care in looking at because it’s a non-trivial amount of cash.

I’ve ended up with B&W 683 speakers, an Arcam amplifier and CD player (Tivoli HiFi had a special on saving about $800+ on the amp and CD when bought with speakers).

The basic summary is that this sounds awesome. Tool especially sounds awesome. The only real downside is that i’m starting to get really used to this as the standard of audio I listen to.

Even better, no complaints from neighbours (yet).

MemberDB speed improvements

Posted on by
Reply

So I finally installed the xdebug PHP extension and started doing some performance analysis of MemberDB using xdebug and kcachegrind. The upshot of which is a number of commits to the bzr tree that dramatically improve performance in several key areas. The answer? Caching.

I’m not even talking using memcached or caching things in database tables or anything like that – just about everything is still the same dynamically produced content as before, but I’m now caching some simple things avoiding many round-trips to the database while executing a script.

There were a few things that were taking a fair bit of execution time:

  1. The generation of the menu. In MemberDB, there’s a menu on the left. There’s also a powerful (read: non-trivial) permissions system allowing relatively fine grained granting of permissions. So, we need to check that the user has permission to go to the page before showing the page in the menu.
    Previously, for each item in the menu, we’d do a lookup to the database – checking if they have the permission or they are an admin. This ended up taking a bit of time – up to 30% of the time for the front page was taken up just generating the menu!
    So, now I cache the set of permissions for the user. One function to fetch it from the DB into a structure, another function to check the permissions of the user in that struct.
    While testing this, I actually used memcached to cache the menu to see how much of an improvement I could get… I’m about 69/70ths of the speed of using memcached with a purely PHP implementation caching the permissions info.
  2. Getting the information about a member is done in a variety of places. On some pages, you want information on the current logged in user (or just need to find their member ID). These are now cached for the duration of the script. Saved quite a few DB round trips
  3. When viewing an election (not the results, just the normal “view election” page that lists candidates), we need to get the membership information on a number of users (okay… so technically I should rewrite some of the queries to use joins in the DB… but this was easier). I now have a (limited) cache of membership info. So now, when a member has nominated multiple people, we only pull the member info out of the database once.
  4. Rewrite the “current_members” view. The old one was not as efficient as it could be. While the new one has slightly different semantics (can have duplicate rows, it turns out the use of DISTINCT was adding a bit of execution time, which for a bunch of queries is not needed) it’s significantly quicker.

I used the faithful Apache Bench (ab) to do benchmarks against the modified PHP code. I think the biggest improvement was the view election page which went from about 6seconds/page to 0.2seconds/page.

Enabling bluetooth on a hp laptop (6710b IIRC)

Posted on by
8

A friend of mine has a HP 6710b (if i remember the model name correctly). Bluetooth wasn’t working. Nada… not even a USB device showing up in lsusb for the inbuilt bluetooth.

It turns out, if you disable the bluetooth radio from the Windows thingy, it does something you cannot undo from Linux (at least with Ubuntu Gutsy) .

Except… after hours of fucking around, we reset the BIOS to default settings and this made it work.

Yep, no option in the BIOS setup to do this (there’s an Enable/Disable switch… but that doesn’t fix this problem). But a reset does it.

I really hope this saves somebody else the hours of “fun” we had.

Baking Bread

Posted on by
3

Sometime in what we laughingly call the past (okay… the last week) I used a (slight) modification to this Basic white bread recipe and baked bread. Today, I’m doing it again. Why on earth I’m baking in hot weather is another question…. but anyway. The only real change to the recipe was using vegetable oil (just normal cheap cooking oil) instead of butter (i.e. to make it Vegan).

Last time I made the bread it was quite good – the crust was totally awesome. Hopefully will be this time too. Maybe I’ll even take pictures :)

Vegan Choc Chip Cookies

Posted on by
2

I was inspired by this recipe: Best Vegan Choc Chip Cookies! and slightly modified it. Namely, I didn’t seem to have any maple syrup, so I used Golden syrup and i wasn’t sure it was chocolately enough, so added 1/3rd cup cocoa powder.

I quite like the outcome, and so at least does Michael – so it so far gets one friend approval.

The complete recipe (what I did, not exactly what’s at the link):

  • 1 1/4 cups plain flour
  • 1 1/4 cup oat flour (to make, grind approximately 2 cups rolled oats in food processor until a fine, flour consistency)
  • 1 1/8 cups rolled oats
  • 1 teaspoon salt
  • 1 1/8 teaspoons baking soda
  • 1 1/2 cups semi-sweet chocolate chips (I used some I found that were about 63% cocoa and supplemented with some crush soy chocolate – if making this vegan, check the choc chips – even dark chocolate chips may contain milk)
  • 1 cup golden syrup
  • 1 cup canola oil (i just used normal vegetable cooking oil… likely canola..)
  • 1 tablespoon vanilla extract
  • 2 tablespoons water
  • 1/3 cup cocoa powder

and then:

  1. Mix everything together in a bowl. It will be a moist dough, and yummy
  2. Put it on biscuit trays
  3. 180 degrees C for 18-20 minutes (or a bit longer if your oven is crap like mine)

My 2nd book is available! (MySQL 5.1 Cluster DBA Certification Study Guide)

Posted on by
3

Neither of the books I’ve been an author of has been just me. For Practical MythTV (Christmas is coming, buy it for all your TV and tech loving friends!), Michael Still and I worked hard to get a well rounded and practical (not to mention good) book. I think we succeeded – certainly has gotten positive reviews (check the amazon page).

For my second endeavor (just to make it fun, I was working on both at the same time) we have a much longer list of authors. The aim was to write a study guide for those wishing to be certified in MySQL Cluster. Being a developer with a fair bit of knowledge in the product (and somebody who also presents and writes) – I was a natural fit to join the team (some may say “roped into the team”… and they could possibly be me, but I couldn’t possibly comment).

My fellow authors:

  • Jon Stephens
    Among other things, he’s brought the MySQL Cluster section of the MySQL Manual forward leaps and bounds. He also edited the study guide – no doubt a daunting task.
  • Mike Krukenberg
    Author of Pro MySQL (along with Jay Pipes) and numerous blog posts on MySQL and related topics.
  • Roland Bouman
    Who took on the brave task of the actual Certification. This is a certification to be proud of – really makes sure that people deserve it without being overly hard or tricky.
  • Solomon Chang
    founding member of LAMPSIG of Los Angeles and a professional DBA.

So, the book is now shipping, from lulu.com (an on-demand printing service) for $49.99USD.
MySQL 5.1 Cluster DBA Certification Study Guide by Jon Stephens, Mike Kruckenberg, Roland Bouman, Stewart Smith, Solomon Chang

It feels good to have it out there now. Daniel va Eeden has received his copy (shipping box and all!). This is another book you should buy for all your database friends  everybody you’ve ever met.

Coming to the end of OSDC

Posted on by
Reply

Sitting in the last lightning talk session of OSDC2007 and in a couple of hours will be (surprise) jumping on a plane back home.

Has been a good and interesting conference. Met cool people, done cool hacking and chatting around the place.

Gave my talk today, and seemed to go down well…. at least Paul Fenwick liked it :)

libeatmydata

Posted on by
1

Following my successful linux.conf.au talk “Eat My Data: How Everybody Gets POSIX File I/O Wrong“, I started to feel the need to easily be able to have my data eaten.

Okay, not quite. However, when you’ve written your software properly, so it uses fsync() correctly, opening files with O_SYNC or whatever – tests take longer as you’re having to wait for things to hit the rust.

So….. LD_PRELOAD=libeatmydata.so to the rescue! With a POSIX compliant fsync() (that does nothing) and filtering on open(2), it can take your test run times down dramatically.

The only time you shouldn’t use it for your tests is when you end up crashing the machine to test durability (i.e. when the OS doesn’t have the opportunity to cleanly write out the data to disk).

See the libeatmydata project page: http://www.flamingspork.com/projects/libeatmydata/

and the bazaar repository: http://www.flamingspork.com/src/libeatmydata

(it’s seemed to have saved somewhere between 20 and 30% of the time for innodb/ndb tests in mysql-test-run).

Better disk allocation with MythTV and XFS

Posted on by
Reply

Running MythTV on XFS? Noticed that all your recordings end up rather fragmented? (use xfs_bmap to find out) Well, the culprit is MythTV not being too nice to the file system. Good news is, it’s rather fixable.

From the MythTV source code, edit libs/libmythtv/ThreadedFileWrite.cpp and look for the following:

void ThreadedFileWriter::Sync(void)
{
if (fd >= 0)
{
#ifdef HAVE_FDATASYNC
fdatasync(fd);
#else
fsync(fd);
#endif
}
}
You then want to, after the first squiggly bracket (a { ) put in a “return;” so that it looks like this:

void ThreadedFileWriter::Sync(void)
{
return;
if (fd >= 0)
{
#ifdef HAVE_FDATASYNC
fdatasync(fd);
#else
fsync(fd);
#endif
}
}

Recompile MythTV and go!

(Before anybody says, yes – this patch could be neater and all that… I just haven’t had time yet).

mysql-5.1.22-stew2

Posted on by
1

New:

  • Updated NDB Compressed LCP and BACKUP patches (now with O_DIRECT support)
  • InnoDB patch for Windows that should give ~5x improvement on commits/sec (Bug31876)
  • Everything in current telco-6.3 tree (ndb ~6.3.5)
    • Lots of NDB improvements and new features over regular 5.1.
        • WL3686 Remove read before update
        • WL2680 NDB Batched Update
        • WL2679 NDB Batched Delete
        • WL4108 NDB Handler statistics
        • WL4096 NDB Realtime performance and settings
        • WL3126 and WL3127 Client and Replication bind address
        • NDB Online ALTER TABLE ADD COLUMN
        • NDB Multi-Master replication conflict resolution (limitations apply :)
        • NDB prepare for endian independence
        • NDB micro-gcp (reduces replication lag)
        • NDB SendBuffer throttling
        • NDB MySQL Server TC selection (improve performance)

    Old (In previous patchset too):

    • Remove ndb_use_exact_count giving up to 300% performance improvements on Joins in NDB
    • INFORMATION_SCHEMA table for NDB node status
    • NDB Cluster Log as CSV file (suitable for ENGINE=CSV)
    • Skeleton Engine (build from storage/skeleton)
    • MyHTTP Engine (build from storage/myhttp)
    • PBXT Engine (build from storage/pbxt)
    • Make ARCHIVE faster at compressing (at slight expense of space usage)

    Availability:

    • Patch (apply with -p1 to mysql 5.1.22) 4.0 MB
      • Applies cleanly on a BK source tree… a few files don’t exist in the tarball on dev.mysql.com (due to the way it’s built)… so when asked for “file to patch” just hit enter and then choose y to skip that patch.
    • README (list of patches, descriptions) 13kb
    • quilt patch series tarball (individual patches) 4.1MB
    • diffstat 228k

    Feedback much appreciated.

    Speaking at VITTA (Victorian IT Teachers Association Inc) Conference

    Posted on by
    1

    I’m speaking at the upcoming VITTA conference.

    Title:MySQL database administration for non DBAs

    Abstract: MySQL is incredibly ubiquitous. MySQL database administrators are not everywhere; MySQL is. Often MySQL is run to power a small web site or two, an application or two, or run on a machine purely for someone else’s use (and the install made MySQL just work so you don’t have to care). This session goes over the things you need to know about your MySQL installations to keep them healthy without burdening you with work, including MySQL Basics, installation, security, backup, restore, performance and upgrades.

    When: 12:15 PM, Wednesday 21 November 2007

    Should be fun!