I have a $10 bet (AUD or USD… whichever is worth more at the time) with Jeffrey r.e. a 5.1 GA release in $TIMEFRAME. He’s betting that it will be and I want to be proved wrong :)
(of course… who’s definition of a GA release we use hasn’t been mentioned… which will lead to a good laugh. :)
Here for the MySQL All Company Meeting. Arrived okay… albeit late (got to stop in two cities I didn’t intend to). Had breakfast with Monty, Monty and Giuseppe and a few cups of coffee later… it sorta feels like morning.
So I finally installed the xdebug PHP extension and started doing some performance analysis of MemberDB using xdebug and kcachegrind. The upshot of which is a number of commits to the bzr tree that dramatically improve performance in several key areas. The answer? Caching.
I’m not even talking using memcached or caching things in database tables or anything like that – just about everything is still the same dynamically produced content as before, but I’m now caching some simple things avoiding many round-trips to the database while executing a script.
There were a few things that were taking a fair bit of execution time:
I used the faithful Apache Bench (ab) to do benchmarks against the modified PHP code. I think the biggest improvement was the view election page which went from about 6seconds/page to 0.2seconds/page.
Neither of the books I’ve been an author of has been just me. For Practical MythTV (Christmas is coming, buy it for all your TV and tech loving friends!), Michael Still and I worked hard to get a well rounded and practical (not to mention good) book. I think we succeeded – certainly has gotten positive reviews (check the amazon page).
For my second endeavor (just to make it fun, I was working on both at the same time) we have a much longer list of authors. The aim was to write a study guide for those wishing to be certified in MySQL Cluster. Being a developer with a fair bit of knowledge in the product (and somebody who also presents and writes) – I was a natural fit to join the team (some may say “roped into the team”… and they could possibly be me, but I couldn’t possibly comment).
My fellow authors:
(along with Jay Pipes) and numerous blog posts on MySQL and related topics.So, the book is now shipping, from lulu.com (an on-demand printing service) for $49.99USD.
MySQL 5.1 Cluster DBA Certification Study Guide by Jon Stephens, Mike Kruckenberg, Roland Bouman, Stewart Smith, Solomon Chang
It feels good to have it out there now. Daniel va Eeden has received his copy (shipping box and all!). This is another book you should buy for all your database friends everybody you’ve ever met.
Bug ID: 6493264 Solaris libc should provide posix_memalign
At least it’s acknowledged as a problem :)
Although I now have to go and implement it in the NDB portlib.
I wish we could drop support for everything other than linux 2.6… would make this stuff a lot nicer.
Following my successful linux.conf.au talk “Eat My Data: How Everybody Gets POSIX File I/O Wrong“, I started to feel the need to easily be able to have my data eaten.
Okay, not quite. However, when you’ve written your software properly, so it uses fsync() correctly, opening files with O_SYNC or whatever – tests take longer as you’re having to wait for things to hit the rust.
So….. LD_PRELOAD=libeatmydata.so to the rescue! With a POSIX compliant fsync() (that does nothing) and filtering on open(2), it can take your test run times down dramatically.
The only time you shouldn’t use it for your tests is when you end up crashing the machine to test durability (i.e. when the OS doesn’t have the opportunity to cleanly write out the data to disk).
See the libeatmydata project page: http://www.flamingspork.com/projects/libeatmydata/
and the bazaar repository: http://www.flamingspork.com/src/libeatmydata
(it’s seemed to have saved somewhere between 20 and 30% of the time for innodb/ndb tests in mysql-test-run).
New:
Old (In previous patchset too):
Availability:
Feedback much appreciated.
I’m speaking at the upcoming VITTA conference.
Title:MySQL database administration for non DBAs
Abstract: MySQL is incredibly ubiquitous. MySQL database administrators are not everywhere; MySQL is. Often MySQL is run to power a small web site or two, an application or two, or run on a machine purely for someone else’s use (and the install made MySQL just work so you don’t have to care). This session goes over the things you need to know about your MySQL installations to keep them healthy without burdening you with work, including MySQL Basics, installation, security, backup, restore, performance and upgrades.
When: 12:15 PM, Wednesday 21 November 2007
Should be fun!
So… stealing the idea from Peter, does anybody have any suggestions for MySQL Cluster related things to have at the UC next year (April)? Either leave a comment or email me (first name at mysql dot com).
I’ve decided to publish my patch series. The goal of the -stew patches is to collect things I find interesting and that at some point could (should) make it into the main MySQL tree (even if others don’t think so).
It’s not designed for use in production.. I don’t really care if there’s failing test cases…. if it builds it’s perfect.
It includes the following which could be interesting:
Currently the additional engines have to be built separately in their storage/ENGINE directories. I have some preliminary patches to get them to build in-tree via the plug.in file, but it’s not finished (patches welcome).
This is all currently based off the 5.1-ndb tree. In future, it will likely be based off 5.1-telco.
 http://www.flamingspork.com/mysql/patch-5.1-ndb-stew1-20071016.patch.gz
and broken out in:
http://www.flamingspork.com/mysql/patch-5.1-ndb-stew1-20071016/
(including the not-quite-working ENGINE_in_tree_build patches)
Known to apply against this tree:
http://www.flamingspork.com/mysql/mysql-5.1-ndb-20071016.tar.bz2
Comments, thoughts, patches to include, all welcome!
Today’s xkcd is just wonderful. I think we need giant ones of these on the wall of offices around the world.
I am very good at just staying up late to adjust to a timezone. I can do this fairly reliably. Going to the US and Europe can be done by this method (rather well). Coming back is another story though. Going to sleep at an earlier time (for me) doesn’t come easy. Grr…
Quick experiment with online changing of enabling/disabling compressed backups and local checkpoints (LCPs).
Backup is incredibly trivial and correct (even have some nodes do compressed, some not).
LCPs are a bit trickier when it comes to restore… currently how the code sits is that a block using the compressed file interface in NDBFS must specify if it wants to use the compressed read/write interface or not. So when you have LCPs that differ in compressed/non-compressed than the current config file setting, you’re not going to be able to restore them (although setting CompressedLCP=1 should let you restore either compressed or non-compressed LCPs).
At some point, I’ll probably move AsyncFile (our async file IO class) to just use azio alway, and modify azio to be transparent for non-compressed files…. I just have to fix up azio for direct io.
This year, it seesm that whenever I go out for significant travel, the following things will break on my trip:
At least this time the disk is part of a RAID1 array.
Oh, and for some reason my mythbackend stopped doing anything a few days ago…. and I wasn’t checking it. grr… annoying. At least there’s not much on TV.
possibly:
“Achieving Web 2.0 Social Networking Synergies with NDBAPI through MySQL Proxy”
(yet another possible cool thing coming from a quick hack at DevConf)
So, here is an Alpha quality port of the MySQL Cluster management server to Win32 based on the current MySQL 5.0 tree.
This isn’t going into 5.0, so don’t expect to ever have that.
This isn’t going into 5.1 either, so don’t expect it there.
It’ll go into some future release at some level of general “supported” status that has yet to be decided.
ONLY USE THIS FOR EXPERIMENTAL PURPOSES.
IT IS EARLY RELEASE – IT HARMS PUPPIES!
But, it would be great for those who may be interested in having a ndb_mgmd on Win32 at some point to grab the binary, have a play and find some bugs.
For any bugs filed, please submit to bugs.mysql.com and explicitly mention that it’s version “5.0.50-ndbwin32r1” and mention that it’s the specific build (i.e. it shouldn’t go through the normal bug verification procedure and instead end up with me looking at it directly).
So, here’s the files:
Hopefully this brings you joy.
Oh, and yes, you can go and run it under WINE so you don’t have to actually use MS Windows.
enjoy!
“MySQL Cluster: We don’t need no stinkin FOREIGN KEYS”
and, of course: “Ban HP-UX Now!”
Any design for a “I’m Highly Available” shirt is pure speculation…. but totally awesome (and a cookie for anybody who a) makes them or b) wears them)
Yes, I really, really really dislike the Microsoft Windows platform. I think you have to approach insanity to even remotely consider using it in a HA environment.
That doesn’t mean that we shouldn’t support it. Switching an entire software stack can be a lot of work. Much better to gradually move to complete freedom and sanity.
Well… there’s been some work. Even some in-progress patches. Being involved with this has just perfectly refreshed my memory of why I left the platform. Oh my it’s a horrible, horrible platform. Everything from UI to API… ick.
Expect something around soon….
Go see either my Flickr stream, and I’ll get things onto my Gallery as well.