a funky thing done last week…

still have to talk to people about standards for this sort of thing and all that. But as a first checkin – funkyness++!

mysql> select * from INFORMATION_SCHEMA.DATAFILES;  select * 
from INFORMATION_SCHEMA.TABLESPACES;
Empty set (0.03 sec)

Empty set (0.00 sec)

mysql> CREATE TABLESPACE ts1 ADD DATAFILE 'datafile.dat' USE 
LOGFILE GROUP lg1 INITIAL_SIZE = 12M ENGINE NDB;
Query OK, 0 rows affected (2.35 sec)

mysql> select * from INFORMATION_SCHEMA.DATAFILES;  select * 
from INFORMATION_SCHEMA.TABLESPACES;
+--------------+--------+--------------+----------+------+------------+
| NAME         | ENGINE | PATH         | SIZE     | FREE | TABLESPACE |
+--------------+--------+--------------+----------+------+------------+
| datafile.dat | NDB    | datafile.dat | 12582912 |   11 |            |
+--------------+--------+--------------+----------+------+------------+
1 row in set (0.00 sec)

+------+--------+---------+-------------+-----------------------+
| NAME | ENGINE | VERSION | EXTENT_SIZE | DEFAULT_LOGFILE_GROUP |
+------+--------+---------+-------------+-----------------------+
| ts1  | NDB    |       1 |     1048576 |                     0 |
+------+--------+---------+-------------+-----------------------+
1 row in set (0.00 sec)

mysql> CREATE TABLE t1 
(pk1 int not null primary key auto_increment,
 b int not null, 
c int not null) 
tablespace ts1 storage disk engine ndb; 
Query OK, 0 rows affected (0.62 sec)

mysql> insert into t1 (b,c) values (1,2),(2,3),(3,4),(1,2),(2,3),(3,4),(1,2),(2,3),(3,4),
(1,2),(2,3),(3,4),(1,2),(2,3),(3,4),(1,2),(2,3),(3,4),(1,2),(2,3),(3,4),(1,2),
(2,3),(3,4),(1,2),(2,3),(3,4),(1,2),(2,3),(3,4),(1,2),(2,3),(3,4),(1,2),(2,3),(3,4);
Query OK, 36 rows affected (0.11 sec)
Records: 36  Duplicates: 0  Warnings: 0

mysql> select * from INFORMATION_SCHEMA.DATAFILES;  select * 
from INFORMATION_SCHEMA.TABLESPACES; 
+--------------+--------+--------------+----------+------+------------+ 
| NAME         | ENGINE | PATH         | SIZE     | FREE | TABLESPACE |
+--------------+--------+--------------+----------+------+------------+
| datafile.dat | NDB    | datafile.dat | 12582912 |    9 |            |
+--------------+--------+--------------+----------+------+------------+
1 row in set (0.02 sec)

+------+--------+---------+-------------+-----------------------+
| NAME | ENGINE | VERSION | EXTENT_SIZE | DEFAULT_LOGFILE_GROUP |
+------+--------+---------+-------------+-----------------------+
| ts1  | NDB    |       1 |     1048576 |                     0 |
+------+--------+---------+-------------+-----------------------+
1 row in set (0.00 sec)

mysql> CREATE TABLESPACE ts2 ADD DATAFILE 'datafile2.dat' 
USE LOGFILE GROUP lg1 INITIAL_SIZE = 12M ENGINE NDB;
Query OK, 0 rows affected (2.18 sec)

mysql> select * from INFORMATION_SCHEMA.DATAFILES;  select * 
from INFORMATION_SCHEMA.TABLESPACES;
+---------------+--------+---------------+----------+------+------------+
| NAME          | ENGINE | PATH          | SIZE     | FREE | TABLESPACE |
+---------------+--------+---------------+----------+------+------------+
| datafile2.dat | NDB    | datafile2.dat | 12582912 |   11 |            |
| datafile.dat  | NDB    | datafile.dat  | 12582912 |    9 |            |
+---------------+--------+---------------+----------+------+------------+
2 rows in set (0.02 sec)

+------+--------+---------+-------------+-----------------------+
| NAME | ENGINE | VERSION | EXTENT_SIZE | DEFAULT_LOGFILE_GROUP |
+------+--------+---------+-------------+-----------------------+
| ts1  | NDB    |       1 |     1048576 |                     0 |
| ts2  | NDB    |       1 |     1048576 |                     0 |
+------+--------+---------+-------------+-----------------------+
2 rows in set (0.00 sec)

mysql> ALTER TABLESPACE ts1 ADD DATAFILE 'datafile3.dat' 
INITIAL_SIZE=12M ENGINE NDB;
Query OK, 0 rows affected (1.85 sec)

mysql> select * from INFORMATION_SCHEMA.DATAFILES;  select *
 from INFORMATION_SCHEMA.TABLESPACES;
+---------------+--------+---------------+----------+------+------------+
| NAME          | ENGINE | PATH          | SIZE     | FREE | TABLESPACE |
+---------------+--------+---------------+----------+------+------------+
| datafile2.dat | NDB    | datafile2.dat | 12582912 |   11 |            |
| datafile3.dat | NDB    | datafile3.dat | 12582912 |   11 |            |
| datafile.dat  | NDB    | datafile.dat  | 12582912 |    9 |            |
+---------------+--------+---------------+----------+------+------------+
3 rows in set (0.02 sec)

+------+--------+---------+-------------+-----------------------+
| NAME | ENGINE | VERSION | EXTENT_SIZE | DEFAULT_LOGFILE_GROUP |
+------+--------+---------+-------------+-----------------------+
| ts1  | NDB    |       1 |     1048576 |                     0 |
| ts2  | NDB    |       1 |     1048576 |                     0 |
+------+--------+---------+-------------+-----------------------+
2 rows in set (0.00 sec)

the ‘free’ column is really the number of free extents. Not exactly ideal… maybe… but since that’s the unit of allocation in the data files, it sort of makes sense. The other option is to list number of extents * extent size. Maybe that’s clearer for people… there is the option of denormalising the tables and have extent size in the DATAFILES table too. There is something in my brain that makes that a hard leap though.

Although…. if you’re going to be querying the tables directly and not just using a pretty gui on top of it all, you should probably know what you’re doing anyway.

Although, both a great benefit (and curse) of commoditising the database market is the fact that you get all sorts as users. This is interesting in cluster as it is naturally a bit more complex than a simple client-server RDBMS.

we also need a NODE column as well. which will probably cause confusion for non-cluster users and the like :)

(for the unintiated, this is work being done in a branch off the 5.1 tree for NDB disk data. we’ll push it to the main 5.1 tree at some point). don’t go thinking this is production ready any time soon (in other words insert a standard disclaimer).

Melbourne MySQL User Group

I’m getting responses of people wanting to come to the next meeting. This is all good. Looks like we’ll have more people than last time (which is good). So growing it is.

Considering that a lot of people got information on the last one less than a week before the event, and this time it’s about two weeks – I feel good about it.

Full details at: http://mysql.meetup.com/93/

the bastedo blog – replication in mysql 5

the bastedo blog

(I did matching versions [5.x] don’t know how diff versions will work)

Setting up a slave with a newer version of MySQL is quite a common setup. It has a couple of advantages:
– it lests you test a new version before deploying on the master (to test that everything goes smoothly)
– it lets you test new major versions (e.g. 5.0) before they are released GA (helps find bugs that may affect your setup).

I know at least one customer generally has a slave runnin the latest BK tree – just to be sure that nothing is going to even potentially break for them. Kudos to them :)

Having a slave that you use for backups is a great idea. No extra load on the master (i.e. you can safely stop the db on the slave and back things up quickly – without having locks held on your master!).

Also, if your master suffers a meltdown, you have a recent live backup system ready to take its place!

Microsoft loses in Eolas patent ruling | CNET News.com

Microsoft loses in Eolas patent ruling | CNET News.com

Come on Microsoft – join us in the fight against software patents. This clearly hurts the entire industry – be it big vendors like yourself or small ones.

Let’s not all get royally screwed.

MySQL Melbourne Meetup

When:
Tuesday September 13th 7:00pm

Where:
Miro International Pty Ltd
Level 18, 31 Queen Street
Melbourne 3000, Australia

Thanks to Miro for offering their offices for the meeting.

What’s happening?
Stewart Smith will talk about:
– What’s new in MySQL land
– Introduction to MySQL AB the company
– what it does
– what it offers
– Graphical tools for MySQL
– MySQL Administrator
– MySQL Query Browser

RSVP
Please RSVP via our meetup.com site:
http://mysql.meetup.com/93/

After
We can head to a pub or out for curry.

Naturally, it was going to hit Slashdot with something like “MySQL and SCO Join Forces”

(insert disclaimer about this being my own views – no that of MySQL AB)

Slashdot | MySQL and SCO Join Forces

Some people seem to think that porting your application to a newer version of an OS, having a trial version of your subscribtion-based support shipping with every copy of that OS and access through that OS vendors reseller channel is a bad thing.

Granted, a lot of people think that certain actions of said OS vendor are just plain retarded. Myself included – it would be much better if they actually focused on products. That being said, there’s more than one OS vendor that does just plain dumb stuff – or, to use the more emotive “evil” word.

Of course, there’s part of the /. crowd that seem to think we must be evil for porting to a SCO platform – but by their silence (and sometimes “screw you guys, I’m going to X RDBMS”) it must be okay for others to do it (note that X RDBMS already supports SCO platforms)?

Besides, anybody who’s really used MySQL will know how easy it is to move your database from one platform to another – really empowering you to make sure you OS vendor gives you the best deal possible – because you can easily move to where the grass is greener.

MemberDB election-results performance on new laptop

So I picked up my new laptop on friday. It’s an ASUS V6V – nice and fast, light, good resolution screen and lots of disk and RAM (it came with 1GB, I’ve got 2GB).

Anyway, the transfer of data from my PowerBook went fine. I waited for xfsdump to dump /home from the powerbook to a firewire drive (and for “waiting” I do mean going out and seeing Charlie and the Chocolate Factory – which was very good).

Installing Ubuntu on the ASUS went like a dream. Everything, and i do mean everything worked out-of-the-box with only one tweak. That was uncommented the ACPI sleep configuration option do-dad in /etc/default/acpi-something-foo to get suspend to ram working.

The WEP didn’t work in the installer, so I initially just used the GigE adapter until the first reboot.

The firewire drivers don’t really behave with this laptop atm… that dreaded “aborted sbp2 command” error too often – so abandoned that and futzed around with a private net and NFS to xfsrestore /home.

Go to bed, awake later to find /home on new laptop (with an extra 23GB of free space!). I had to, of course – rebuild those essential packages for x86 instead of ppc – namely wesnoth.

oh, and cleaning out the ppc binaries from my mysql bk trees and doing a x86 build (I also had to change my CC from ‘ccache distcc powerpc-linux-gcc’ to ‘ccache distcc i386-linux-gcc’). One thing is for certain, it’s quicker at building things – even if the fan ramps up a bit when doing so :)

MySQL builds pretty quickly when you have a 2.8Ghz P4 and a 2.13Ghz Pentium M building it.

Anyway, set up all the apache foo for hacking on the LA website and MemberDB today. A load of the elections-result page on digital (the LA server – dual PIII 1.133Ghz) takes about fourteen or fifteen seconds using PostgreSQL as the database.

I previously reported that using MySQL (InnoDB tables) I got about twice the performance on my old laptop (1Ghz G4).

Well, on this one (2.13Ghz Pentium M) I’m getting the page loading in under three seconds. Sweet. Maybe I won’t go ahead and try to optimise some of the queries :)

(the query cache is probably coming into this – but i did do the query several times – so it’s not as if there’s any unfair advantage anywhere).

I’m using the 5.0.12-max-beta gcc dynamic build as downloaded from mysql.com for these runs. All other packages (apache2, php) are as shipped in Ubuntu. The my.ini file is as-shipped (err.. i think so: no query log, no binlog, slow query log enabled and some paths changed)

lathiat: Avahi

lathiat: Avahi

Trent has been blogging quite often about Avahi. It does look like a good project to watch. Promises ease of use (for the coder who really doesn’t care about Rendezvous/Bonjour/ZeroConf/whatever-they-call-it-this-week internals and just wants Cool Functionality(tm) in their app).

Maybe integrating this would be cool for mysqld and the gui tools. (i’ve toyed with the thoughts of using it in cluster… but do we relaly want another thing that can possibly fail in a HA environment… probably not – considering using DNS is usually a bad idea).

Why “returns -1 on error” is bad

(a general note on what’s good practice)

In C, 0 is false and !0 is true.

In the dim past there was an elsewhere where 0 was true and !0 was false. Why? Because there can be more than one error state and this is usually more interesting than how many ways success could have been acheived.

Well, that sucks too – there’s information on success that could be useful (e.g. we succeeded, but only n bytes worth instead of the m you asked for).

So, the way of <0 on failure and else success came about for packing the maximum amount of information into the int that we commonly return from functions (and usually fits nicely in a register and it all leads to hugs, puppies and a warm feeling inside).

So what do most people do on error? Return -1.

Hrrmm… this casually (if not totally) defeats the point. In any function that does any real work, there’s going to be more than one place where failure could occur (even if it’s an error path that should never really happen… it will, but never to you… always to a guy somewhere in a country that you didn’t know existed and knows less $native_language than you have digits).

So if you get a bug report in with a log message (because you do print log messages when errors occur! – especially non-totally-fatal ones!) about a failure, and you go to look at that function and go “aha! this function must have returned -1!” Well, it just so happens that there are five places that could return -1. Where did your program fail? Without a core dump or something, you will never know.

So, what if these five places returned different error codes (which, of course, you wrote to the log)? Then you’d be able to narrow down the search for buggy code!

It doesn’t have to be a unique number, or even user understandable (especially when these are places that shouldn’t fail – or so you think) but it makes your job a hell of a lot easier if you can quickly jump to the bit of code you should look at.

In cluster, we have this great system where when really bad stuff happens, we get these nice trace logs of what signals have been cruising around the cluster recently. This greatly helps with debugging. It sort of makes you go “wow” when you first see a crash reported, trace file follows, and then a patch a few hrs later that fixes the problem. This is because it’s an aid in tracking down exactly where to look for the problem.

“It crashed” is never a useful bug report. But only having the facilities in your software for only being able to say “it crashed” unless you’re a developer guru dude isn’t very useful either.

The various backtrace reporting tools do a bit to help. As always, the more information the better. This is certainly the case when you look at the backtrace and go “how on earth did we ever get there?” or the stack is just completely hosed and you have no hope of finding your arse from your elbow (although these days valgrind will help you here).

Here endith the lesson.

Comments Are Evil

When a comment above a function says “returns -1 on error” and the code does the exact oposite (returns -1 anyway except if there was out of memory error, which may be #defined to -1 anyway) it’s a bit annoying when you first look at it.

Remember kids, comments in code are evil. They are wrong – or misleading at best. They only ever say what one person at some point in the past thought they beleived the code did. The definitive record is the code itself.

(there are possible exceptions to this rule… maybe… internals can be good to document – but arguably it should be *away* from the code so that you don’t start thinking the documentation is accurate and up to date – because it’s not).

ndb_mgmd restart

One of the things I’m working on is adding the ability to use ndb_mgm to issue a restart command to ndb_mgmd (i.e. from a management client, get management servers to restart). At the moment, you have to go and shut it down then start it yourself.

So why would you ever want to restart a management server? Well, bugs aren’t really a reason – I’ve never heard of anyone having to restart the management server “just to get something to work again”.

The reason is online configuration upgrades.

There are a bunch of parameters you can change without having to restart your cluster. We call it a “rolling upgrade” as it’s the same procedure as upgrading one compatible version to another.

The whole procedure would be really easy if we only ever had one management server (which is what a lot of people have anyway – you only ever need a mgm server to have a node join the cluster).

It’s also tricky because you don’t want management servers up and serving different configurations. This would tend to be bad and never lead to hugs and puppies.

For supporting more dramatic configuration changes (e.g. add/drop node) we’ll be needing configuration locks and enforcing that everyone agrees on the config they’re serving out.

There exists some code from a previous effort a few years ago. So I’m having a look through it and trying to work out the state of everything. There seems to be a bit of bitrot and I’m trying to work ouf if anything is worth using.

The approach that I’ve come up with is to have a “single user mode” for the mgm server – i.e. nobody but one connection can do anything. This is where we’d do updates and changes before unlocking.

I wasn’t really caring about notifying ndbd about changes as the way you do things atm is to restart each ndbd and they then pick up the changes.

Otherwise, we really want a “this parameter changed” rather than “there’s a new configuration”.

So, the ‘mgm restart’ thing is really going to be implemented as “config reload” – getting the mgmds to stop and restart in the right order and so there is never more than one version of the configuration is being served at a time.

hrrm… back to the code to figure out what’s going on with this older stuff.

Fancy shortcuts to MySQL Bugs

So Elliot Murphy is talking about QuickSearch shortcut for bugs.mysql.com which is quite useful if you use Firefox.

However, I’m using Epiphany (which is based on the same rendering engine, but is a bit more GNOMEy – and besides, i’ve been using it for a while, i have all my bookmarks and saved passwords etc there).

So, a quick PHP script later, and I have a nice little command line version of the same script.


#!/usr/bin/php
< ?php system('sensible-browser ' .'http://bugs.mysql.com/search.php?' .'cmd=display\&limit=10\&' .'status=All\&search_for=' .urlencode(implode(' ',array_slice($argv,1)))); ?>

Useful! Usage is like:


$ mybug 10950
$ mybug ndb

hope it helps.

maybe i’ll switch to firefox one day… when it’s faster.

MySQL 5.0.10 released!

The next beta of 5.0 is out – 5.0.10. So go ahead and download it. You’re not cool unless you download it (you can find it here http://dev.mysql.com/downloads/mysql/5.0.html)

File swap it, torrent it, burn it to cds and give it to friends. Don’t you just love free software!

This release fixes a bunch of bugs that were holding back MemberDB from being usable on MySQL. This was bugs in new features, and the main one had already been reported.

There’s also been a few fixes for cluster which is always a good thing. Nothing show stopping though. We must be good :)

noise cancelling headphones a no-no

Well, this review over at ZDnet seems to say that the Bose Quiet Comfort 2 Acoustic Noice Cancelling headphones have leather. The suck. Well, that strikes them off my list.

In case you didn’t know, I don’t do the leather thing.

There’s also a disturbing review over at Amazon from a guy who seems to know what he’s talking about (Grado make great ‘phones – mine are supurb).

So, the sound quality may not be all that great (although people rave about the noise reduction – and that’s worth something, but maybe for $299USD I’d expect something that at least matches my $150AUD Grado ‘phones).

So, anyone got ideas for something decent for travelling with (planes) that doesn’t involve a dead cow?

Maybe in-ear earphones is what i’m looking for….

A warmer place to work…

So, aparrently our 7 year heater is really old and we shouldn’t expect much from it.

No, really.

A switch had broken. Again. The same one that was replaced less than two years ago.

So, I ask our Northern European friends – what’s a good brand and model for a natural gas central heater that’s not going to require fixing every year?