Coming to the end of OSDC

Sitting in the last lightning talk session of OSDC2007 and in a couple of hours will be (surprise) jumping on a plane back home.

Has been a good and interesting conference. Met cool people, done cool hacking and chatting around the place.

Gave my talk today, and seemed to go down well…. at least Paul Fenwick liked it :)

reading maildirs…. fast…

So, for a side project i’m hacking on, i’m wanting to read in Maildirs really fast (and then pump them into something else… for current purposes I’m just putting everything in one file.. getting the read speed up is of current importance).

I’ve done a bit of experimenting and my current method (which seems to be as fast as any):

  1. read the directory (cur)
  2. sort by inode number
  3. foreach 1000 inodes:
    1. sort by start block number
    2. read message

This makes a couple of assumptions:

  • sequential inode numbers are close to each other on disk (making stat(2) cheaper)
  • mail messages are small… likely to be in 1 extent, so start block is a good metric for locality.

Oh, some of this is specific to XFS… which is what I care about (and it turns out you don’t need to be root to get an extents list for a file on XFS).

Working desktop (with Ubuntu 7.04 beta)

yay – got an install of Ubuntu 7.04 on my RAID5 with LVM with RAID1 /boot. Not only that, it’s booted – twice. Special.

Even plugged in my USB RAID1 array and managed to get it to mount… and am now importing photos into the f-spot store on it. Yay.

Lets see if it survives any amount of time…

RAID, LVM2 on USB disks with Ubuntu makes me a sad panda

Talk about a total pain to operate. After a reboot, the following is needed:

  1. modprobe dm_snapshot (if you don’t do this, you get “device-mapper: reload ioctl failed: Invalid argument” in step 6 – a really helpful error message. no dmesg or verbose things gets you any closer)
  2. some magic foo to scan the md array back
  3. pvscan
  4. vgscan
  5. lvscan
  6. lvchange -ay FooVG/barLV
  7. mount
  8. urgh.

Why this doesn’t just all detect itself on boot who knows. I’m surely not the only one doing this….

bluetooth fun

ffor Whatever reason it Seems to be impossible to
do more than one thing via bluetooth at once. i.e. i cannot fetch a photo from my phone while connected by gprs. the suck

also, the handwriting recognition on the 770 sucks. i totally miss this from the newton. it should, but obviously doesn’t tae into account the wrd you may be trying to write. thelikelyhood of me writing t9 is quite small compared to the probability of writing to.

i guess this could be an interesting thing to hack on though…

maybe i should start a ‘to hack on’ list.’ list.

more fun on Melbourne trains

just made the train, by about 4 seconds :)

but the fun continues, according to the display in the train, it’s been Parkdale station.for the last 5 stops. glad i’m not a tourist.

i’ve brought my camera in for this trip into the cbd – desperately hoping that if needed, the batteries for my flash hold up. i’ve got a few for the camera with me, so i can resort to the inbuilt one.. but it just doesn’t seem as good as the sb800 :)

written on a nokia 770 connected through gprs while listening to an ipod. so geeky geeky

Trains totally suck

sitting here at Patterson station, where i was early for the train. then get told it’s running 12 minutes late. suck

i’m on way into city to look more closely at facities at RMIT.

at least the weather is nice.

back in town later tonight for luv too. going to drive for that as it’s just so much more efficient.

pluggable NDB

Spoke with Brian the other day on what was required to get NDB to be a pluggable engine – and started hacking.

The tricky bits invole dependencies of things like mysqldump and ndb_restore on some headers to determine what tables shouldn’t be dumped (hint: the cluster database used for replication).

Also, all those command line parameters and global variables – they’re fun too. It turns out InnoDB and PBXT are also waiting on this. In the meantime, I’ve done a hack that puts config options in a table.

Currently blocked on getting the embedded server (libmysqld) to build properly – but i now have a sql/mysqld binary with pluggable NDB. All libtool foo too.

Hopefully i’ll be able to post soon with a “it works” post