So, probably like lots of people – i run a few web apps locally that I use for various purposes. In my case, this also includes some cool custom developed things.

I also use Zeroconf to easily discover all this foo around a network.

I run my critical mysql install by hand – it’s not constantly up. This is so, as somebody noticed (during Eben’s keynote at the MySQL Conference where he talked a lot about privacy) that one of the apps i run is entitled “tax”.

Since I’m somewhere other than at home, my mysql instance was stopped (much harder for people to grab the data out of it if the process isn’t running to begin with).

So yeah… good points – check what random people out on the network may have access to on your laptop – and know what you should not run as default (I’m careful there).

Nine dollars (US) of Water (how many hours would somebody on minimum wage have to work to buy this 1.5L of water?):

Apart from that, jetlagged – managed to find food, TV, internet. All good.

I’ll be putting photos up on my gallery (which is running a MySQL Cluster 5.1 backend – with disk data) over at:

http://saturn.flamingspork.com/gallery/v/conf/mysql2007/

Tomorrow morning (11.5hrs time actually) I’ll be on a plane to SFO (then down to Santa Clara) in preparation for the MySQL Conference.

So, if you’re in the area – give us a buzz. My aussie phone will work, as will traditional email.

Also on IRC… should be easy to find me (freenode).

I’ve used OfflineIMAP for quite a while now. On the whole I’m fairly happy with it. Today I sent this to the list:

Forgive the potentially bad python, not my native tongue :)

This patch is motivated by three things:
- offlineimap is extremely slow at syncing lots of locally deleted
messages
- offlineimap uses lots of memory
- LocalStatus files aren't written safely (a hard crash can cause
corruption)
        - I've been bitten by this in the past, causing a complete resync of
the folder... so I get duplicate messages.

I am currently using 4.0.14 (from Debian) with this patch. I used it to
convert the files and everything. Seems quite reliable and quick.

In my tests, execution time for a normal sync is relatively the same.

Execution time for when lots of messages have been deleted in a
reasonably sized folder (e.g. during re-organisation of mail folders) is
as much as 10x faster.

In my tests, running with 1 thread uses as much as 20% less memory with
this patch (i.e. about 160MB instead of 200MB+ for my maildir)

Disk space used by the LocalStatus files isn't much more... for me it
looks like it's 6.5MB now versus 4.5MB then. We get the added benefit of
indexes for all our queries... nice :)

I had disable the threading for copying messages as this means that
LocalStatus objects are shared between threads, which pysqlite doesn't
like (it asserts).

I think the part of this patch that implements the uidexists does
actually slow things down compared with having the messagelist.... a
more optimal implementation may be possible, but I think the other speed
improvements (and memory savings) are worth it.

A future patch may convert other storage types to sqlite (or similar) to
further reduce memory consumption (and hopefully runtime).

This does add a dependency on pysqlite... which is packaged in debian
(and ubuntu) - and i'm using the stock packages for these.

Comments very much appreciated.
Of course, the patch is here. I’m using it now… although I’ll warn you that it does update your .offlineimap to a new format (and doesn’t provide you a way to go back, without restoring the backed-up LocalStatus files and probably getting message duplicates).

So, those around the MySQL circles I tend to hang around may ask “Why not libmysqld?” (the embedded MySQL server). Well… a few reasons… sqlite is file-per-db (even though I’m essentially using file-per-table here), the python bindings are everywhere (and work), it’s tiny and crash safe.

You may also ask “Why?”… well, I’ve been re-organising a bunch of mail folders, which means deleting a *lot* of messages from some folders (and moving them to others).. offlineimap has been really slow at this. So I fixed it, with code (not whining).

I also wrote a bit-of-a-hack perl script to remove duplicate messages from a bunch of folders (a bug in offlineimap had caused me to get several copies of each message in a bunch of my folders a while ago). So that script is here. Commented out are bits to do comparison via md5 as well as message-id. Don’t use unless you know what you’re doing… it may also use a few hundred MB RAM on large (few hundred thousand messages) folder.

Hopefully these will help improve my productivity.
Now, back to my regular programming….

Okay, so one of the disks in a JBOD (well… single LVM) has been on the way out (hopefully can recover some stuff off it… there’s nothing completely important… but still).

I’ve now learnt and desktop has three new 320GB drives in a RAID5.

Currently installing Ubuntu 7.04 on it. I do have to say that the alternate install disk (which uses debian-installer) has a REALLY nice RAID and LVM setup now. If only it also let you pass parameters to mkfs it would be ideal.

Update: It got the bootloader horribly wrong though and I’ve gotten to piss-fart around trying to get LILO to install and boot. Current result? Blinking cursor in top left of screen. Fantastic… fucking fantastic.

Currently in the MySQL Cluster team office in Stockholm – and have been since Wednesday. I’ll be here for the next 3 weeks working in the office. This will be the longest amount of time I’ve worked in an actual office (instead of working from home) in more than 2.25 years!

I found Veronica Mars on TV last night… which is great, because I’ve sort of become addicted. Unfortunately, Sweden is a few episodes ahead of Australia…. so I’ve skipped a few now (go MythTV, record them for me baby). One really good thing about Swedish TV is that things are subtitled instead of dubbed – excellent if your Swedish isn’t that great (mine isn’t). Whenever here, I also seem to find some TV shows that look really interesting, except for the fact that it’s all in a language I don’t understand… certainly an interesting dilemma.

Today I’ve been working on material for the MySQL Cluster: The Complete Tutorial at the MySQL Conference and Expo. The conference is April 23-26 in Santa Clara, California (and if you register by March 14 you save $200 on registration). It’s going to be a SIX hour hands on tutorial (with breaks, don’t worry).  The hands on part (I think) is very important… that way you walk away with real-world knowledge that you can directly apply – not just with theory that you could have gotten from reading a bit here and there.

I’m really hoping that as many people with existing knowledge (esp MySQLers) can be around during the session to help people when needed… I have a feeling there’ll be a few.

In about 4.5hrs, I’ll be in a cab to the airport. At about lunchtime (1:30pm or so) London time, I’ll be in London. I’ll be there for a few days – until the 27th. If you’re around London or can make it, it’d be cool to hang. I plan to be a bit of a tourist here and there as I haven’t seen heaps of London and I do hear it’s nice :)

After that, I’ll be in Stockholm for about three weeks (I leave on the 19th… as it’s currently planned). So if you’re around, give me a yell!

My cell (mobile) number is pretty easy to find (hint: google my name along with my employer and look for a post on a mailing list… my work email sig has my phone number).

I’m in Stockholm for work, I’m going to be working in the office (which will be the longest amount of time I’ve gone to work in an office in over 2 years).