innodb and memcached

Posted on by

I had a quick look at the source tree (I haven’t compiled it, just read the source – that’s what I do. I challenge any C/C++ compiler to keep up with my brain!) that’s got a tarball up on labs.mysql.com for the memcached interface to innodb. A few quick thoughts:

  • Where’s the Bazaar tree on launchpad? I hate pulling tarballs, following the dev tree is much more interesting from a tech perspective (especially for early development releases). I note that the NDB memcached stuff is up on launchpad now, so yay there. I would love it if the InnoDB team in general was much more open with development, especially with having source trees up on launchpad.
  • It embeds a copy of the memcached server engines branch into the MySQL tree. This is probably the correct way to go. There is no real sense in re-implementing the protocol and network stack (this is about half what memcached is anyway).
  • The copy of the memcached engine branch seems to be a few months old.
  • The current documentation appears to be the source code.
  • The innodb_memcached plugin embeds a memcached server using an API to InnoDB inside the MySQL server process (basically so it can access the same instance of InnoDB as a running MySQL server).
  • There’s a bit of something that kind-of looks similar to the Embedded InnoDB (now HailDB) API being used to link InnoDB and memcached together. I can understand why they didn’t go through the MySQL handler interface… this would be bracing to say the least to get correct. InnoDB APIs, much more likely to have fewer bugs.
  • If this accepted JSON and spat it back out… how fast would MongoDB die? weeks? months?
  • The above dot point would be a lot more likely if adding a column to an InnoDB table didn’t involve epic amounts of IO.
  • I’ve been wanting a good memcached protocol inside Drizzle, we have ,of course, focused on stability of what we do have first. That being said…. upgrade my flight home so I can open a laptop… probably be done well before I land….. (assuming I don’t get to it in the 15 other awesome things I want to hack on this week)

5 thoughts on “innodb and memcached”

  1. Yes, if the could talk JSON that would be neat for everyone. As it stands, I just need drizzle to talk memcached. Hell, I already have this sort of on my front end MySQL servers. I have a two column table for misc data that small applications use. The apps only use 10-20 pieces of data. A class with get/set/delete that uses PHP serialize and makes some queries was all I needed. How does it scale? Well, not sure. It is working fine for its current use on our most requested page. But, the data set is small.

    Now, if I could talk memcached protocol using json directly into a table…. Wow, I would have immediate use for Drizzle in our stack. I plan to transition our front end MySQL servers already. But, this would let me start sneaking it in sooner. We were just talking about having a more persistent memcache solution that could be replicated. The possibilities are huge here for Drizzle.

  2. Hi Stewart! Hope to see you here somewhere at the MySQL conf.

    Source for memcached is on github. I’m sure you’ve heard of github. :)

    Those tracking the project have seen a few email updates recently. Yes, it’d been a while since there were major updates, but the engine-pu branch up there is a proposed update and we’ve been pointing out the new stuff on the mailing list.

    With regard to something memcapable which internalizes JSON and does more with it, absolutely agree. You probably know, that’s exactly what we’re working through right now, having hooked up with the CouchDB folks and merging Membase/CouchOne to form Couchbase.

    Keeping the IO below “epic” is part of what we already do in Membase, and that won’t change. Having the ability to add/remove nodes elastically is also important here, and something we bring from Membase. CouchDB brings incremental map-reduce to the party.

    This is obviously different than the “talk JSON to a table” that Brian mentions, but I’m not certain we need tables for this. To be sure, there are times going relational with the data makes sense (check out my session at the conference Wednesday), but relaxing things a bit opens up other possibilities.

  3. Yeah, i have the memcached git tree in ~/src/memcached :) Was more thinking of git/bzr tree for the innodb memcached engine

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.