C bitfields considered harmful

Posted on by

In C (and C++) you can specify that a variable should take a specific number of bits of storage by doing “uint32_t foo:4;” rather than just “uint32_t foo”. In this example, the former uses 4 bits while the latter uses 32bits. This can be useful to pack many bit fields together.

Or, that’s what they’d like you to think.

In reality, the C spec allows the compiler to do just about anything it wants with these bitfields – which usually means it’s something you didn’t expect.

For a start, in a struct -e.g. “struct foo { uint32_t foo:4; uint32_t blah; uint32_t blergh:20; }” the compiler could go and combine foo and blergh into a single uint32_t and place it somewhere… or it could not. In this case, sizeof(struct foo) isn’t defined and may vary based on compiler, platform, compiler version, phases of the moon or if you’ve washed your hands recently.

Where this can get interesting is in network protocols (OMG DO NOT DO IT), APIs (OMG DO NOT DO IT), protecting different parts of a struct with different mutexes (EEP, don’t do it!) and performance.

I recently filed MySQL bug 74831 which relates to InnoDB performance on POWER8. InnoDB uses C bitfields which are themselves bitfields (urgh) for things like “flag to say if this table is compressed”. At various parts of the code, this flag is checked.

When you apply this simple patch:

--- mysql-5.7.5-m15.orig/storage/innobase/include/dict0mem.h
+++ mysql-5.7.5-m15/storage/innobase/include/dict0mem.h
@@ -1081,7 +1081,7 @@ struct dict_table_t {
        Use DICT_TF_GET_COMPACT(), DICT_TF_GET_ZIP_SSIZE(),
        DICT_TF_HAS_ATOMIC_BLOBS() and DICT_TF_HAS_DATA_DIR() to parse this
        flag. */
-       unsigned                                flags:DICT_TF_BITS;
+       unsigned                                flags;

I get 10,000 key lookups/sec more than without it!

Why is this? If you go and read the bug, you’ll see that the amount of CPU time spent on the instruction checking the bit flag is actually about the same… and this puzzled me for a while. That is, until Anton reminded me that the PMU can be approximate and perhaps I should look at the loads.

Sure enough, the major difference is that with the bitfield in place (i.e. MySQL 5.7.5 as it stands today), there is a ld instruction doing the load – which is a 64bit load. In my patched version, it’s a lwx instruction – which is a 32bit load.

So, basically, we were loading 8 bytes instead of 4 every time we were checking if it was a compressed table.

So, along with yesterday’s lesson of never, ever, ever use volatile, today’s lesson is never, ever, ever use bitfields.

15 thoughts on “C bitfields considered harmful”

  2. No reason to suggest that other architectures aren’t affected. The compiler is completely within its rights to do this on any platform. It probably made some decision somewhere to pack them all into a 64bit type as there’s probably some part in the code where this appears to make the most sense. There’s also no reason it couldn’t make a different decision at different optimization levels.

  3. unsigned foo:XX cannot have more bits that “unsigned” which is “unsigned int” which is , unless you run on a Cray, 32 bits. Making it 32 bits does not make much sense. i’m not telling that compiler is prohibited from doing it, but it is not a good idea

  4. “making it 64 bits” does not make sense, of course, sorry for typo. 32 bit makes perfect sense.

  6. You can blame me for introducing bitfields into the InnoDB dict_*_t structs. The goal at that time (MySQL 5.0 or 5.1) was to reduce the memory consumption of the data dictionary cache. InnoDB did not have any cache eviction. If you touched any table, its metadata would remain in memory until DROP TABLE or serve shutdown.

    In MySQL 5.6, Sunny Bains fixed this problem by enabling cache eviction for table definitions. We can say that the bitfield usage got past is ‘best before’ day at that time.

  7. Ahh… back in the good old days where every bit mattered. My guess is this came out of the quest to have SAP run on MySQL with its eleventy billion tables it loaded before you did anything. Also back at the time when the majority of mysqld was 32bits I guess too.

  8. Actually… how awesome is it that *this* is a bottleneck now? Remember all the other problems and bottlenecks that were there 10 years ago in the 5.0 days?

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.