Yesterday, our team ran smack into the perils of using bitfields in C/C++.

Some background: Bitfields are useful because when the number of objects/structures allocated is in the millions every byte counts! We write for performance, and saving 100-200 MB of memory is something we strive for.

Of course, it is a delicate balancing act because bitfields have a host of other issues:

• Accessing and storing values to and from bitfields are comparatively slow
• Memory padding rules vary from compiler to compiler
• Somewhat complicates threading (trust me, you really don't need threading to be made more complicated! :-D)
• Whether the bitfield is signed or unsigned is implementation defined

The first issue we ran into was when we assigned 5 bits to a variable that would store 20 values. The program ran fine for a short while then crashed. When we traced the issue it turned out that our compiler (MSVC) was treating the bitfield as signed and so values greater than 15 would overflow.

Anyhow that wasn't difficult to trace, so we added a bit and chugged along nicely.

The second issue was subtler and far more dangerous.

So Bitfields were working fine and we happily chugged along. Our product uses less memory, customers were happy, we were happy. All was right with the world.

Due to some scaling issues we are rewriting some modules, including the core object that contains these bitfields. During the rewrite we re-examined the object. By now, many developers had added fields. But they had been careful. 21 members had been cajoled into exactly 32 bits! Not bytes - bits! Looking good, looking good.

Then we noticed that the bitfields had varying types. Most were byte (char) flags, but one was a Word, another was an enum, and yet another was a long. They were interspersed between the flags.

From our previous experience with memory layouts this seemed...somewhat wrong. There was a packing pragma but still...

So we measured again.

The blasted object took 9 bytes!

9 bytes instead of 4! That's a >100% jump in size!!

The first thing we did was move all defined types to byte and casted out when needed. The size dropped immediately to 5 bytes. This was with no functional changes at all! Why couldn't you do this yourself compiler? Why? Why? Why?

Still we were one byte too much. We needed to group the fields carefully so that every field would fall correctly within a byte (i.e. there was no bit 'padding' needed).

Voila! 4 bytes!

Bugs are part of a developers life, but they are especially aggravating when they creep into your "clever" code that you were proud of. Nevertheless, the upside is the next release can expect to have even less memory usage and improved customer experience.