Programmers love their editors. Considering we spend most of our work lives within an editor that is not surprising. Most of what we do is in plain text and our editors are geared to do very sophisticated things with text (
:g/^/m0 anyone?)
But occasionally we do have the requirement to view and edit binary files. This is common when experimenting with new data formats or checking corrupt data. So what's the best way to do this?
EMacs Hexl Mode
EMacs, as expected, has an excellent Hex mode for viewing binary dumps (
M-x hexl-mode):
Basic editing is also supported:
- Normal chars or
C-q<keystroke> can be used directly.
-
C-M-x,
C-M-o,
C-M-d will insert a given hex/octal/decimal value.
Great! EMacs gives us basic viewing and editing capabilities. This is enough for most purposes but for truly powerful editing capabilities read on...
Vim and xxd
xxd is a powerful tool for dealing with binary files textually. Vim, of course, allows us to integrate
xxd into our editing sessions and the combination works as a fantastic binary editor. It takes a tiny bit more work to set up but it's worth it!
Step 1: Start vim in binary mode:
vim -b <filename>
This tells vim not to do any clever tricks with our file.
Step 2: Lets see those unprintable characters in hex:
:se display=uhex
This step is not really needed but it's sometimes useful to get a feel of the data as a raw dump.
Step 3: Convert using xxd
This is the most important step. xxd allows us to convert to and from binary and text thus allowing us to edit the binary file as if it were text! Let's see basic editing first:
:%!xxd
Hmmm...all that and it finally just looks like EMacs hexl!
However, there is an important difference. Here we can add and remove lines which will actually change the size of the output. This was not possible in hexl and is actually very useful.
Step 4: Convert back to binary before saving and we're done!
:%!xxd -r
Note two important points:
- xxd only pays attention to the hex-dump section, thus you cannot edit the plain text (changes will be silently ignored when converting back).
- xxd pays attention to the line numbers at the beginning of each line. It will
lseek() to the correct position in the output file for each line. Hence, if you want to move data around, be careful not to move line numbers as well. xxd will fill gaps with null data.
Advanced Steps:
:%!xxd -p
:%!xxd -p -r
This gives us a "plain" dump that we can edit without worrying about line numbers or ascii characters. In order to map it with the hex dump we can use the same number of columns:
:%!xxd -p -c 16
:%!xxd -p -c 16 -r
This is helpful because we can see the line numbers we want to change in the basic dump then jump to the same line number in the "plain" dump and make our changes.
Conclusion
For simple editing of binary files use Emacs hexl, but for more advanced editing Vim+xxd is the way to go.