Linux print file in hex

How to create a hex dump of file containing only the hex characters without spaces in bash?

How do I create an unmodified hex dump of a binary file in Linux using bash? The od and hexdump commands both insert spaces in the dump and this is not ideal. Is there a way to simply write a long string with all the hex characters, minus spaces or newlines in the output?

9 Answers 9

Or if you want it all on a single line:

Format strings can make hexdump behave exactly as you want it to (no whitespace at all, byte by byte):

1/1 means «each format is applied once and takes one byte», and «%.2x» is the actual format string, like in printf. In this case: 2-character hexadecimal number, leading zeros if shorter.

I wonder if hexdump itself is able to append the newline (only) to the end of output.. (obvious appendage of ; echo makes it impossible to use as bash alias)

The iteration count and byte count default to one, so 1/1 may be omitted, leaving the hexdump -ve ‘»%.2x»‘

It seems to depend on the details of the version of od . On OSX, use this:

(That’s print as type hex bytes, with no address. And whitespace deleted afterwards, of course.)

perl -e 'local $/; print unpack "H*", <>' file 

Update to last comment: The «local $/» is unnecessary for «pack». For unpack, you need it but can alternatively put «undef $/». $/ is the line separator (default NL). undefined puts it into slurp-mode. So <> referenced in a string context pulls the whole binary file without parsing it into lines.

The other answers are preferable, but for a pure Bash solution, I’ve modified the script in my answer here to be able to output a continuous stream of hex characters representing the contents of a file. (Its normal mode is to emulate hexdump -C .)

tldr;

Explanation:

Use the od tool to print single hexadecimal bytes ( -t x1 ) — without address offsets ( -A n ) and without eliding repeated «groups» (-v) — from empty.zip , which has been redirected to standard input. Pipe that to tr which deletes ( -d ) the complement ( -c ) of the hexadecimal character set ( ‘[:xdigit:]’ ). You can optionally print a trailing newline ( echo ) as I’ve done here to separate the output from the next shell prompt.

References:

You can use Python for this purpose:

python -c "print(open('file.bin','rb').read().hex())" 

. where file.bin is your filename.

1. Open file.bin in rb (read binary) mode.
2. Read contents (returned as bytes object).
3. Use bytes method .hex() , which returns hex dump without spaces or new lines.
4. Print output.

I think this is the most widely supported version (requiring only POSIX defined tr and od behavior):

cat "$file" | od -v -t x1 -A n | tr -d ' \n' 

This uses od to print each byte as hex without address without skipping repeated bytes and tr to delete all spaces and linefeeds in the output. Note that not even the trailing linefeed is emitted here. (The cat is intentional to allow multicore processing where cat can wait for filesystem while od is still processing previously read part. Single core users may want replace that with < "$file" od . to save starting one additional process.)

I agree that historically connecting stdin is considered better and it’s still the correct solution for single core CPUs. However, with modern systems with many idle cores in a single socket CPU, I think it’s better to allow cat to read pipe buffer worth of input in parallel with the od at all times. This reduces the possibility of od stalling for reading the file.

I did some microbenchmarks and it seems that using the cat at the start does allow using multiple CPU cores for the task in parallel but in practice, the intercommunication between the cores causes so much extra work that avoiding the use of cat at the start of pipeline is still faster – at least for locally accessible files. In case the file was on remote network drive with slow connection, using the cat here could improve performance. That said, the od can only handle about 4 MB/s so it will be the bottleneck in most cases.

This code produces a «pure» hex dump string and it runs faster than the all the other examples given. It has been tested on 1GB files filled with binary zeros, and all linefeeds. It is not data content dependent and reads 1MB records instead of lines.

Dozens of timing tests show that for 1GB files, these other methods below are slower. All tests were run writing output to a file which was then verified by checksum. Three 1GB input files were tested: all bytes, all binary zeros, and all LFs.

hexdump -ve '1/1 "%.2x"' # ~10x slower od -v -t x1 -An | tr -d "\n " # ~15x slower xxd -p | tr -d \\n # ~3x slower perl -e 'local \$/; print unpack "H*", <>' # ~1.5x slower - this also slurps the whole file into memory 

Источник

How to convert file data to plain hex?

How to easily convert to/from plain machine-readable hexadecimal data (without any paddings/offsets/character view) with xdd or hexdump ? I’m tired of digging of some special format strings (and finding out that it suddenly starts wrapping lines after N characters or skip lines) or writing Perl one-liners every time. Why is it not as simple as base64 / base64 -d ?

3 Answers 3

If you get tired of writing this every time, create an alias.

I’m tired of looking at man page and gettings confused either by necessity to compose hexdump’s formt strings or by xxd’s «postscript» thing. (Is this «postscript» something about printers or about letters?)

@Vi, think of -p as «plain». I surmise the Postscript comment is because the Postscript language allows for in-line data (typically images) encoded exactly this way. So Postscript programmers may use xxd to convert binary images into a convenient form for embedding in a Postscript file.

How to easily convert to/from plain machine-readable hexadecimal data

$ xxd -plain test.txt > test.hex $ xxd -plain -revert test.hex test2.txt $ diff test.txt test2.txt $

$ xxd -plain test.txt > test.hex 

This writes a hex encoding of the data in test.txt into new file test.hex. The -p or -plain option makes xxd use «plain» hex format with no spaces between pairs of hex digits (i.e. no spaces between byte values). This converts «abc ABC» to «61626320414243». Without the -p it would convert the text to a 16-bit word oriented traditional hexdump format, which is arguably easier to read but less compact and therefore less suitable as a transmission format and slightly harder to reverse.

$ xxd -plain -revert text.hex test2.txt 

This uses the -r or -revert option for reverse operation. The -plain option is used again to indicate that the input hex file is in plain format.

I make the output filename different from the original filename so we can later compare the results with the original file.

The diff command outputs nothing — this means there is no difference between the original and reconstituted file contents.

I’m tired of digging of some special format strings

Use alias or declare functions in your .profile to create mnemonics so you don’t have to remember or dig about in man pages.

or just remember -plain and -revert .

Yes, there are new-line characters in the output. You want to avoid that. You can use the -c or -cols option to specify how long you want the output lines to be to attempt to avoid line-wrapping of the output. -c 0 gives the default length and the man page suggests 256 is the limit but it seems to work beyond that.

$ xxd -plain -cols 9999 test.txt > test.hex $ wc test.txt test.hex 121 880 4603 test.txt 1 1 9207 test.hex 

The wc wordcount command tells us how many lines, words and characters are in each file.

So 121 lines (880 words, 4603 bytes) of ASCII text were encoded as 1 line of hex digits.

Источник

Shell: How to read the bytes of a binary file and print as hexadecimal?

In shell, how can I read the bytes of a binary file I have, and print the output as hexadecimal numbers?

7 Answers 7

$ hexdump -x /usr/bin/hexdump 0000000 feca beba 0000 0300 0001 0700 0080 0300 0000010 0000 0010 0000 5080 0000 0c00 0000 0700 0000020 0000 0300 0000 00a0 0000 b06f 0000 0c00 0000030 0000 1200 0000 0a00 0100 0010 0000 107c 0000040 0000 0c00 0000 0000 0000 0000 0000 0000 0000050 0000 0000 0000 0000 0000 0000 0000 0000 

For information, the first column is the hexadecimal offset of the bytes, the rest of the line is 8 sets of two-byte displays, i.e. 16 bytes, which is why the second line starts with an offset of 10 , which is 16 in hexadecimal. The two-byte representation depends on the endianness of the system. Type man hexdump for the full details.

od has many options for finetuning.

If it can help somebody else, I wanted to have the last 4 bytes in hex of a binary file. Here is what I did that works. od -A n -t x1 -w1 -v myFile.bin | tail -n 4 | sed ‘s/ //g’ | paste -sd »

While we’re on od and hexdump , two more similar tools:

$ hd /usr/bin/od | head 00000000 7f 45 4c 46 01 01 01 00 00 00 00 00 00 00 00 00 |.ELF. | 00000010 02 00 03 00 01 00 00 00 20 8e 04 08 34 00 00 00 |. . 4. | 00000020 a4 a2 00 00 00 00 00 00 34 00 20 00 08 00 28 00 |. 4. . (.| 00000030 1b 00 1a 00 06 00 00 00 34 00 00 00 34 80 04 08 |. 4. 4. | 00000040 34 80 04 08 00 01 00 00 00 01 00 00 05 00 00 00 |4. | 00000050 04 00 00 00 03 00 00 00 34 01 00 00 34 81 04 08 |. 4. 4. | 00000060 34 81 04 08 13 00 00 00 13 00 00 00 04 00 00 00 |4. | 00000070 01 00 00 00 01 00 00 00 00 00 00 00 00 80 04 08 |. | 00000080 00 80 04 08 c4 9d 00 00 c4 9d 00 00 05 00 00 00 |. | 00000090 00 10 00 00 01 00 00 00 00 a0 00 00 00 20 05 08 |. ..| $ xxd /usr/bin/od | head 0000000: 7f45 4c46 0101 0100 0000 0000 0000 0000 .ELF. 0000010: 0200 0300 0100 0000 208e 0408 3400 0000 . . 4. 0000020: a4a2 0000 0000 0000 3400 2000 0800 2800 . 4. . (. 0000030: 1b00 1a00 0600 0000 3400 0000 3480 0408 . 4. 4. 0000040: 3480 0408 0001 0000 0001 0000 0500 0000 4. 0000050: 0400 0000 0300 0000 3401 0000 3481 0408 . 4. 4. 0000060: 3481 0408 1300 0000 1300 0000 0400 0000 4. 0000070: 0100 0000 0100 0000 0000 0000 0080 0408 . 0000080: 0080 0408 c49d 0000 c49d 0000 0500 0000 . 0000090: 0010 0000 0100 0000 00a0 0000 0020 0508 . .. 

Or, if you want to read the bytes one at a time and print them in your own format, try something like:

while read -n 1 byte; do ord=$(printf "%b" "$" | od -t x1 | < read offset hex; echo $hex; >) echo "$ord" done 

7f 45 4c 46 01 01 01 00 00 00