Linux create binary file

Create Binary files in UNIX

This question was out there for a while and I thought I should offer some bonus points if I can get it to work.

What did I do…

Recently at work, I wrote a parser that would convert a binary file in a readable format. Binary file isn’t an Ascii file with 10101010 characters. It has been encoded in binary. So if I do a cat on the file, I get the following —

[jaypal~/Temp/GTP]$ cat T20111017153052.NEW ==?sGTP?ղ?N. W. &Xx1?T?&Xx1?; ?d@#e? ?0H. |?X?@@(?ղ??VtPOC01 cceE??k@9??W傇??R?K?i2??d@#e. &Xx1&Xx. blackberrynet?/. . ??#ripassword??W傅?W傆??0H?? #R??@Vtc@@(?ղ??n?POC01 

So I used hexdump utility to make the file display following content and redirected it to a file. Now I had my output file which was a text file containing Hex values.

[jaypal~/Temp/GTP]$ hexdump -C T20111017153052.NEW 00000000 3d 3d 01 f8 73 47 54 50 02 f1 d5 b2 be 4e e4 d7 |==..sGTP. N..| 00000010 00 01 01 00 01 80 00 cc 57 e5 82 00 00 00 00 00 |. W. | 00000020 00 00 00 00 00 00 00 00 87 d3 f5 13 00 00 00 00 |. | 00000030 00 00 00 00 00 00 00 00 00 00 00 00 00 01 00 10 |. | 00000040 01 01 0f 00 00 00 00 00 26 58 78 31 00 b3 54 c5 |. &Xx1..T.| 00000050 26 58 78 31 00 b4 3b 0a 00 00 ad 64 13 40 01 03 |&Xx1..;. d.@..| 00000060 23 16 65 f3 01 01 0b 91 30 19 48 99 f2 ff ff ff |#.e. 0.H. | 00000070 ff ff ff 02 00 7c 00 dc 01 58 00 a0 40 40 28 02 |. |. X..@@(.| 00000080 f1 d5 b2 b8 ca 56 74 50 4f 43 30 31 00 00 00 00 |. VtPOC01. | 00000090 00 04 0a 63 63 07 00 00 00 00 00 00 00 00 00 00 |. cc. | 000000a0 00 00 00 65 45 00 00 b4 fb 6b 40 00 39 11 16 cd |. eE. k@.9. | 000000b0 cc 57 e5 82 87 d3 f5 52 85 a1 08 4b 00 a0 69 02 |.W. R. K..i.| 000000c0 32 10 00 90 00 00 00 00 ad 64 00 00 02 13 40 01 |2. d. @.| 

After tons of awk , sed and cut , the script converted hex values into readable text. To do so, I used the offset positioning which would mark start and end position of each parameter converted. The resulting file after all conversion looks like this

[jaypal:~/Temp/GTP] cat textfile.txt Beginning of DB Package Identifier: == Total Package Length: 508 Offset to Data Record Count field: 115 Data Source: GTP Timestamp: 2011-10-25 Matching Site Processor ID: 1 DB Package format version: 1 DB Package Resolution Type: 0 DB Package Resolution Value: 1 DB Package Resolution Cause Value: 128 Transport Protocol: 0 SGSN IP Address: 220.206.129.47 GGSN IP Address: 202.4.210.51 

Why did I do it

I am a test engineer and to manually validate binary files was a major pain. I had to manually parse through the offsets and use a calculator to convert them and validate it against Wireshark and GUI.

Now the question part

• Have an easy to read Input text file which would have Parameters : Values .
• User can simply put values next to them (eg Date would be a parameter and user can give date they want the data file to have).
• The script will cut out all relevent information (user provided information) from the Input text file and convert them into hex values.
• Once the file has been converted in to hex values, I wish to encode it back into binary.

First three steps are done

Problem

Once my script converts the Input text file in to a text file with hex values, I get a file like follows (notice I can do cat on it).

[visdba@hw-diam-test01 ParserDump]$ cat temp_file | sed 's/.\/&\n/g' | sed 's/../& /g' 3d 3d 01 fc 73 47 54 50 02 f1 d6 55 3c 9f 49 9c 00 01 01 00 01 80 00 dc ce 81 2f 00 00 00 00 00 00 00 00 00 00 00 00 00 ca 04 d2 33 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 10 01 01 0f 00 00 07 04 ea 00 00 ff ff 00 00 14 b7 00 00 ff ff 00 00 83 ec 00 00 83 62 54 14 59 00 60 38 34 f5 01 01 0b 58 62 70 11 60 f6 ff ff ff ff ff ff 02 00 7c 00 d0 01 4c 00 b0 40 40 28 02 f1 d6 55 38 cb 2b 23 50 4f 43 30 31 00 00 00 00 00 04 0a 63 63 07 00 00 00 00 00 00 00 00 00 00 

My intension is to encoded this converted file in to a binary so that when I do cat on the file, I get bunch of garbage values.

[jaypal~/Temp/GTP]$ cat temp.file ==?sGTP?ղ?N. W. &Xx1?T?&Xx1?; ?d@#e? ?0H. |?X?@@(?ղ??VtPOC01 cceE??k@9??W傇??R?K?i2??d@#e. &Xx1&Xx. blackberrynet?/. . 

So the question is this. How do I encode it in this form?

Why I want to do this?

We don’t have a lot of GTP (GPRS Tunnelling Protocol) messages on production. I thought if I reverse engineer this, I could effectively create a data generator and make my own data.

Sum things up

There may be sophisticated tools out there, but I don’t want to spend too much time learning them. It’s been around 2 months, I have started working on the *nix platform and just getting hand around it’s power tools like sed and awk .

What I do want is some help and guidance to make this happen.

Thanks again for reading! 200 points awaits for someone who can guide me in the right direction. 🙂

Sample Files

Here is a sample of Original Binary File

Here is a sample of Input Text File that would allow the User to punch in values

Here is a sample of File that my script creates after all the conversion from the Input Text File is complete.

How do I change the encoding of File 3 to File 1 ?

Источник

How to Edit and Convert Binary Files on Linux

If a file stores data in contiguous bytes format, a program trying to read this file will need to be instructed on how to read it since such files do not directly define a compatible method for reading their associated content.

This type of file is called a binary file. Opening such a file on a normal text editor program will only display unreadable characters. It is because binary data store data as bytes and not as textual characters.

The headers of a binary file are accompanied by an instruction set that reveals how its stored data should be read. Since binary files can store any data type, we can broadly classify all file types as either binary or text.

Create Binary File in Linux

We are going to create a sample binary file that we will try to edit. We will first create a text file with some data in it and then convert the text file to a binary file using the hexdump command.

$ echo "LinuxShellTips changed my Linux perspective!" > simple.txt $ hexdump simple.txt > simple.bin

The cat command should confirm to us that the binary conversion was a success.

Editing Binary Files in Linux

We are going to use the xxd command associated with the vim editor package. We first need to open the file on Vim editor using the -b flag since we are dealing with a binary file.

Use keyboard key [i] to enter insert mode and edit the binary file where needed. For instance, we can remove the first-line hex entries 694c to see what happens.

Convert Binary File to Text in Linux

To convert the binary file to text mode to view the implemented changes, we will switch to command mode using the keyboard key [Esc] and then key in the vim command:

Once we hit [Enter] on the keyboard, we should see the edits we made.

To save changes and quit vim use:

We have successfully demonstrated the possibility of editing a binary file in Linux using vim editor. Know of other cool ways of editing binary files? Feel free to leave a comment or feedback.

Источник

How to Create Binary File from Shell Script

you could try echo, that also allows arbitrary ascii chars (those numbers are octal numbers). I needed to write binary files from hex using busybox within an old Android shell. This printf with a redirect worked in my use case.

How do I make a shell script unreadable?

1. Write your script (script-base.sh) #!/bin/sh echo «Hello World»
2. Encrypt your script (give a password): openssl enc -e -aes-256-cbc -a -in script-base.sh > script-enc.
3. Write de Wrapper (script-final.sh): #!/bin/sh openssl enc -d -aes-256-cbc -a -in script-enc | sh —

What is the difference between a text file and a binary file?

A text file stores data in the form of alphabets, digits and other special symbols by storing their ASCII values and are in a human readable format. . Whereas, a binary file contains a sequence or a collection of bytes which are not in a human readable format. For example, files with .exe, . mp3, etc extension.

What is a binary file and how do I open it?

Opening a binary file is very easy. For example, use any hex editor to open the file and view its contents in multiple formats such as hexadecimal and Ascii. Use Google to find a free hex editor for your operating system. Many programmer’s editors have this feature built in or as an optional plugin.

How do I run a binary executable in Linux?

To make the file executable first, right-click on the binary file and then properties and go to permissions. On Permissions thick the checkbox with the option allows executing the file as program close the program and double click on the binary.

Is an executable a binary file?

Executable, a type of binary file that contains machine code for the computer to execute. Binary code, the digital representation of text and data.

What is a binary file in Linux?

Binaries are files that contain compiled source code (or machine code). Binary files are the files which contain compiled source code (or machine code). They are also called executable files because they can be executed on the computer. Binary directory contains following directories: /bin.

How do you convert binary to text?

1. Simply go here.
2. Select the Binary unit from the left box. Add the combination of 0 and 1 that you want to convert into a readable text form.
3. Then all you have to do is select the Text unit from the box next to it on the right and click “Convert”.
4. That’s all!

Where are binary commands stored?

Purpose. Utilities used for system administration (and other root-only commands) are stored in /sbin , /usr/sbin , and /usr/local/sbin . /sbin contains binaries essential for booting, restoring, recovering, and/or repairing the system in addition to the binaries in /bin .

How do you read binary?

The key to reading binary is separating the code into groups of usually 8 digits and knowing that each 1 or 0 represents a 1,2,4,8,16,32,64,128, ect. from the right to the left. the numbers are easy to remember because they start at 1 and then are multiplied by 2 every time.

Patterns

Can you use regex in awk?What is awk regex?What is expression in awk?How do you match patterns with awk?How do you escape quotes in awk?How do you use.

Mongodb

How do I download MongoDB on Ubuntu?Where is MongoDB installed in Ubuntu?How do I know if MongoDB is installed on Ubuntu?How do I start MongoDB in Lin.

Boot

Just boot into Windows and head to Control Panel > Programs and Features. Find Ubuntu in the list of installed programs, and then uninstall it like.

Latest news, practical advice, detailed reviews and guides. We have everything about the Linux operating system

Источник