Cutting columns in linux

Split output of command by columns using Bash?

Just as an example, let’s say I want to get the command name from a $PID (please note this is just an example, I’m not suggesting this is the easiest way to get a command name from a process id — my real problem is with another command whose output format I can’t control).

 PID TTY TIME CMD 11383 pts/1 00:00:00 bash 11771 pts/1 00:00:00 ps 

Now I do ps | egrep 11383 and get

Next step: ps | egrep 11383 | cut -d» » -f 4 . Output is:

The problem is that cut cuts the output by single spaces, and as ps adds some spaces between the 2nd and 3rd columns to keep some resemblance of a table, cut picks an empty string. Of course, I could use cut to select the 7th and not the 4th field, but how can I know, specially when the output is variable and unknown on beforehand.

10 Answers 10

One easy way is to add a pass of tr to squeeze any repeated field separators out:

$ ps | egrep 11383 | tr -s ' ' | cut -d ' ' -f 4 

Will not work if you happen to have a process with PID that contains the PID you are interested in as a subtring.

And also, the field nunbers will be off if some PID:s are space padded on the left while others are not.

I think the simplest way is to use awk. Example:

$ echo "11383 pts/1 00:00:00 bash" | awk '< print $4; >' bash 

For compatibility with the original question, ps | awk «\$1==$PID» or (better) ps | awk -v»PID=$PID» ‘$1=PID’ . Of course, on Linux you could simply do xargs -0n1

Is the ; in < print $4; >required? Removing it seems to have no effect for me on Linux, just curious as to it’s purpose

@igniteflow wouldn’t it indicate the end of the command if you wanted to continue adding on past the print statement?

Please note that the tr -s ‘ ‘ option will not remove any single leading spaces. If your column is right-aligned (as with ps pid).

$ ps h -o pid,user -C ssh,sshd | tr -s " " 1543 root 19645 root 19731 root 

Then cutting will result in a blank line for some of those fields if it is the first column:

Unless you precede it with a space, obviously

Now, for this particular case of pid numbers (not names), there is a function called pgrep :

Shell functions

However, in general it is actually still possible to use shell functions in a concise manner, because there is a neat thing about the read command:

$ | while read a b; do echo $a; done 

The first parameter to read, a , selects the first column, and if there is more, everything else will be put in b . As a result, you never need more variables than the number of your column +1.

while read a b c d; do echo $c; done 

will then output the 3rd column. As indicated in my comment.

A piped read will be executed in an environment that does not pass variables to the calling script.

out=$(ps whatever | < read a b c d; echo $c; >) arr=($(ps whatever | < read a b c d; echo $c $b; >)) echo $ # will output 'b'` 

The Array Solution

So we then end up with the answer by @frayser which is to use the shell variable IFS which defaults to a space, to split the string into an array. It only works in Bash though. Dash and Ash do not support it. I have had a really hard time splitting a string into components in a Busybox thing. It is easy enough to get a single component (e.g. using awk) and then to repeat that for every parameter you need. But then you end up repeatedly calling awk on the same line, or repeatedly using a read block with echo on the same line. Which is not efficient or pretty. So you end up splitting using $ and so on. Makes you yearn for some Python skills because in fact shell scripting is not a lot of fun anymore if half or more of the features you are accustomed to, are gone. But you can assume that even python would not be installed on such a system, and it wasn’t ;-).

Источник

How to cut first n and last n columns?

The first part of your question is easy. As already pointed out, cut accepts omission of either the starting or the ending index of a column range, interpreting this as meaning either “from the start to column n (inclusive)” or “from column n (inclusive) to the end,” respectively:

$ printf 'this:is:a:test' | cut -d: -f-2 this:is $ printf 'this:is:a:test' | cut -d: -f3- a:test 

It also supports combining ranges. If you want, e.g., the first 3 and the last 2 columns in a row of 7 columns:

$ printf 'foo:bar:baz:qux:quz:quux:quuz' | cut -d: -f-3,6- foo:bar:baz:quux:quuz 

However, the second part of your question can be a bit trickier depending on what kind of input you’re expecting. If by “last n columns” you mean “last n columns (regardless of their indices in the overall row)” (i.e. because you don’t necessarily know how many columns you’re going to find in advance) then sadly this is not possible to accomplish using cut alone. In order to effectively use cut to pull out “the last n columns” in each line, the total number of columns present in each line must be known beforehand, and each line must be consistent in the number of columns it contains.

If you do not know how many “columns” may be present in each line (e.g. because you’re working with input that is not strictly tabular), then you’ll have to use something like awk instead. E.g., to use awk to pull out the last 2 “columns” (awk calls them fields, the number of which can vary per line) from each line of input:

$ printf '/a\n/a/b\n/a/b/c\n/a/b/c/d\n' | awk -F/ '' /a a/b b/c c/d 

Источник

Using cut command to remove multiple columns

You should be able to continue the sequences directly in your existing -f specification.

As you’re skipping a single sequential column, this can also be written as:

To keep it going, if you wanted to skip 5, 7, and 11, you would use:

To put it into a more-clear perspective, it is easier to visualize when you use starting/ending columns which go on the beginning/end of the sequence list, respectively. For instance, the following will print columns 2 through 20, skipping columns 5 and 11:

So, this will print «2 through 4», skip 5, «6 through 10», skip 11, and then «12 through 20».

@JonathanLeffler Yeah, I know — but I kept the 6-6 in the example the show what it would look like to use a wider-range of columns. For instance, if the OP wanted to skip 5 and 10, he would need -f-4,6-9,11- . This may (or may not) be visualized if I collapsed it to just 6 . However, I edited to add an example with sequential columns too — so thank you for the tip =]

Sometimes it’s easier to think in terms of which fields to exclude.

If the number of fields not being cut (not being retained in the output) is small, it may be easier to use the —complement flag, e.g. to include all fields 1-20 except not 3, 7, and 12 — do this:

You are able to cut all odd/even columns by using seq:

This would print all odd columns

echo 1,2,3,4,5,6,7,8,9,10 | cut -d, -f$(seq -s, 1 2 10) 

To print all even columns you could use

echo 1,2,3,4,5,6,7,8,9,10 | cut -d, -f$(seq -s, 2 2 10) 

By changing the second number of seq you can specify which columns to be printed.

If the specification which columns to print is more complex you could also use a «one-liner-if-clause» like

echo 1,2,3,4,5,6,7,8,9,10 | cut -d, -f$(for i in $(seq 1 10); do if [[ $i -lt 10 && $i -lt 5 ]];then echo -n $i,; else echo -n $i;fi;done) 

This would print all columns from 1 to 5 — you can simply modify the conditions to create more complex conditions to specify weather a column shall be printed.

Источник

10 Practical Linux Cut Command Examples to Select File Columns

Linux command cut is used for text processing. You can use this command to extract portion of text from a file by selecting columns. This tutorial provides few practical examples of cut command that you can use in your day to day command line activities.
For most of the example, we’ll be using the following test file.

$ cat test.txt cat command for file oriented operations. cp command for copy files or directories. ls command to list out files and directories with its attributes.

1. Select Column of Characters

To extract only a desired column from a file use -c option. The following example displays 2nd character from each line of a file test.txt

2. Select Column of Characters using Range

Range of characters can also be extracted from a file by specifying start and end position delimited with -. The following example extracts first 3 characters of each line from a file called test.txt

$ cut -c1-3 test.txt cat cp ls

3. Select Column of Characters using either Start or End Position

The following specifies only the start position before the ‘-‘. This example extracts from 3rd character to end of each line from test.txt file.

$ cut -c3- test.txt t command for file oriented operations. command for copy files or directories. command to list out files and directories with its attributes.

The following specifies only the end position after the ‘-‘. This example extracts 8 characters from the beginning of each line from test.txt file.

$ cut -c-8 test.txt cat comm cp comma ls comma

The entire line would get printed when you don’t specify a number before or after the ‘-‘ as shown below.

$ cut -c- test.txt cat command for file oriented operations. cp command for copy files or directories. ls command to list out files and directories with its attributes.

4. Select a Specific Field from a File

Instead of selecting x number of characters, if you like to extract a whole field, you can combine option -f and -d. The option -f specifies which field you want to extract, and the option -d specifies what is the field delimiter that is used in the input file. The following example displays only first field of each lines from /etc/passwd file using the field delimiter : (colon). In this case, the 1st field is the username. The file

$ cut -d':' -f1 /etc/passwd root daemon bin sys sync games bala

5. Select Multiple Fields from a File

You can also extract more than one fields from a file or stdout. Below example displays username and home directory of users who has the login shell as “/bin/bash”.

$ grep "/bin/bash" /etc/passwd | cut -d':' -f1,6 root:/root bala:/home/bala

To display the range of fields specify start field and end field as shown below. In this example, we are selecting field 1 through 4, 6 and 7

$ grep "/bin/bash" /etc/passwd | cut -d':' -f1-4,6,7 root:x:0:0:/root:/bin/bash bala:x:1000:1000:/home/bala:/bin/bash

6. Select Fields Only When a Line Contains the Delimiter

In our /etc/passwd example, if you pass a different delimiter other than : (colon), cut will just display the whole line. In the following example, we’ve specified the delimiter as | (pipe), and cut command simply displays the whole line, even when it doesn’t find any line that has | (pipe) as delimiter.

$ grep "/bin/bash" /etc/passwd | cut -d'|' -f1 root:x:0:0:root:/root:/bin/bash bala:x:1000:1000:bala. /home/bala:/bin/bash

But, it is possible to filter and display only the lines that contains the specified delimiter using -s option. The following example doesn’t display any output, as the cut command didn’t find any lines that has | (pipe) as delimiter in the /etc/passwd file.

$ grep "/bin/bash" /etc/passwd | cut -d'|' -s -f1

7. Select All Fields Except the Specified Fields

In order to complement the selection field list use option –complement. The following example displays all the fields from /etc/passwd file except field 7

$ grep "/bin/bash" /etc/passwd | cut -d':' --complement -s -f7 root:x:0:0:root:/root bala:x:1000:1000:bala. /home/bala

8. Change Output Delimiter for Display

By default the output delimiter is same as input delimiter that we specify in the cut -d option. To change the output delimiter use the option –output-delimiter as shown below. In this example, the input delimiter is : (colon), but the output delimiter is # (hash).

$ grep "/bin/bash" /etc/passwd | cut -d':' -s -f1,6,7 --output-delimiter='#' root#/root#/bin/bash bala#/home/bala#/bin/bash

9. Change Output Delimiter to Newline

In this example, each and every field of the cut command output is displayed in a separate line. We still used –output-delimiter, but the value is $’\n’ which indicates that we should add a newline as the output delimiter.

$ grep bala /etc/passwd | cut -d':' -f1,6,7 --output-delimiter=$'\n' bala /home/bala /bin/bash

10. Combine Cut with Other Unix Command Output

The power of cut command can be realized when you combine it with the stdout of some other Unix command. Once you master the basic usage of cut command that we’ve explained above, you can wisely use cut command to solve lot of your text manipulation requirements. The following example indicates how you can extract only useful information from the ps command output. We also showed how we’ve filtered the output of ps command using grep and sed before the final output was given to cut command. Here, we’ve used cut option -d and -f which we’ve explained in the above examples.

$ ps axu | grep python | sed 's/\s\+/ /g' | cut -d' ' -f2,11- 2231 /usr/bin/python /usr/lib/unity-lens-video/unity-lens-video 2311 /usr/bin/python /usr/lib/unity-scope-video-remote/unity-scope-video-remote 2414 /usr/bin/python /usr/lib/ubuntuone-client/ubuntuone-syncdaemon 2463 /usr/bin/python /usr/lib/system-service/system-service-d 3274 grep --color=auto python

Источник