Linux cat all but first line

get first X characters from the cat command?

I have a text file I’m outputting to a variable in my shell script. I only need the first 50 characters however. I’ve tried using cat $ cut -c1-50 but I’m getting far more than the first 50 characters? That may be due to cut looking for lines (not 100% sure), while this text file could be one long string— it really depends. Is there a utility out there I can pipe into to get the first X characters from a cat command?

8 Answers 8

This returns the first 50 bytes.

Mind that the command is not always implemented the same on all OS. On Linux and macOS it behaves this way. On Solaris (11) you need to use the gnu version in /usr/gnu/bin/

Note that this answer assumes that the file contains only ASCII characters, as the OP asked for the first X characters, not bytes.

POSIX doesn’t define -c as a valid option, however, so it is definitely dependent on your local environment. unix.com/man-page/posix/1/head

@Calimo Yes, I know, but I tried making a text file with 100 characters then running my command and it printed 50 characters. But you’re right about ASCII, but since OP flagged this as answered there were none in his case.

Your cut command works if you use a pipe to pass data to it:

Or, avoiding a useless use of cat and making it a little safer:

Note that the commands above will print the first 50 characters (or bytes, depending on your cut implementation) of each input line. It should do what you expect if, as you say, your file is one huge line.

dd status=none bs=1 count=50 if=$

This returns the first 50 bytes.

dd has no status=none flag. Use 2>/dev/null instead (and quote properly): dd if=»$filename» bs=1 count=50 2>/dev/null (even so, consider using bs=50 count=1 to reduce the number of syscalls involved).

@mirabilos dd does have status=none when using Ubuntu 14.04, coreutils 8.21, but you’re right to use 2>/dev/null if using a earlier version.

There is no GNU coreutils (nor does dd(1) have a version) on most Unix systems, which is why I urge you to use the portable version. After all, this is not the “Ask Ubuntu” SO site, but the “Unix & Linux” one.

@mirabilos Most Linux distros use GNU coreutils as does FreeBSD and other BSDs. It is available on Solaris as package gnu-coreutils. Yes, this is «Unix & Linux» and both Unix and Linux systems use GNU coreutils.

No, Unix systems do not generally use GNU utilities. GNU is an acronym for “GNU is not Unix”, even. Please stick to portable solutions, or, if you must give GNU-only solutions, state so, and, if at all possible, show an equivalent portable solution.

Most answers so far assume that 1 byte = 1 character, which may not be the case if you are using a non-ASCII locale.

A slightly more robust way to do it:

testString=$(head -c 200 < "$") && printf '%s\n' "$" 

1. You are using ksh93 , bash (or a recent zsh or mksh (though the only multi-byte charset supported by mksh is UTF-8 and only after set -o utf8-mode )) and a version of head that supports -c (most do nowadays, but not strictly standard).
2. The current locale is set to the same encoding as the file (type locale charmap and file -- "$filename" to check that); if not, set it with ie. LC_ALL=en_US.UTF-8 )
3. I took the first 200 bytes of the file with head , assuming the worst-case UTF-8 where all the characters are encoded on at most 4 bytes. This should cover most cases I can think of.

Of course, this also assumes GNU head , or another implementation of it which adds the nōn-standard -c option. But you’re requiring GNU bash already. (Note: mksh ’s UTF-8 mode could do this for UTF-8 encoded files.) I’d ask the OP if they require octets or multibyte characters, just “characters” is a vague/gerneric term.

That also assumes $filename or $testString doesn't contain blank newline or wildcards or start with - .

The $ construct you're using here actually comes from ksh93 and is also supported by recent versions of zsh ( zsh has its own $testString[1,50] ). You need $ in ksh93 and zsh however.

Other variant (for first line in file)

This is abuse of high-level tools – and prone to not doing what you want, e.g. if they’re locale-aware.

grep (regexp), and yes, the use of shell here (hint: the first line may be large). (That being said, the bashism is also not in POSIX, but most shells implement that.)

1. For ASCII files, do like @DisplayName says:

will print out the first 50 chars of file.txt, for example.

2. For binary data, use hexdump to print it out as hex chars:

will print out the first 50 bytes of file.bin, for example.

Note that without the -v verbose option, hexdump would replace repeated lines with an asterisk ( * ) instead. See here: https://superuser.com/questions/494245/what-does-an-asterisk-mean-in-hexdump-output/494613#494613.

To read and output 50 characters (not bytes), with zsh , you can do:

If the input contains sequences of bytes that don't form valid characters in the current locale, each of those bytes will be counted as one character.

• -e : echoes what is read instead of storing it in a variable:
• -k50 : reads 50 characters. read -k was initially meant for reading key presses on the terminal (and would put the terminal in the correct mode to get one keypress at a time), but when used with -u , it reads characters from the corresponding file descriptor instead.
• -u0 reads those characters from file descriptor 0 (stdin) which here we redirect from the file.

You can use sed for this which will tackle the problem pretty easily

-E allows us to use Extended regular expressions, instead of basic regular expressions, so we don't have to use backslashes to escape the more advanced regular expression operators.

s/x/y/ substitutes x with y in each line, where x is a regular expression and y is an expression which can contain literal values or references to capture groups.

^(.) matches up to the first 50 characters of each line and marks it as a capture group.

.* matches the rest of the line (if there were more than 50 characters), since we want to replace the whole thing.

\1 is a backreference referring to the first capture group.

Источник

Unix cat starting from line

What is the best way to output from a file starting from a specific line (big number like 70000). Something like:

5 Answers 5

Take a look at tail, more precisecly, it's --lines=+N switch:

Wow. I didn't know this even after using this on linux for 8 years ! I always used a bash fn ! Thanks ! getFromLine () < lineno= wc -l $1 | awk '' ; lineno= expr $lineno - $2 ; tail -n $lineno $1 ; >

As a note, this does not work on Mountain Lion (Darwin Kernel Version 13.1.0). The variant for Mountain lion is tail -n

The most obvious way is tail . The syntax might be slightly different depending on what OS you are using:

If you can not get tail to work, you could use sed , but it might end up slower:

tail worked just fine in MinGW (on a 600 MB text file). The runtime was only a few seconds (but the input file could have been in the file cache already).

You can use NR parameter with the awk command:

You can use this command with other limits. As a sample: cat messages | awk '=7000 && NR <7003) print>' shows you row 7000, 7001 and 7002 only.

If instead of a line number you need to start listing at the line containing a given $phrase , try the following.

more -1000 +/"$phrase" yourfilename | sed '1,4d' 

The -1000 will continuously list text for up to 1000 lines; you can change this as needed. The sed command will chop off the first 4 lines of output, which were automatically inserted by more , containing a blank line, the message ". skipping", and the two lines preceding your intended starting line. I guess this may vary depending on your system.

Источник

With the Linux "cat" command, how do I show only certain lines by number

If I use cat -n text.txt to automatically number the lines, how do I then use the command to show only certain numbered lines.

6 Answers 6

$ cat file Line 1 Line 2 Line 3 Line 4 Line 5 Line 6 Line 7 Line 8 Line 9 Line 10 

To print multiple lines (5 & 8)

$ sed -n -e 5p -e 8p file Line 5 Line 8 

To print specific range (5 - 8)

$ sed -n 5,8p file Line 5 Line 6 Line 7 Line 8 

To print range with other specific line (5 - 8 & 10)

$ sed -n -e 5,8p -e 10p file Line 5 Line 6 Line 7 Line 8 Line 10 

Note that for a range including the first line (for example first 5 lines), you'll use sed -n 1,5p . not sed -n 0,5p . . At least for the version of sed installed by default on Mac OS 10.15.

One way of doing it is by using sed :

where 11 is the number of the line you want removed.

And cat -n isn't even needed:

You can use awk straight up.

replacing '1' with the desired line number.

I concur that this is the very best answer. To add on top you can use equivalence operators to filter data out ie. awk 'NR>=20' file.txt

Depending on goals I like head or grep

cat /var/log/syslog -n | head -n 50 | tail -n 10

will return lines 41 thru 50.

cat /var/log/syslog -n | grep " 50" -b10 -a10

will show lines 40 thru 60. The problem with the grep method is that you have to use account for padding of the line numbers (notice the space)

Both are quite handy for parsing log files.

well yeah. but. but. . There are better ways. The question asked about using cat though, so I used it.

As others have shown you, there is no need to use cat -n . Other programs will do it for you. If, however, you really need to parse the output of cat -n and show only specific lines (for example, 4-8, 12 and 42), you could do:

In awk , $1 is the first field, so this command prints all lines whose first fields are i) between 4 and 8 (inclusive) or ii) 12 or iii) 42.

If you also want to remove the field added by cat -n to get the original lines from the file, you can do:

$ cat -n file | awk '$1>=4 && $1' Line 4 Line 5 Line 6 Line 7 Line 8 Line 12 Line 42 

Источник

How to read first and last line from cat output?

When reading from stdin if would look like this (for example ps -ef ):

ps -ef | sed -e 1b -e '$!d' UID PID PPID C STIME TTY TIME CMD root 1931 1837 0 20:05 pts/0 00:00:00 sed -e 1b -e $!d 

head & tail Solution:

When data is coming from a command ( ps -ef ):

ps -ef 2>&1 | (head -n1 && tail -n1) UID PID PPID C STIME TTY TIME CMD root 2068 1837 0 20:13 pts/0 00:00:00 -bash 

awk Solution:

And also the piped example with ps -ef :

ps -ef | awk 'NR==1; END' UID PID PPID C STIME TTY TIME CMD root 1935 1837 0 20:07 pts/0 00:00:00 awk NR==1; END

Thank you! It's the best answer because i cannot do (head -n1 file;tail -n1 file) i have very big command and pipe as last symbol. So | sed '1p;$!d' shorter one.

Sorry for my English, if you don't understand me - it's my problem - tell me about it and i prepare better presentation for you.

@DavidConrad, to elaborate on what chaos said, -e 1b -- on the first line, branch to the end of the sed script, at which point the implicit "print" happens. If the input is only one line long, sed ends. Otherwise, for all lines except the last, delete. On the last line, the implicit "print" happens again.

@mikeserv: 😉 check it when you get a chance. On a single line it efectively prevents doubling it on output, but on a multiple line work-case, it just preserves the last line. at least on GNU sed 4.2.2. Cheers.

sed -n '1p;$p' file.txt will print 1st and last line of file.txt .

Note that if the input has only one line, it will be printed twice. You may prefer sed -e 1b -e '$!d' if you don't want that.

cb() < (($1-1>0)) && unset "ary[$1-1]"; > mapfile -t -C cb -c 1 ary < file 

After this, you'll have an array ary with first field (i.e., with index 0 ) being the first line of file , and its last field being the last line of file . The callback cb (optional if you want to slurp all lines in the array) unsets all the intermediate lines so as to not clutter memory. As a free by-product, you'll also have the number of lines in the file (as the last index of the array+1).

$ mapfile -t -C cb -c 1 ary < <(printf '%s\n' ) $ declare -p ary declare -a ary='([0]="a" [25]="z")' $ # With only one line $ mapfile -t -C cb -c 1 ary < <(printf '%s\n' "only one line") $ declare -p ary declare -a ary='([0]="only one line")' $ # With an empty file $ mapfile -t -C cb -c 1 ary < <(:) declare -a ary='()' 

Источник