Linux concat text files

Concatenate multiple files but include filename as section headers

I would like to concatenate a number of text files into one large file in terminal. I know I can do this using the cat command. However, I would like the filename of each file to precede the «data dump» for that file. Anyone know how to do this? what I currently have:

file1.txt = bluemoongoodbeer file2.txt = awesomepossum file3.txt = hownowbrowncow cat file1.txt file2.txt file3.txt 

file1 bluemoongoodbeer file2 awesomepossum file3 hownowbrowncow 

i am actually using uuencode & uudecode to do something similar, i do not need the file readable inbetween, only need the result and pipe them to another command again

21 Answers 21

Was looking for the same thing, and found this to suggest:

tail -n +1 file1.txt file2.txt file3.txt 

==> file1.txt ==> file2.txt ==> file3.txt

If there is only a single file then the header will not be printed. If using GNU utils, you can use -v to always print a header.

Works great with both BSD tail and GNU tail on MacOS X. You can leave out the space between -n and +1 , as in -n+1 .

works on MacOsX 10.14.4 sudo ulimit -n 1024; find -f . -name «*.rb» | xargs tail -n+1 > ./source_ruby.txt

I used grep for something similar:

It does not give you a ‘header’, but prefixes every line with the filename.

+1 for showing me a new use for grep. this met my needs perfectly. in my case, each file only contained one line, so it gave me a neatly formatted output that was easily parsable

grep will only print file headers if there is more than one file. If you want to make sure to print the file path always, use -H . If you don’t want the headers use -h . Also note it will print (standard input) for STDIN.

You can also use ag (the silver searcher): by default, ag . *.txt prefixes each file with its name, and each line with its number.

Of note, passing -n to grep also yields line numbers which allows you to write simple linters with pinpointing that could be picked up by, e.g., emacs.

This should do the trick as well:

$ find . -type f -print -exec cat <> \; ./file1.txt Content of file1.txt ./file2.txt Content of file2.txt 

Here is the explanation for the command-line arguments:

find = linux `find` command finds filenames, see `man find` for more info . = in current directory -type f = only files, not directories -print = show found file -exec = additionally execute another linux command cat = linux `cat` command, see `man cat`, displays file contents <> = placeholder for the currently found filename \; = tell `find` command that it ends now here 

You further can combine searches trough boolean operators like -and or -or . find -ls is nice, too.

Источник

Concatenate Files in Linux Command Line

Want to concatenate two files in the Linux command line? Here’s how to do that with the cat command.

In simple terms, concatenation refers to connecting two character stings.

For example, there are two character strings: snow and ball . So when you’d concatenate them, it will form a single string snowball .

While using Linux, you’d often find yourself in a situation where you have to concatenate two or more files.

So in this guide, I will share multiple examples to concatenate files.

How to concatenate files in Linux

When you consider the concatenation of text files in Linux, the cat command rules the whole category.

For demonstration, I will be using two text files named TextFile1 and TextFile2 and their file contains are as follows:

[email protected]:~$ cat TextFile1 File contents of TextFile1 [email protected]:~$ cat TextFile2 File contents of TextFile2

Now, let’s jump to the examples part.

Concatenate multiple files in standard output

This is the simplest concatenation that is used by most of the users where you combine the output of multiple files in the standard output:

Concatenate and save file contents to a new file

Above, I explained how you can concatenate multiple files but it won’t save any changes.

So if you want to concatenate files and save the output to another file, you can redirect the output data stream to a new file.

For example, here, I have concatenated the file contents of TextFile1 and TextFile2 and saved the output to a new file named Output_File :

cat TextFile1 TextFile2 > Output_File

But it will override the file contents of Ouput_File (if any) and you may lose sensitive data.

To concatenate and save the output to another file without overriding, use the >> instead of > :

cat TextFile1 TextFile2 >> Output_File

And as you can see, the contamination of two files was saved below the content of Output_File .

Add additional data from standard input while concatenation

It’s not always the case where you just want to concatenate the file contents of two different files, you may also want to add additional data to it.

So let’s say I want to concatenate two files TextFile1 and TextFile2 but I want to add some additional data and in that case, using the standard input is the best choice.

Which can be invoked by the — flag.

For example, I want to add data above the content of TextFile1 , so I will be using the following:

cat - TextFile1 >> TextFile2

It will be executed as a normal cat command where you add simple text.

Once you are done adding text, save the file and exit from using the Ctrl + D .

For your reference I added the following line:

Text from the standard output that should be placed above the file contents of TextFile1

And as you can see, here, I concatenated two files with additional data from the standard input.

Similarly, if you want to add additional data after the contamination of two files instead of the above, you can refer to the following:

cat TextFile1 - >> TextFile2

Here, I have entered the following in the standard input:

Content from standard input that will be placed after contents of two files

Concatenate command output to file

If you want to concatenate the output of a command to a specific file using the cat command, you can refer to the given command syntax:

For example, here, I have concatenated the kernel version that I’m currently using to a file named Kernels :

But did you know that you can also redirect the output, errors, and standard input to a file?

It is quite easy and can be very helpful in cases where you just want to concatenate the errors only:

Get more out of the cat command

While most of the users use the cat command to create and read files but what if I tell you that you can do a lot more with it?

Even I wasn’t aware of its power until I read the following guide:

I hope you will find this guide helpful.

And if you have any queries, feel free to ask in the comments.

Источник

Concatenating multiple text files into a single file in Bash

What is the quickest and most pragmatic way to combine all *.txt file in a directory into one large text file? Currently I’m using windows with cygwin so I have access to BASH. Windows shell command would be nice too but I doubt there is one.

12 Answers 12

This appends the output to all.txt

you may run into a problem where it cats all.txt into all.txt. I have this problem with grep sometimes, not sure if cat has the same behavior.

@rmeador yes, that is true, if all.txt already exists you will have this problem. This problem is solved by providing the output file with a different extension, or moving all.txt to a different folder.

Just remember, for all the solutions given so far, the shell decides the order in which the files are concatenated. For Bash, IIRC, that’s alphabetical order. If the order is important, you should either name the files appropriately (01file.txt, 02file.txt, etc. ) or specify each file in the order you want it concatenated.

$ cat file1 file2 file3 file4 file5 file6 > out.txt 

The Windows shell command type can do this:

Type type command also writes file names to stderr, which are not captured by the > redirect operator (but will show up on the console).

Just be aware that if you put the output file in the same directory as the original file it will cause a duplication because it will also combine the new output file twice.

You can use Windows shell copy to concatenate files.

To append files, specify a single file for destination, but multiple files for source (using wildcards or file1+file2+file3 format).

This as the IMHO cleanest solution with basically no side effects that beginners could trip over unfortunately does not get appreciated enough 🙁

Worked pretty well, except at the very end of my file I got a weird SUB special unicode character. Deleting it is pretty easy programmatically but not sure why that happened.

Be careful, because none of these methods work with a large number of files. Personally, I used this line:

for i in $(ls | grep ".txt");do cat $i >> output.txt;done 

EDIT: As someone said in the comments, you can replace $(ls | grep «.txt») with $(ls *.txt)

EDIT: thanks to @gnourf_gnourf expertise, the use of glob is the correct way to iterate over files in a directory. Consequently, blasphemous expressions like $(ls | grep «.txt») must be replaced by *.txt (see the article here).

Good Solution

for i in *.txt;do cat $i >> output.txt;done 

Mandatory ParsingLs link, together with a downvote (and you deserve more than one downvote, because ls | grep is a seriously bad antipattern).

Got an upvote from me because it allows for arbitrary testing/ operations by file name prior to output and it’s quick and easy and good for practice. (In my case I wanted: for i in *; do echo -e «\n$i:\n»; cat $1; done )

find . -type f -name '*.txt' -exec cat <> + >> output.txt 

Since OP says the files are in the same directory, you may need to add -maxdepth 1 to the find command.

This should be the correct answer. It will work properly in a shell script. Here is a similar method if you want output sorted: sort -u —output=»$OUTPUT_FILE» —files0-from=- < <(find "$DIRECTORY_NAME" -maxdepth 1 -type f -name '*.txt' -print0)

This is a very flexible approach relying on all the strengths of the find . My favourite! Surely cat *.txt > all.txt does the job within the same directory (as pointed out above). To me, however, becoming comfortably fluent in using find has been a very good habit. Today they’re all in one folder, tomorrow they have multiple file-endings across nested directory hierarchies. Don’t overcomplicate, but also, do make friends with find . 🙂

Источник

How to merge all (text) files in a directory into one?

This is technically what cat («concatenate») is supposed to do, even though most people just use it for outputting files to stdout. If you give it multiple filenames it will output them all sequentially, and then you can redirect that into a new file; in the case of all files just use ./* (or /path/to/directory/* if you’re not in the directory already) and your shell will expand it to all the filenames (excluding hidden ones by default).

Make sure you don’t use the csh or tcsh shells for that which expand the glob after opening the merged-file for output, and that merged-file doesn’t exist before hand, or you’ll likely end up with an infinite loop that fills up the filesystem.

The list of files is sorted lexically. If using zsh , you can change the order (to numeric, or by age, size. ) with glob qualifiers.

To include files in sub-directories, use:

find . ! -path ./merged-file -type f -exec cat <> + > merged-file 

Though beware the list of files is not sorted and hidden files are included. -type f here restricts to regular files only as it’s unlikely you’ll want to include other types of files. With GNU find , you can change it to -xtype f to also include symlinks to regular files.

Would do the same ( (-.) achieving the equivalent of -xtype f ) but give you a sorted list and exclude hidden files (add the D qualifier to bring them back). zargs can be used there to work around argument list too long errors.

Источник