Concatenate files on linux

How to append contents of multiple files into one file

and it did not work. I want my script to add the newline at the end of each text file. eg. Files 1.txt, 2.txt, 3.txt. Put contents of 1,2,3 in 0.txt How do I do it ?

12 Answers 12

You need the cat (short for concatenate) command, with shell redirection ( > ) into your output file

@blasto it depends. You would use >> to append one file onto another, where > overwrites the output file with whatever’s directed into it. As for the newline, is there a newline as the first character in file 1.txt ? You can find out by using od -c , and seeing if the first character is a \n .

@blasto You’re definitely heading in the right direction. Bash certainly accepts the form <. >for filename matching, so perhaps the quotes messed things up a bit in your script? I always try working with things like this using ls in a shell. When I get the command right, I just cut-n-paste it into a script as is. You might also find the -x option useful in your scripts — it will echo the expanded commands in the script before execution.

To maybe stop somebody from making the same mistake: cat 1.txt 2.txt > 1.txt will just override 1.txt with the content of 2.txt . It does not merge the two files into the first one.

Another option, for those of you who still stumble upon this post like I did, is to use find -exec :

find . -type f -name '*.txt' -exec cat <> + >> output.file 

In my case, I needed a more robust option that would look through multiple subdirectories so I chose to use find . Breaking it down:

Look within the current working directory.

Only interested in files, not directories, etc.

Whittle down the result set by name

Execute the cat command for each result. «+» means only 1 instance of cat is spawned (thx @gniourf_gniourf)

As explained in other answers, append the cat-ed contents to the end of an output file.

There are lots of flaws in this answer. First, the wildcard *.txt must be quoted (otherwise, the whole find command, as written, is useless). Another flaw comes from a gross misconception: the command that is executed is not cat >> 0.txt <> , but cat <> . Your command is in fact equivalent to < find . -type f -name *.txt -exec cat '<>‘ \; ; > >> 0.txt (I added grouping so that you realize what’s really happening). Another flaw is that find is going to find the file 0.txt , and cat will complain by saying that input file is output file.

Thanks for the corrections. My case was a little bit different and I hadn’t thought of some of those gotchas as applied to this case.

You should put >> output.file at the end of your command, so that you don’t induce anybody (including yourself) into thinking that find will execute cat <> >> output.file for every found file.

Starting to look really good! One final suggestion: use -exec cat <> + instead of -exec cat <> \; , so that only one instance of cat is spawned with several arguments ( + is specified by POSIX).

Good answer and word of warning — I modified mine to: find . -type f -exec cat <> + >> outputfile.txt and couldn’t figure out why my output file wouldn’t stop growing into the gigs even though the directory was only 50 megs. It was because I kept appending outputfile.txt to itself! So just make sure to name that file correctly or place it in another directory entirely to avoid this.

if you have a certain output type then do something like this

cat /path/to/files/*.txt >> finalout.txt 

Keep in mind that you are losing the possibility to maintain merge order though. This may affect you if you have your files named, eg. file_1 , file_2 , … file_11 , because of the natural order how files are sorted.

If all your files are named similarly you could simply do:

If all your files are in single directory you can simply do

Files 1.txt,2.txt, .. will go into 0.txt

Already answered by Eswar. Keep in mind that you are losing the possibility to maintain merge order though. This may affect you if you have your files named, eg. file_1 , file_2 , … file_11 , because of the natural order how files are sorted.

for i in ; do cat "$i.txt" >> 0.txt; done 

I found this page because I needed to join 952 files together into one. I found this to work much better if you have many files. This will do a loop for however many numbers you need and cat each one using >> to append onto the end of 0.txt.

as brought up in the comments:

sed r 1.txt 2.txt 3.txt > merge.txt 

sed h 1.txt 2.txt 3.txt > merge.txt 

sed -n p 1.txt 2.txt 3.txt > merge.txt # -n is mandatory here 

sed wmerge.txt 1.txt 2.txt 3.txt 

Note that last line write also merge.txt (not wmerge.txt !). You can use w»merge.txt» to avoid confusion with the file name, and -n for silent output.

Of course, you can also shorten the file list with wildcards. For instance, in case of numbered files as in the above examples, you can specify the range with braces in this way:

if your files contain headers and you want remove them in the output file, you can use:

for f in `ls *.txt`; do sed '2,$!d' $f >> 0.out; done 

All of the (text-) files into one

find . | xargs cat > outfile 

xargs makes the output-lines of find . the arguments of cat.

find has many options, like -name ‘*.txt’ or -type.

you should check them out if you want to use it in your pipeline

You should explain what your command does. Btw, you should use find with —print0 and xargs with -0 in order to avoid some caveats with special filenames.

If the original file contains non-printable characters, they will be lost when using the cat command. Using ‘cat -v’, the non-printables will be converted to visible character strings, but the output file would still not contain the actual non-printables characters in the original file. With a small number of files, an alternative might be to open the first file in an editor (e.g. vim) that handles non-printing characters. Then maneuver to the bottom of the file and enter «:r second_file_name». That will pull in the second file, including non-printing characters. The same could be done for additional files. When all files have been read in, enter «:w». The end result is that the first file will now contain what it did originally, plus the content of the files that were read in.

Send multi file to a file(textall.txt):

Источник

Concatenating Files in Linux

The Kubernetes ecosystem is huge and quite complex, so it’s easy to forget about costs when trying out all of the exciting tools.

To avoid overspending on your Kubernetes cluster, definitely have a look at the free K8s cost monitoring tool from the automation platform CAST AI. You can view your costs in real time, allocate them, calculate burn rates for projects, spot anomalies or spikes, and get insightful reports you can share with your team.

Connect your cluster and start monitoring your K8s costs right away:

1. Introduction

Sometimes, we need to do some operations that require using multiple files at the same time. This can be something as common as searching for some text in multiple files or merging multiple files into a new one.

In this quick tutorial, we’ll show some useful operations that can make our life easier when concatenating files in Linux.

2. The cat Command

The most frequently used command to concatenate files in Linux is probably cat, whose name comes from concatenate.

The command syntax follows the form:

In the next sections, we’ll dig deeper into the command and the options we can use.

3. Displaying a File

Let’s first go quickly over the basics of the cat command. The most straightforward operation we can do is to display a file:

This displays myfile in the standard output:

4. Creating a File

We can also use cat to create new files without a text editor.

It’s as easy as using the redirection operator:

After that, we can start typing what we want to add to the file:

When we want to save the file, we have to press CTRL+D. Notice that if the file exists, it will be overwritten.

5. Concatenating Files

One of the most common functions of the cat command is to concatenate files, as its name suggests.

The most simple concatenation is to display multiple files in the standard output:

The command above displays the files sequentially:

We can also use wildcards to display all the files that match a common pattern:

So far, we’ve been displaying the files in the standard output, but we can write the output into a new file:

Also, we can append a file to an existing file:

Another useful option is to read from the standard input, which we represent by using ‘-‘ :

Then, we can type the text we want to concatenate before file1:

Now, if we type cat file2 to display the file, we can see the text we’ve introduced concatenated with file1:

text from standard input My file 1

Also, we could append the standard input after the file instead of before:

If we go a bit further, we can also concatenate the output of any other command to cat:

Finally, we can pipe cat output to other utilities to create more powerful commands:

cat file1 file2 file3 | sort > file4

In this case, we’ve concatenated three files, sorted the result of the concatenation, and written the sorted output to a new file called file4.

6. Other Options

In the help of the cat command, we can find some other useful options that we can add to our commands:

cat --help Usage: cat [OPTION]. [FILE]. Concatenate FILE(s) to standard output. With no FILE, or when FILE is -, read standard input. -A, --show-all equivalent to -vET -b, --number-nonblank number nonempty output lines, overrides -n -e equivalent to -vE -E, --show-ends display $ at end of each line -n, --number number all output lines -s, --squeeze-blank suppress repeated empty output lines -t equivalent to -vT -T, --show-tabs display TAB characters as ^I -u (ignored) -v, --show-nonprinting use ^ and M- notation, except for LFD and TAB --help display this help and exit --version output version information and exit

For instance, we can use the -n option:

That displays the number of each line:

1 This is a test file. 2 It contains multiple lines.

Alternatively, we can use -e:

In this case, it shows a $ at the end of each line:

This is a test file.$ It contains multiple lines.$

These are just some quick examples that show how to use these options.

7. Conclusion

In this quick tutorial, we’ve shown some examples of how to use the cat command in Linux.

We covered the basics pretty quickly to focus later on the file concatenation.

And, we’ve also seen that cat can be handy when combined with other commands and can be used in many different situations.

Источник

Объединение файлов командой cat

Команду cat в большинстве случаев используют только для просмотра файлов. На самом деле одно из ее предназначений это конкатенация (объединение) файлов. А название команды происходит от слова catenate (сцепить), то есть присоединение одного после другого.

Под объединением файлов понимается их соединение. Например, если мы имеем два текстовых файла и в одном из них записана строка:
My text file 1

А в другом:
My text file 2

То после объединения мы получим файл с двумя строками:
My text file 1
My text file 2

То есть происходит простое соединение файлов. К содержимому одного добавляется содержимое другого. Это касается не только текстовых файлов, но и всех остальных (бинарных, например).

Чтобы объединить два файла командой cat нужно просто указать в качестве аргументов названия этих файлов и направить результат выполнения в новый файл. Например, мы хотим объединить два файла file1 и file2, а результат записать в новый файл file12. Тогда мы должны выполнить следующую команду:

Вы можете объединить неограниченное количество файлов. Например, чтобы объединить четыре файла и записать результат в файл myfile, выполните команду:

cat file1 file2 file3 file4 > myfile

Источник