Linux lines to string

How to concatenate multiple lines of output to one line?

If I run the command cat file | grep pattern , I get many lines of output. How do you concatenate all lines into one line, effectively replacing each «\n» with «\» » (end with » followed by space)? cat file | grep pattern | xargs sed s/\n/ /g isn’t working for me.

By the way: (1) you need to put your sed script in single-quotes so that Bash doesn’t mess with it (since sed s/\n/ /g calls sed with two arguments, namely s/n/ and /g ); (2) since you want the output of cat file | grep pattern to be the input to sed , not the arguments to sed , you need to eliminate xargs ; and (3) there’s no need for cat here, since grep can take a filename as its second argument. So, you should have tried grep pattern file | sed ‘s/\n/ /g’ . (In this case it wouldn’t have worked, for reasons given at the above link, but now you know for the future.)

Question with 68 votes (140k views) duplicated with post which has only 1 vote (12k views)? This isn’t right.

12 Answers 12

Use tr ‘\n’ ‘ ‘ to translate all newline characters to spaces:

Note: grep reads files, cat concatenates files. Don’t cat file | grep !

tr can only handle single character translations. You could use awk to change the output record separator like:

$ grep pattern file | awk '' ORS='" ' 

This works out nicely, however I don’t want the separator to show up on the very last entry. Example, per your » , it shows up like one» two» three» , however I’d want it to show up like one» two» three . Note the last three doesn’t have a parenthesis. Any ideas?

@oink you could pipe the results into sed ‘$s/..$//’ to delete the last 2 characters on the last line.

If you want to replace newlines with nothing, you need to use the —delete option as the default expects two arguments. e.g. tr —delete ‘\n’ .

Piping output to xargs will concatenate each line of output to a single line with spaces:

Or any command, eg. ls | xargs . The default limit of xargs output is ~4096 characters, but can be increased with eg. xargs -s 8192 .

| tr ‘\n’ ‘ ‘ was not working for me when called through php exec function. It was ignoring tr, and just giving last match from grep. | xargs worked.

In bash echo without quotes remove carriage returns, tabs and multiple spaces

Severely underrated answer here, this is really simple and works like a charm, with a trailing newline as well.

Or just echo $(

Works very nicely on Unix systems — but Windows (eg. MSYS2 / Git Bash) doesn’t behave the same way — could be other systems that don’t work the same way as well. awk works most reliably for me cross-platform.

This could be what you want

cat file | grep pattern | paste -sd' ' 

As to your edit, I’m not sure what it means, perhaps this?

cat file | grep pattern | paste -sd'~' | sed -e 's/~/" "/g' 

(this assumes that ~ does not occur in file )

@Stephan there was no need to assume that cat file will actually be cat , or even a file. (I just left that part unchanged as it was irrelevant to the question)

This is an example which produces output separated by commas. You can replace the comma by whatever separator you need.

The fastest and easiest ways I know to solve this problem:

When we want to replace the new line character \n with the space:

xargs has own limits on the number of characters per line and the number of all characters combined, but we can increase them. Details can be found by running this command: xargs —show-limits and of course in the manual: man xargs

When we want to replace one character with another exactly one character:

When we want to replace one character with many characters:

First, we replace the newline characters \n for tildes ~ (or choose another unique character not present in the text), and then we replace the tilde characters with any other characters ( many_characters ) and we do it for each tilde (flag g ).

Here is another simple method using awk :

# cat > file.txt a b c # cat file.txt | awk '< printf("%s ", $0) >' a b c 

Also, if your file has columns, this gives an easy way to concatenate only certain columns:

# cat > cols.txt a b c d e f # cat cols.txt | awk '< printf("%s ", $2) >' b e 

I like the xargs solution, but if it’s important to not collapse spaces, then one might instead do:

That will replace newlines for spaces, without substituting the last line terminator like tr ‘\n’ ‘ ‘ would.

This also allows you to use other joining strings besides a space, like a comma, etc, something that xargs cannot do:

$ seq 1 5 | sed ':b;N;$!bb;s/\n/,/g' 1,2,3,4,5 

Here is the method using ex editor (part of Vim):

Join all lines and print to the standard output:

Probably the best way to do it is using ‘awk’ tool which will generate output into one line

$ awk ' /pattern/ ' ORS=' ' /path/to/file 

It will merge all lines into one with space delimiter

You don’t need since its the default action of awk.

Here’s what worked for me on mac using bash

cat file | grep pattern | paste -d' ' -s - 

-d list Use one or more of the provided characters to replace the newline characters instead of the default tab. The characters in list are used circularly, i.e., when list is exhausted the first character from list is reused. This continues until a line from the last input file (in default operation) or the last line in each file (using the -s option) is displayed, at which time paste begins selecting characters from the beginning of list again. The following special characters can also be used in list: \n newline character \t tab character \\ backslash character \0 Empty string (not a null character). Any other character preceded by a backslash is equivalent to the character itself. -s Concatenate all of the lines of each separate input file in command line order. The newline character of every line except the last line in each input file is replaced with the tab character, unless otherwise specified by the -d option. If ‘-’ is specified for one or more of the input files, the standard input is used; standard input is read one line at a time, 

circularly, for each instance of ‘-’.

Источник

Converting a multi-line string to a single string separated by commas

Solution 1 involves using the commands «sed» and «paste» on a GNU coreutils based system to add quotes and merge lines respectively. On the other hand, Solution 2 suggests splitting the string from a multy line text box into an array and then joining the list to form a string.

Turning multiple lines into one comma separated line [duplicate]

file

xargs

result

xargs improoved

cat file | xargs | sed -e 's/ /,/g' 

result

There are diverse methods to attain it, and the choice of tool typically relies on individual inclination or familiarity.

Convert text file into a comma delimited string, 5 Answers 5 · I know this is an old answer. But trying my luck here — How do I add a single quote to each string after the third step? · Do it in

Turning multi-line string into single comma-separated list in Bash

When the input is sorted based on the first column (as illustrated in your example), you have the option to pipe it to sort or utilize the awk command.

awk -F, 'NR == 1 < currentHost=$1; currentApps=$2 >NR > 1 && currentHost == $1 < currentApps=currentApps "," $2 >NR > 1 && currentHost != $1 < print currentHost ";" currentApps; currentHost=$1; currentApps=$2 >END < print currentHost ";" currentApps >' 

This approach has a benefit that sets it apart from other methods mentioned thus far. Unlike other methods that require holding the entire dataset in memory, this approach can avoid that. However, to achieve this advantage, the input data must be sorted beforehand. Otherwise, sorting the data at runtime could potentially consume a lot of memory.

• The initial line sets the values of the currentHost and currentApps variables to those of the input’s first line.
• The subsequent line deals with a host that matches the one in the previous line. It appends the application referenced in the line to the currentApps variable.
• The third line deals with a host that is different from the previous one. It prints the information for the previous host and then resets the variables to match the current line of input.
• The information about the current host is printed in the last line when the input has been fully processed.

Although there may be room for improvement (due to excessive repetition), I will defer to someone who is more proficient in awk .

$ awk ' BEGIN < FS=","; ORS="" >$1!=prev < print ors $1; prev=$1; ors=RS; OFS=";" > < print OFS $2; OFS=FS >END < print ors >' file host1;app1,app2,app3 host2;app4,app5,app6 host3;app1 

Maybe something like this:

#!/bin/bash declare -A hosts while IFS=, read host app do [ -z "$" ] && hosts["$host"]="$host;" hosts["$host"]+=$app, done < testfile printf "%s\n" "$" | sort 

The data is extracted from testfile by the script and then sent to stdout for output.

Bash turning single comma-separated column into multi-line string, With awk awk '' file. Splits the second column on commas and then for each split value,

Turning separate lines into a comma separated list with quoted entries

One way to combine quotes and merge lines is by using the sed command followed by the paste command. This allows for a streamlined process of adding quotes and merging lines.

On a GNU coreutils based operating system, such as Linux, it is unnecessary to include the trailing '-' .

To modify the command as suggested by @phk for data with DOS-style line endings, make sure your input data has such line endings.

awk 'BEGIN < ORS="" > < print p"'"'"'"$0"'"'"'"; p=", " >END < print "\n" >' /path/to/list 

awk 'BEGIN < ORS="" > < print p"\047"$0"\047"; p=", " >END < print "\n" >' /path/to/list 

'd3heatmap', 'data.table', 'ggplot2', 'htmltools', 'htmlwidgets', 'metricsgraphics', 'networkD3', 'plotly', 'reshape2', 'scales', 'stringr' 

The awk script can be written without escaping by using BEGIN < > < print p"'"$0"'"; p=", " >END < print " " >. Initially, the p variable is empty, but it gets set after the first entry is printed. This variable p is used to prefix and add single quotes around each record in awk . There is no need for the awk output record separator variable since the prefix takes care of it. It can be set to an empty value at the BEGIN step. To ensure compatibility with other text processing tools, a newline can be added at the end of the file using END . If this is not necessary, the part of the string after END (inside the single quotes) can be removed.

Before starting the pipeline, it is necessary to convert the Windows/DOS-style line endings ( ) to UNIX style ( ). To achieve this, adding tr -d ' ' at the start of the pipeline can be done.

(It is a highly secure assumption that you have no intention of utilizing in your document.)

To convert the file in-place, you can choose to run dos2unix /path/to/input.list just once.

According to the answer linked by @don_crissti, the paste function is impressively speedy due to the efficiency of linux kernel's piping. Surprisingly, using a single comma to separate list items instead of a comma followed by a space can result in a faster paste pipeline.

(paste -d\' /dev/null - /dev/null | paste -sd, -) 

The program is so fast that it outpaces even a moderately speedy flex program!

%option 8bit main fast %% .* < printf("'%s'",yytext); >\n/(.|\n)

Assuming you don't plan on running a stress test to measure any constant-factor discrepancies (which are all immediate), and you are okay with decent performance, then if you desire both versatility in your separators and concise one-liner style, this option is suitable.

Your pass to extract all information may appear like a jumbled mess, but mastering the H;1h;$!d;x technique makes it effortless to comprehend. The procedure involves s/.*/'&'/ and a slurp, ending with s/ /, /g .

With some tweaking, flex can easily outperform other tools. One way to achieve this is by informing stdio that the built-in multithread/signalhandler sync is not required.

%option 8bit main fast %% .+ < putchar_unlocked('\''); fwrite_unlocked(yytext,yyleng,1,stdout); putchar_unlocked('\''); >\n/(.|\n)

Under pressure, the speed of the paste pipelines is at least 5 times faster than conventional methods, while the speed of the process under stress is 2-3 times faster than that of the paste pipelines.

How to Join Multiple Lines Into One, -v d=”…” – we created a variable d so that we can avoid hardcoding the delimiter string in code; s=(NR==1?s:s d)$0 – we concatenate each line (

Delimit a newline string to comma separated in javascript

Employing a regular expression along with the split function offers increased resilience in comparison to solely splitting based on the pattern followed by a newline character.

var data = "1234\n5678 1235 \n8884";console.log(data.split(/\s+/).join(","));

12346 12347 12348 12342 12345 

Initially, separate the string from a multi-line text box into a single array.

var numbers = myText.split("\n"); 

After that just join list to string

 // If you want to sort: var sortedNumbers = numbers.sort() // then join with comma: var result = sortedNumbers.join(','); 

12342,12345,12346,12347,12348 

How can I split comma separated values into multiple rows?, It does not mater if comma separated values are consecutive, as long as you do not mix 2 or 3 comma on the same line.

Источник