Replace whole line containing a string using Sed
The search keyword is TEXT_TO_BE_REPLACED I need to write a shell script for this. How can I achieve this using sed ?
16 Answers 16
You can use the change command to replace the entire line, and the -i flag to make the changes in-place. For example, using GNU sed:
sed -i '/TEXT_TO_BE_REPLACED/c\This line is removed by the admin.' /tmp/foo @MarcusDowning GNU sed does not require the space; it works just fine as originally posted. If your particular sed requires the space, then by all means note which sed is incompatible and add the necessary invocation as a comment. However, please don’t change working code in an accepted answer.
How can I use a variable instead of the text «This. «? If I replace it by $variable, it does not print its content but the variable name.
There is a problem with c\ when followed directly by a variable: …c\$VAR… The backslash will escape the dollar. In this case I (bash/sed on Ubuntu 15.10) had to write …c\\$VAR…
on a mac use: sed -i » ‘/TEXT_TO_BE_REPLACED/c\This line is removed by the admin.’ /tmp/foo ; (when the first param is blank it edits in-file, otherwise creates a backup)
You need to use wildcards ( .* ) before and after to replace the whole line:
sed 's/.*TEXT_TO_BE_REPLACED.*/This line is removed by the admin./' This is working on Mac OS X Yosemite with the exception that I am using the -i and -e flags as follows: sed -i -e «s/.*search_string.*/Replacement_line/’ file_being_searched.txt
sed -i '/TEXT_TO_BE_REPLACED/c\This line is removed by the admin.' /tmp/foo Works fine if the replacement string/line is not a variable.
The issue is that on Redhat 5 the \ after the c escapes the $ . A double \\ did not work either (at least on Redhat 5).
Through hit and trial, I discovered that the \ after the c is redundant if your replacement string/line is only a single line. So I did not use \ after the c , used a variable as a single replacement line and it was joy.
The code would look something like:
sed -i "/TEXT_TO_BE_REPLACED/c $REPLACEMENT_TEXT_STRING" /tmp/foo Note the use of double quotes instead of single quotes.
This variation worked on Ubuntu/Debian: sed -i «/TEXT_TO_BE_REPLACED/c\\$REPLACEMENT_TEXT_STRING» /tmp/foo
The accepted answer did not work for me for several reasons:
- my version of sed does not like -i with a zero length extension
- the syntax of the c\ command is weird and I couldn’t get it to work
- I didn’t realize some of my issues are coming from unescaped slashes
So here is the solution I came up with which I think should work for most cases:
function escape_slashes < sed 's/\//\\\//g' >function change_line < local OLD_LINE_PATTERN=$1; shift local NEW_LINE=$1; shift local FILE=$1 local NEW=$(echo "$" | escape_slashes) # FIX: No space after the option i. sed -i.bak '/'"$"'/s/.*/'"$"'/' "$" mv "$.bak" /tmp/ > So the sample usage to fix the problem posed:
change_line "TEXT_TO_BE_REPLACED" "This line is removed by the admin." yourFile All of the answers provided so far assume that you know something about the text to be replaced which makes sense, since that’s what the OP asked. I’m providing an answer that assumes you know nothing about the text to be replaced and that there may be a separate line in the file with the same or similar content that you do not want to be replaced. Furthermore, I’m assuming you know the line number of the line to be replaced.
The following examples demonstrate the removing or changing of text by specific line numbers:
# replace line 17 with some replacement text and make changes in file (-i switch) # the "-i" switch indicates that we want to change the file. Leave it out if you'd # just like to see the potential changes output to the terminal window. # "17s" indicates that we're searching line 17 # ".*" indicates that we want to change the text of the entire line # "REPLACEMENT-TEXT" is the new text to put on that line # "PATH-TO-FILE" tells us what file to operate on sed -i '17s/.*/REPLACEMENT-TEXT/' PATH-TO-FILE # replace specific text on line 3 sed -i '3s/TEXT-TO-REPLACE/REPLACEMENT-TEXT/' for manipulation of config files
i came up with this solution inspired by skensell answer
configLine [searchPattern] [replaceLine] [filePath]
- create the file if not exists
- replace the whole line (all lines) where searchPattern matched
- add replaceLine on the end of the file if pattern was not found
function configLine < local OLD_LINE_PATTERN=$1; shift local NEW_LINE=$1; shift local FILE=$1 local NEW=$(echo "$" | sed 's/\//\\\//g') touch "$" sed -i '/'"$"'/"'/;h>;$;x>' "$" if [[ $? -ne 100 ]] && [[ $ != '' ]] then echo "$" >> "$" fi > In my makefile I use this:
@sed -i '/.*Revision:.*/c\'"`svn info -R main.cpp | awk '/^Rev/'`"'' README.md PS: DO NOT forget that the -i changes actually the text in the file. so if the pattern you defined as «Revision» will change, you will also change the pattern to replace.
Abc-Project written by John Doe
So if you set the pattern «Revision: 1190» it’s obviously not the same as you defined them as «Revision:» only.
bash-4.1$ new_db_host="DB_HOSTNAME=good replaced with 122.334.567.90" bash-4.1$ bash-4.1$ sed -i "/DB_HOST/c $new_db_host" test4sed vim test4sed ' ' ' DB_HOSTNAME=good replaced with 122.334.567.90 ' To do this without relying on any GNUisms such as -i without a parameter or c without a linebreak:
sed '/TEXT_TO_BE_REPLACED/c\ This line is removed by the admin. ' infile > tmpfile && mv tmpfile infile In this (POSIX compliant) form of the command
text can consist of one or multiple lines, and linebreaks that should become part of the replacement have to be escaped:
where s/x/y/ is a new sed command after the pattern space has been replaced by the two lines
cat find_replace | while read pattern replacement ; do sed -i "/$/c $" file done find_replace file contains 2 columns, c1 with pattern to match, c2 with replacement, the sed loop replaces each line conatining one of the pattern of variable 1
No, this is wrong on several counts. Run sed once with a script file containing all the replacements you want to perform. Running sed -i on the same file repeatedly is a horrible antipattern.
To replace whole line containing a specified string with the content of that line
Row: 0 last_time_contacted=0, display_name=Mozart, _id=100, phonebook_bucket_alt=2 Row: 1 last_time_contacted=0, display_name=Bach, _id=101, phonebook_bucket_alt=2 $ sed 's/.* display_name=\([[:alpha:]]\+\).*/\1/' output: 100 101 Multiple strings delimited by white-space:
$ sed 's/.* display_name=\([[:alpha:]]\+\).* _id=\([[:digit:]]\+\).*/\1 \2/' output: Mozart 100 Bach 101 Adjust regex to meet your needs
sed -i .bak -e '7s/.*version.*/ version = "4.33.0"/' - -i : The extension for the backup file after the replacement. In this case, it is .bak .
- -e : The sed script. In this case, it is ‘7s/.*version.*/ version = «4.33.0»/’ . If you want to use a sed file use the -f flag
- s : The line number in the file to be replaced. In this case, it is 7s which means line 7.
If you want to do a recursive find and replace with sed then you can grep to the beginning of the command:
grep -rl --exclude-dir= --include=\* "" ./ | sed -i 's/.*.*//' The question asks for solutions using sed , but if that’s not a hard requirement then there is another option which might be a wiser choice.
The accepted answer suggests sed -i and describes it as replacing the file in-place, but -i doesn’t really do that and instead does the equivalent of sed pattern file > tmp; mv tmp file , preserving ownership and modes. This is not ideal in many circumstances. In general I do not recommend running sed -i non-interactively as part of an automatic process—it’s like setting a bomb with a fuse of an unknown length. Sooner or later it will blow up on someone.
To actually edit a file «in place» and replace a line matching a pattern with some other content you would be well served to use an actual text editor. This is how it’s done with ed , the standard text editor.
printf '%s\n' '/TEXT_TO_BE_REPLACED/' d i 'This line is removed by the admin' . w q | \ ed -s /tmp/foo > /dev/null Note that this only replaces the first matching line, which is what the question implied was wanted. This is a material difference from most of the other answers.
That disadvantage aside, there are some advantages to using ed over sed :
- You can replace the match with one or multiple lines without any extra effort.
- The replacement text can be arbitrarily complex without needing any escaping to protect it.
- Most importantly, the original file is opened, modified, and saved. A copy is not made.
How it works
- printf will use its first argument as a format string and print each of its other arguments using that format, effectively meaning that each argument to printf becomes a line of output, which is all sent to ed on stdin.
- The first line is a regex pattern match which causes ed to move its notion of «the current line» forward to the first line that matches (if there is no match the current line is set to the last line of the file).
- The next is the d command which instructs ed to delete the entire current line.
- After that is the i command which puts ed into insert mode;
- after that all subsequent lines entered are written to the current line (or additional lines if there are any embedded newlines). This means you can expand a variable (e.g. «$foo» ) containing multiple lines here and it will insert all of them.
- Insert mode ends when ed sees a line consisting of .
- The w command writes the content of the file to disk, and
- the q command quits.
- The ed command is given the -s switch, putting it into silent mode so it doesn’t echo any information as it runs,
- the file to be edited is given as an argument to ed ,
- and, finally, stdout is thrown away to prevent the line matching the regex from being printed.
Some Unix-like systems may (inappropriately) ship without an ed installed, but may still ship with an ex ; if so you can simply use it instead. If have vim but no ex or ed you can use vim -e instead. If you have only standard vi but no ex or ed , complain to your sysadmin.
How can I replace a specific line by line number in a text file?
I have a 2GB text file on my linux box that I’m trying to import into my database. The problem I’m having is that the script that is processing this rdf file is choking on one line:
mismatched tag at line 25462599, column 2, byte 1455502679: =^ I want to replace the with . I can’t do a search/replace on all lines but I do have the line number so I’m hoping theres some easy way to just replace that one line with the new text. Any ideas/suggestions?
5 Answers 5
sed -i yourfile.xml -e '25462599s!!!' sed -i '25462599 s|||' nameoffile.txt The tool for editing text files in Unix, is called ed (as opposed to sed , which as the name implies is a stream editor).
ed was once intended as an interactive editor, but it can also easily scripted. The way ed works, is that all commands take an address parameter. The way to address a specific line is just the line number, and the way to change the addressed line(s) is the s command, which takes the same regexp that sed would. So, to change the 42nd line, you would write something like 42s/old/new/ .
FILENAME=/path/to/whereever LINENUMBER=25462599 ed -- "$" s!!! w q HERE The advantage of this is that ed is standardized, while the -i flag to sed is a proprietary GNU extension that is not available on a lot of systems.
Using sed replace line in file with another file
I have a very large tab delimited file, I would like to replace a single line in this file with another. As the line has >100 columns, a simple sed ‘s/find/replace/’ is not desirable. My newline is stored in file newline.txt How do I achieve:
sed 's/find/newline.txt/' infile Not sure I understand, if you find a certain pattern in your original file then you would like the entire line to be replaced by a corresponding line in newline.txt? Does this mean the pattern is unique and only returns one line?
4 Answers 4
Find line in file file.csv which contains find , append content ( r ) of file newline.txt and delete ( d ) line which contains find :
Based on GNU sed 4.2.2, also includes answers from Cyrus and Aaron
$ cat foo.txt 1 abc 2 ijk! 3 pqr 4 xyz $ cat f1.txt a/b/c $ cat f2.txt line $ cat f3.txt line a line b 1) Pattern and replacement not containing characters that’ll affect sed command or act weirdly due to bash substitution inside double quotes
$ sed "/3/c $(< f2.txt)" foo.txt 1 abc 2 ijk! line 4 xyz $ sed "s/.*3.*/$(< f2.txt)/" foo.txt 1 abc 2 ijk! line 4 xyz $ sed -e '/3/' foo.txt 1 abc 2 ijk! line 4 xyz 2) Pattern getting affected due to bash substitution
$ sed "/!/c $(< f2.txt)" foo.txt bash: !/c: event not found $ sed '/!/c '"$(< f2.txt)" foo.txt 1 abc line 3 pqr 4 xyz $ sed "s/.*!.*/$(< f2.txt)/" foo.txt bash: !.*/$: event not found $ sed 's/.*!.*/'"$(< f2.txt)/" foo.txt 1 abc line 3 pqr 4 xyz $ sed -e '/!/' foo.txt 1 abc line 3 pqr 4 xyz 3) Replacement line (single line only) containing characters affecting sed
$ sed "/3/c $(< f1.txt)" foo.txt 1 abc 2 ijk! a/b/c 4 xyz $ sed "s/.*3.*/$(< f1.txt)/" foo.txt sed: -e expression #1, char 11: unknown option to `s' $ sed "s|.*3.*|$(< f1.txt)|" foo.txt 1 abc 2 ijk! a/b/c 4 xyz $ sed -e '/3/' foo.txt 1 abc 2 ijk! a/b/c 4 xyz 4) Replacement with multiple lines
$ sed "/3/c $(< f3.txt)" foo.txt sed: -e expression #1, char 14: extra characters after command $ sed "s/.*3.*/$(< f3.txt)/" foo.txt sed: -e expression #1, char 14: unterminated `s' command $ sed -e '/3/' foo.txt 1 abc 2 ijk! line a line b 4 xyz