Delete the last character of a string using string manipulation in shell script
Notice that for older bash ( for example, bash 3.2.5 on OS X), you should leave spaces between and after colons:
This works for bash version 4.2-alpha and above, too bad the version I have access to is earlier. :-/
@iamaziz: From bash changelog, the negative length in $
In a POSIX shell, the syntax $ means something different — it expands to the value of t if t is set and non null, and otherwise to the value 2 . To trim a single character by parameter expansion, the syntax you probably want is $
Note that in ksh93 , bash or zsh , $ or $ (note the space) are legal as a substring expansion but are probably not what you want, since they return the substring starting at a position 2 characters in from the end (i.e. it removes the first character i of the string ijk ).
See the Shell Parameter Expansion section of the Bash Reference Manual for more info:
@afraisse $
Using sed it should be as fast as
Your single echo is then echo ljk | sed ‘s/.$//’ .
Using this, the 1-line string could be any size.
Note that in the general case, it doesn’t delete the last character of the string, but the last character of every line of the string.
Yes, \n is a character, @LuisA.Florit 🙂 You can amend the regex easily if you know more about your input, for example /.\n$/\n/ in your case to keep the last newline and remove the character before the last newline. I think even /.\n?$/\n/ might be an option to ensure a new line at the end of stream.
for removing the last n characters from a line that makes no use of sed OR awk :
> echo lkj | rev | cut -c (n+1)- | rev so for example you can delete the last character one character using this:
> echo lkj | rev | cut -c 2- | rev > lk DESCRIPTION
The rev utility copies the specified files to the standard output, reversing the order of characters in every line. If no files are speci- fied, the standard input is read.
if you don’t know the length of the string, try:
A few options depending on the shell:
- POSIX: t=$
- Bourne: t=`expr » $t» : ‘ \(.*\).’`
- zsh/yash: t=$
- bash/zsh: t=$
- ksh93/bash/zsh/mksh: t=$-1>
- ksh93/bash/zsh/mksh: t=$
- ksh93: t=$
- es: @ ~~ $t *?
Note that while all are supposed to strip the last character, you’ll find that some implementations (those that don’t support multi-byte characters) strip the last byte instead (so would likely corrupt the last character if it was multi-byte).
The expr variant assumes $t doesn’t end in more than one newline character. It will also return a non-zero exit status if the resulting string ends up being 0 (or 000 or even -0 with some implementations). It could also give unexpected results if the string contains invalid characters.
Nice and thorough! But. I assume all of those shells support POSIX, so everyone should just use that one to be the most portable. Smallest character count, too!
@Russ, t=$ is not Bourne but you’re not likely to come across a Bourne shell nowadays. $ does work in all the other ones though.
@rien333. I’d wait for the interface to stabilize a bit. fish is work in progress. 2.3.0 which introduced the string builtin was not released at the time of the Q&A. With the version I’m testing it on, you need string replace -r ‘(?s).\z’ » — $t (and I’d expect they’d want to change that, they should change the flags they pass to PCRE) or more convoluted ones. It also deals poorly with newline characters, and I know they’re planning on changing that as well.
@Timo, not sure what you mean. None of the solutions I gave are bash-specific. The only one invented by bash AFAIK is t=$
The most portable, and shortest, answer is almost certainly:
This works in bash, sh, ash, dash, busybox/ash, zsh, ksh, etc.
It works by using old-school shell parameter expansion. Specifically, the % specifies to remove the smallest matching suffix of parameter t that matches the glob pattern ? (ie: any character).
See «Remove Smallest Suffix Pattern» here for a (much) more detailed explanation and more background. Also see the docs for your shell (eg: man bash ) under «parameter expansion».
As a side note, if you wanted to remove the first character instead, you would use $ , since # matches from the front of the string (prefix) instead of the back (suffix).
Also worth noting is that both % and # have %% and ## versions, which match the longest version of the given pattern instead of the shortest. Both $ and $ would do the same as their single operator in this case, though (so don’t add the useless extra character). This is because the given ? pattern only matches a single character. Mix in a * with some non-wildcards and things get more interesting with %% and ## .
Understanding parameter expansions, or at least knowing about their existence and knowing how to look them up, is incredibly useful for writing and deciphering shell scripts of many flavors. Parameter expansions often look like arcane shell voodoo to many people because. well. they are arcane shell voodoo (although pretty well documented if you know to look for «parameter expansion»). Definitely good to have in the tool belt when you’re stuck in a shell, though.
strip the last and first character from a String
Is fairly easy to strip the first and last character from a string using awk / sed ? Say I have this string ( 1 2 3 4 5 6 7 ) I would like to strip parentheses from it. How should I do this?
4 Answers 4
$ echo '( 1 2 3 4 5 6 7 )' | sed 's/^.\(.*\).$/\1/' 1 2 3 4 5 6 7 $ echo '( 1 2 3 4 5 6 7 )' | awk '' 1 2 3 4 5 6 7 $ var='( 1 2 3 4 5 6 7 )'; var="$"; var="$"; echo "$var" 1 2 3 4 5 6 7 $ var='( 1 2 3 4 5 6 7 )'; echo "$" 1 2 3 4 5 6 7 If you use bash then use the bash way.
If not, prefer the posix-sh way. It is faster than loading sed or awk .
Other than that, you may also be doing other text processing, that you can combine with this, so depending on the rest of the script you may benefit using sed or awk in the end.
why doesn’t this work? sed ‘..’ s_res.temp > s_res.temp ?
This does not work, as the redirection > will truncate the file before it is read. To solve this you have some choices:
- what you really want to do is edit the file. sed is a stream editor not a file editor.
ed though, is a file editor (the standard one too!). So, use ed :
$ printf '%s\n' "%s/^.\(.*\).$/\1/" "." "wq" | ed s_res.temp $ sed 's/^.\(.*\).$/\1/' s_res.temp > s_res.temp.temp $ mv s_res.temp.temp s_res.temp $ (rm test; sed 's/XXX/printf/' > test) < test On Mac OS X (latest version 10.12 - Sierra) bash is stuck to version 3.2.57 which is quite old. One can always install bash using brew and get version 4.x which includes the substitutions needed for the above to work.
There is a collection of bash versions and respective changes, compiled on the bash-hackers wiki
How can I remove the last character of a file in unix?
How can I remove only the last character (the e, not the newline or null) of the file without making the text file invalid?
Listing a bunch of garbage sed and awk commands that strip the last character off of every line didn't feel terribly constructive. Heh, knew I was going to get dinged for that one. Still, couldn't bring myself to leave in the sentence "I tried a bunch of sed and awk, but could only strip out every line's last char in a variety of ways".
9 Answers 9
A simpler approach (outputs to stdout, doesn't update the input file):
- $ is a Sed address that matches the last input line only, thus causing the following function call ( s/.$// ) to be executed on the last line only.
- s/.$// replaces the last character on the (in this case last) line with an empty string; i.e., effectively removes the last char. (before the newline) on the line.
. matches any character on the line, and following it with $ anchors the match to the end of the line; note how the use of $ in this regular expression is conceptually related, but technically distinct from the previous use of $ as a Sed address. - Example with stdin input (assumes Bash, Ksh, or Zsh):
To update the input file too (do not use if the input file is a symlink):
- On macOS, you'd have to use -i '' instead of just -i ; for an overview of the pitfalls associated with -i , see the bottom half of this answer.
- If you need to process very large input files and/or performance / disk usage are a concern and you're using GNU utilities (Linux), see ImHere's helpful answer.
Removes one (-1) character from the end of the same file. Exactly as a >> will append to the same file.
The problem with this approach is that it doesn't retain a trailing newline if it existed.
if [ -n "$(tail -c1 file)" ] # if the file has not a trailing new line. then truncate -s-1 file # remove one char as the question request. else truncate -s-2 file # remove the last two characters echo "" >> file # add the trailing new line back fi This works because tail takes the last byte (not char).
It takes almost no time even with big files.
Why not sed
The problem with a sed solution like sed '$ s/.$//' file is that it reads the whole file first (taking a long time with large files), then you need a temporary file (of the same size as the original):
sed '$ s/.$//' file > tempfile rm file; mv tempfile file And then move the tempfile to replace the file.