Linux command with spaces

Bash and filenames with spaces

where listOfFiles.txt contains a list of filenames to be grepped, one filename per line. The problem occurs when listOfFiles.txt contains filenames with embedded spaces. In all cases I’ve tried (see below), Bash splits the filenames at the spaces so, for example, a line in listOfFiles.txt containing a name like ./this is a file.xml ends up trying to run grep on each piece ( ./this , is , a and file.xml ). I thought I was a relatively advanced Bash user, but I cannot find a simple magic incantation to get this to work. Here are the things I’ve tried.

grep -li 'regex' `cat listOfFiles.txt` 

Fails as described above (I didn’t really expect this to work), so I thought I’d put quotes around each filename:

grep -li 'regex' `sed -e 's/.*/"&"/' listOfFiles.txt` 

Bash interprets the quotes as part of the filename and gives «No such file or directory» for each file (and still splits the filenames with blanks)

This fails as for the original attempt (that is, it behaves as if the quotes are ignored) and is very slow since it has to launch one ‘grep’ process per file instead of processing all files in one invocation. The following works, but requires some careful double-escaping if the regular expression contains shell metacharacters:

eval grep -li 'regex' `sed -e 's/.*/"&"/' listOfFiles.txt` 

Is this the only way to construct the command line so it will correctly handle filenames with spaces?

6 Answers 6

IFS is the Internal Field Separator. Setting it to $’\n’ tells Bash to use the newline character to delimit filenames. Its default value is $’ \t\n’ and can be printed using cat -etv

Enclosing the script in parenthesis starts a subshell so that only commands within the parenthesis are affected by the custom IFS value.

Even if you don’t export it, the value will persist if this wasn’t run in a subshell. Just try running FOO=bar; echo $FOO on one line, then echo $FOO on another. Subshells are automatically started for commands in pipelines, but IFS=$’\n’ is of course not part of a pipeline here. The best solution is to surround the whole statement with parentheses, which manually tell bash to run the command in a subshell.

I still prefer running this in a subshell, but if you do want to save/restore IFS, you definitely want to quote the variable expansions.

Another way to have a variable assignment only apply temporarily is to use a space between the variable assignment and the command it applies to: IFS=$’\n’ grep -li ‘regex’ $(

@Dennis: I thought that as well, but for some reason that does not work on my machine. Perhaps that the scope of IFS=$’\n’ is so limited in this case, that it doesn’t even apply to $(

@Dennis: that doesn’t work because the environment is only set after the processing of the argument list — so ‘grep’ sees the correct value of IFS, but the shell that is processing the argument list does not.

cat listOfFiles.txt |tr '\n' '\0' |xargs -0 grep -li 'regex' 

The -0 option on xargs tells xargs to use a null character rather than white space as a filename terminator. The tr command converts the incoming newlines to a null character.

This meets the OP’s requirement that grep not be invoked multiple times. It has been my experience that for a large number of files avoiding the multiple invocations of grep improves performance considerably.

This scheme also avoids a bug in the OP’s original method because his scheme will break where listOfFiles.txt contains a number of files that would exceed the buffer size for the commands. xargs knows about the maximum command size and will invoke grep multiple times to avoid that problem.

A related problem with using xargs and grep is that grep will prefix the output with the filename when invoked with multiple files. Because xargs invokes grep with multiple files one will receive output with the filename prefixed, but not for the case of one file in listOfFiles.txt or the case of multiple invocations where the last invocation contains one filename. To achieve consistent output add /dev/null to the grep command:

cat listOfFiles.txt |tr '\n' '\0' |xargs -0 grep -i 'regex' /dev/null 

Note that was not an issue for the OP because he was using the -l option on grep; however it is likely to be an issue for others.

Источник

passing string containing spaces as command-line argument from within script

It seems this was closed as being about «a typo,» which is not the case. Here, the OP deliberately concatenated ‘ ‘ with text generated by expanding a parameter. This also isn’t a case of ‘ ‘ being used where » » is needed; both were used. I’m confident that this is a conceptual question, and I’ve expanded my answer to address it better, since it occurs to me that if close-voters did not recognize this, then novices who find this by searching may not either. This really shouldn’t have been closed and I recommend reopening it.

1 Answer 1

When you place double quotes around an expansion, such as the parameter expansion $* , the expanded text is not subject to word splitting. (That’s one of the reasons to use double quotes around $ expansions; the other is to prevent globbing.) Furthermore, inside double quotes, single quotes are not treated specially, so they do not perform quoting and they are not removed.

So, as Michael Homer says, you can just omit the spurious ‘ ‘ marks and your function should work. I suggest writing it like this:

You can use the function keyword to define functions in Bash but the syntax shown above works just as well and is portable across Bourne-style shells.

To address the conceptual issue here more directly, this line in your original code shows that you wanted to expand a parameter whose value contained spaces, in such a way that you could pass it as a single argument to git :

When you write a single argument yourself that contains spaces, you quote the spaces, usually with quotes around the whole string. The presence of ‘ ‘ inside the » » in that line tells me that you were trying to include the quotes that you would normally type.

The reason that approach does not work is that the shell itself is what interprets your quotes; they don’t typically mean anything to the command you run from the shell. Suppose you have the command:

That command does not actually pass any quotation marks to the some-command command. Instead, it runs some-command with foo bar as argument 1 and baz as argument 2. (There is also an argument 0, which tells a program how it was run; the shell passes some-command for that.)

Using quotes enables you to tell the shell where arguments begin and end. Spaces normally tell the shell to treat the text on each side as separate arguments 1 , but you want to suppress the spaces’ special meaning to the shell, which is what quoting does. When quotes are quoted, like the inner ‘ ‘ inside » » , that removes their special meaning too. Then they don’t perform quoting, but are instead passed literally to your command, as git showed in its log:

1 In the operation of the shell, spaces and tabs divide text into separate words in two related but distinct ways. First, when a command is initially parsed, unquoted spaces and tabs separate lexical tokens. Other metacharacters do this, too, but they have additional effects—for example, ; divides a line into multiple commands. Second, where parameter expansion or any other shell expansion signified by a $ 2 is performed in an unquoted context, the result is immediately subjected to word splitting, which uses the characters in $IFS as delimiters. The default value of IFS is a space followed by a tab followed by a newline.

2 Or command substitution, even if the ` ` syntax is used instead of the $( ) syntax.

Источник

How to access files/directories with spaces in the name? [duplicate]

Through terminal I can’t access files or directories with a spaces in their names. The cd command says no such file or directory . Is there any way to do it or should I rename all files with spaces?

2 Answers 2

To access a directory having space in between the name use \ to access it. You can also use Tab button to auto completion of name.

guru@guru-Aspire-5738:~$ cd /media/Data/My\ Data/ guru@guru-Aspire-5738:/media/Data/My Data$. 

To to use files with spaces you can either use the escape character or youse the double quotes.

\ is called escape character, used to not expansion of space, so now bash read the space as part of file name.

Now to rename files, it’s so easy to rename all files with spaces and replace space with underscore:

for file in * ; do mv "$f" "$" ; done 

look at answer here there is a script to rename all files and dirs recursively.

The script is:(All rights go to its owner)

#!/bin/bash # set -o xtrace # uncomment for debugging declare weirdchars=" &\'" function normalise_and_rename() < declare -a list=("$") for fileordir in "$"; do newname="$]/_>" [[ ! -a "$newname" ]] && \ mv "$fileordir" "$newname" || \ echo "Skipping existing file, $newname." done > declare -a dirs files while IFS= read -r -d '' dir; do dirs+=("$dir") done < <(find -type d -print0 | sort -z) normalise_and_rename dirs[@] while IFS= read -r -d '' file; do files+=("$file") done < <(find -type f -print0 | sort -z) normalise_and_rename files[@] 

Источник

How to reference filename with spaces in Linux

For Linux, the user terminal is the most crucial program to manage almost everything in the operating system. But many people do not prefer the command line because they assume that it is a very technical program and require particular expertise to use it. Though knowing it is pretty handy.

Various reasons keep a new Linux user away from the command line, one of them handling files in the terminal. Files and directories in Linux can have different names, and some names can have “spaces” in them. So what’s the big deal? The issue is, the terminal reads the “space” differently; for instance, if your directory has the name “new folder,” the terminal will assume two different directories, i.e., “new” and “folder.” Accessing such files in the terminal can become really frustrating. Luckily, there are various methods to deal with the files/folders with spaces in their names.

This guide is focusing on how to reference filename with spaces in Linux with different approaches. So let’s begin:

How to access files/directories in the terminal with spaces in the name:

In this section, we will be creating a file first with a “space” in its name; then, we will learn what errors we face while referring to it and then how to access it correctly in the terminal:

How to create a file and directory with space in its name:

Creating a file in Linux with “space” in its name is straightforward, open terminal, and run the command mentioned below:

The file can either be created by using apostrophes or quotation marks. The procedure of creating a directory is quite similar:

You can verify it by using the “ls” command in the terminal.

How to read a file with space in its name:

Before we learn the correct way to read a file with “space” in its name, let’s identify the error it can give. So when you try to read the above-created file (my file), you will get an error:

Now, let’s see what happens when you try to write something to the file:

As it can be seen that the above command, instead of writing the “my file,” creating a new file by the name of “my” and saving text to it. So, how to access such a file? Well, there are two approaches:

So first of all, let’s insert some text into the above-created file using ;“\”:

Источник