How do I include a pipe | in my linux find -exec command?
What the OP was trying to accomplish can be met with the suggestions above, but this is the one which actually answers the question asked. There are reasons to do it this way — exec is a lot more powerful than just operating on the files returned by find, especially when combined with test. For instance: find geda-gaf/ -type d -exec bash -c ‘DIR=<>; [[ $(find $DIR -maxdepth 1 |xargs grep -i spice |wc -l) -ge 5 ]] && echo $DIR’ \; Will return all directories in the search path which contain more than 5 lines total among all the files in that directory containing the word spice
Best answer. Grepping the whole output (as other answers suggest) isn’t the same as grep each file. Tip: instead of sh, you can use any other shell you want (I tried that with bash and it’s running ok).
Make sure to not overlook the -c option. Otherwise you will get a puzzling No such file or directory error message.
Example of finding files and renaming them with sed using regular expression find -type f -name ‘*.mdds’ -exec sh -c «echo <> | sed -e ‘s/_9\+//g’ | xargs mv <>» \;
@Roland try to double quote the brackets and single quotes on the outer scope -type f -exec sh -c ‘echo «<>«‘ | grep something\;
The job of interpreting the pipe symbol as an instruction to run multiple processes and pipe the output of one process into the input of another process is the responsibility of the shell (/bin/sh or equivalent).
In your example you can either choose to use your top level shell to perform the piping like so:
find -name 'file_*' -follow -type f -exec zcat <> \; | agrep -dEOE 'grep' In terms of efficiency this results costs one invocation of find, numerous invocations of zcat, and one invocation of agrep.
This would result in only a single agrep process being spawned which would process all the output produced by numerous invocations of zcat.
If you for some reason would like to invoke agrep multiple times, you can do:
find . -name 'file_*' -follow -type f \ -printf "zcat %p | agrep -dEOE 'grep'\n" | sh This constructs a list of commands using pipes to execute, then sends these to a new shell to actually be executed. (Omitting the final «| sh» is a nice way to debug or perform dry runs of command lines like this.)
In terms of efficiency this results costs one invocation of find, one invocation of sh, numerous invocations of zcat and numerous invocations of agrep.
The most efficient solution in terms of number of command invocations is the suggestion from Paul Tomblin:
find . -name "file_*" -follow -type f -print0 | xargs -0 zcat | agrep -dEOE 'grep' . which costs one invocation of find, one invocation of xargs, a few invocations of zcat and one invocation of agrep.
Another advantage of xargs would be, that you can speed it with modern multi core cpu even more up, by using the -P switch (-P 0).
Yes, the -P swich is indeed a nice way to speed up execution in general. Unfortunately, you run the risk of the output of parallel zcat processes being piped into agrep interleaved, which would affect the result. This effect can be demonstrated using: echo -e «1\n2» | xargs -P 0 -n 1 yes | uniq
A simpler and more general answer is at stackoverflow.com/a/21825690/42973: -exec sh -c «… | … » \; .
find . -name "file_*" -follow -type f -print0 | xargs -0 zcat | agrep -dEOE 'grep' Hoping to avoid -print and xargs for efficiency reasons. Maybe that’s really my problem: find cannot handle piped commands through -exec
This doesn’t work with files with spaces in their names; to fix, replace -print with -print0 and add the -0 option to xargs
@someguy — Wha? Avoiding xargs for efficiency reasons? Calling one instance of zcat, and passing it a list of multiple files, is far more efficient than exec-ing a new instance of it for each found file.
@Adam — I’ve made your suggested change. 99% of the time when I’m doing finds, it’s in my source code directories, and none of the files there have spaces so I don’t bother with print0. Now my documents directory, on the other hand, I remember the print0.
Would be a great answer with a little explanation about -print0 , xargs -0 , and filename with spaces.
You can also pipe to a while loop that can do multiple actions on the file which find locates. So here is one for looking in jar archives for a given java class file in folder with a large distro of jar files
find /usr/lib/eclipse/plugins -type f -name \*.jar | while read jar; do echo $jar; jar tf $jar | fgrep IObservableList ; done the key point being that the while loop contains multiple commands referencing the passed in file name separated by semicolon and these commands can include pipes. So in that example I echo the name of the matching file then list what is in the archive filtering for a given class name. The output looks like:
/usr/lib/eclipse/plugins/org.eclipse.core.contenttype.source_3.4.1.R35x_v20090826-0451.jar /usr/lib/eclipse/plugins/org.eclipse.core.databinding.observable_1.2.0.M20090902-0800.jar org/eclipse/core/databinding/observable/list/IObservableList.class /usr/lib/eclipse/plugins/org.eclipse.search.source_3.5.1.r351_v20090708-0800.jar /usr/lib/eclipse/plugins/org.eclipse.jdt.apt.core.source_3.3.202.R35x_v20091130-2300.jar /usr/lib/eclipse/plugins/org.eclipse.cvs.source_1.0.400.v201002111343.jar /usr/lib/eclipse/plugins/org.eclipse.help.appserver_3.1.400.v20090429_1800.jar
in my bash shell (xubuntu10.04/xfce) it really does make the matched classname bold as the fgrep highlights the matched string; this makes it really easy to scan down the list of hundreds of jar files that were searched and easily see any matches.
on windows you can do the same thing with:
for /R %j in (*.jar) do @echo %j & @jar tf %j | findstr IObservableList note that in that on windows the command separator is ‘&’ not ‘;’ and that the ‘@’ suppresses the echo of the command to give a tidy output just like the linux find output above; although findstr is not make the matched string bold so you have to look a bit closer at the output to see the matched class name. It turns out that the windows ‘for’ command knows quite a few tricks such as looping through text files.
Includes with the Linux GCC Linker
Does the compiler extract the appropriate binary code and insert it into the compiled executable OR does the compiler insert a reference to an external binary file (a-la Windows DLL?) I guess a generic version of this question is: Is there an equivalent concept to Windows DLLs under *nix?
5 Answers 5
Well. When you include math.h the compiler will read the file that contains declarations of the functions and macros that can be used. If you call a function declared in that file (header), then the compiler inserts a call instruction into that place in your object file that will be made from the file you compile (let’s call it test.c and the object file created test.o ). It also adds an entry into the relocation table of that object-file:
Relocation section '.rel.text' at offset 0x308 contains 1 entries: Offset Info Type Sym.Value Sym. Name 0000001c 00000902 R_386_PC32 00000000 bar This would be a relocation entry for a function bar. An entry in the symbol table will be made noting the function is yet undefined:
9: 00000000 0 NOTYPE GLOBAL DEFAULT UND bar When you link the test.o object file into a program, you need to link against the math library called libm.so . The so extension is similar to the .dll extension for windows. It means it is a shared object file. The compiler, when linking, will fix-up all the places that appear in the relocation table of test.o , replacing its entries with the proper address of the bar function. Depending on whether you use the shared version of the library or the static one (it’s called libm.a then), the compiler will do that fix-up after compiling, or later, at runtime when you actually start your program. When finished, it will inject an entry in the table of shared libraries needed for that program. (can be shown with readelf -d ./test ):
Dynamic section at offset 0x498 contains 22 entries: Tag Type Name/Value 0x00000001 (NEEDED) Shared library: [libm.so.6] 0x00000001 (NEEDED) Shared library: [libc.so.6] . . . Now, if you start your program, the dynamic linker will lookup that library, and will link that library to your executable image. In Linux, the program doing this is called ld.so . Static libraries don’t have a place in the dynamic section, as they are just linked to the other object files and then they are forgotten about; they are part of the executable from then on.
In reality it is actually much more complex and i also don’t understand this in detail. That’s the rough plan, though.
Where are include files stored — Ubuntu Linux, GCC
the compiler, GCC in my case, knows where that stdio.h (and even the object file) are located on my hard drive. It just utilizes the files with no interaction from me. I think that on my Ubuntu Linux machine the files are stored at /usr/include/ . How does the compiler know where to look for these files? Is this configurable or is this just the expected default? Where would I look for this configuration? Since I’m asking a question on these include files, what are the source of the files? I know this might be fuzzy in the Linux community but who manages these? Who would provide and manage the same files for a Windows compiler. I was always under the impression that they come with the compiler but that was an assumption.
4 Answers 4
When the include file is in brackets the preprocessor first searches in paths specified via the -I flag. Then it searches the standard include paths (see the above link, and use the -v flag to test on your system).
When the include file is in quotes the preprocessor first searches in the current directory, then paths specified by -iquote, then -I paths, then the standard paths.
-nostdinc can be used to prevent the preprocessor from searching the standard paths at all.
Environment variables can also be used to add search paths.
When compiling if you use the -v flag you can see the search paths used.
gcc is a rich and complex «orchestrating» program that calls many other programs to perform its duties. For the specific purpose of seeing where #include «goo» and #include will search on your system, I recommend:
$ touch a.c $ gcc -v -E a.c . #include ". " search starts here: #include search starts here: /usr/local/include /usr/lib/gcc/i686-apple-darwin9/4.0.1/include /usr/include /System/Library/Frameworks (framework directory) /Library/Frameworks (framework directory) End of search list. # 1 "a.c" This is one way to see the search lists for included files, including (if any) directories into which #include «. » will look but #include <. >won’t. This specific list I’m showing is actually on Mac OS X (aka Darwin) but the commands I recommend will show you the search lists (as well as interesting configuration details that I’ve replaced with . here;-) on any system on which gcc runs properly.
how to include lib when compiling C code with gcc command in linux
I want to run my C code located in desktop with the header files located in other location. What should be the appropriate GCC command for compilation and execution? I have attached the code below. I am asking kind considerations and help in this regards.
#include #endif #include #include #include #include #include #include #include #define BUFSIZE 32 int main(int argc, char*argv[]) < /* The Sample format to use */ static const pa_sample_spec ss = < .format = PA_SAMPLE_S16LE, .rate = 44100, .channels = 2 >; pa_simple *s_in, *s_out = NULL; int ret = 1; int error; /* Create a new playback stream */ if (!(s_out = pa_simple_new(NULL, argv[0], PA_STREAM_PLAYBACK, NULL, "playback", &ss, NULL, NULL, &error))) < fprintf(stderr, __FILE__": pa_simple_new() failed: %s\n", pa_strerror(error)); goto finish; >if (!(s_in = pa_simple_new(NULL, argv[0], PA_STREAM_RECORD, NULL, "record", &ss, NULL, NULL, &error))) < fprintf(stderr, __FILE__": pa_simple_new() failed: %s\n", pa_strerror(error)); goto finish; >for (;;) < uint8_t buf[BUFSIZE]; ssize_t r; #if 1 pa_usec_t latency; if ((latency = pa_simple_get_latency(s_in, &error)) == (pa_usec_t) -1) < fprintf(stderr, __FILE__": pa_simple_get_latency() failed: %s\n", pa_strerror(error)); goto finish; >fprintf(stderr, "In: %0.0f usec \r\n", (float)latency); if ((latency = pa_simple_get_latency(s_out, &error)) == (pa_usec_t) -1) < fprintf(stderr, __FILE__": pa_simple_get_latency() failed: %s\n", pa_strerror(error)); goto finish; >fprintf(stderr, "Out: %0.0f usec \r\n", (float)latency); #endif if (pa_simple_read(s_in, buf, sizeof(buf), &error) < 0) < fprintf(stderr, __FILE__": read() failed: %s\n", strerror(errno)); goto finish; >/* . and play it */ if (pa_simple_write(s_out, buf, sizeof(buf), &error) < 0) < fprintf(stderr, __FILE__": pa_simple_write() failed: %s\n", pa_strerror(error)); goto finish; >> /* Make sure that every single sample was played */ if (pa_simple_drain(s_out, &error) < 0) < fprintf(stderr, __FILE__": pa_simple_drain() failed: %s\n", pa_strerror(error)); goto finish; >ret = 0; finish: if (s_in) pa_simple_free(s_in); if (s_out) pa_simple_free(s_out); return ret; > I am new in this platform. Will you write the whole command according to my code, where the headers location is given in include directive? Please give the command for execution also.