Where do executables look for shared objects at runtime?
I understand how to define include shared objects at linking/compile time. However, I still wonder how do executables look for the shared object ( *.so libraries) at execution time. For instance, my app a.out calls functions defined in the lib.so library. After compiling, I move lib.so to a new directory in my $HOME . How can I tell a.out to go look for it there?
A quick solution is to use g++ -Wl,-R. this will force loader to look into the current folder for so libraries. Or -Wl,-R$HOME/path to specidy some other fixed folder.
4 Answers 4
The shared library HOWTO explains most of the mechanisms involved, and the dynamic loader manual goes into more detail. Each unix variant has its own way, but most use the same executable format (ELF) and have similar dynamic linkers¹ (derived from Solaris). Below I’ll summarize the common behavior with a focus on Linux; check your system’s manuals for the complete story.
(Terminology note: the part of the system that loads shared libraries is often called “dynamic linker”, but sometimes “dynamic loader” to be more precise. “Dynamic linker” can also mean the tool that generates instructions for the dynamic loader when compiling a program, or the combination of the compile-time tool and the run-time loader. In this answer, “linker” refers to the run-time part.)
In a nutshell, when it’s looking for a dynamic library ( .so file) the linker tries:
- directories listed in the LD_LIBRARY_PATH environment variable ( DYLD_LIBRARY_PATH on OSX);
- directories listed in the executable’s rpath;
- directories on the system search path, which (on Linux at least) consists of the entries in /etc/ld.so.conf plus /lib and /usr/lib .
The rpath is stored in the executable (it’s the DT_RPATH or DT_RUNPATH dynamic attribute). It can contain absolute paths or paths starting with $ORIGIN to indicate a path relative to the location of the executable (e.g. if the executable is in /opt/myapp/bin and its rpath is $ORIGIN/../lib:$ORIGIN/../plugins then the dynamic linker will look in /opt/myapp/lib and /opt/myapp/plugins ). The rpath is normally determined when the executable is compiled, with the -rpath option to ld , but you can change it afterwards with chrpath .
In the scenario you describe, if you’re the developer or packager of the application and intend for it to be installed in a …/bin , …/lib structure, then link with -rpath=’$ORIGIN/../lib’ . If you’re installing a pre-built binary on your system, either put the library in a directory on the search path ( /usr/local/lib if you’re the system administrator, otherwise a directory that you add to $LD_LIBRARY_PATH ), or try chrpath .
What’s the difference between `-rpath-link` and `-L`?
The man for bfd ld makes it sort of sound like -rpath-link is used for recursively included sos. ld.lld doesn’t even list it as an argument. Could somebody clarify this situation for me?
2 Answers 2
Here is a demo, for GNU ld , of the difference between -L and -rpath-link — and for good measure, the difference between -rpath-link and -rpath .
extern void foo(void); extern void bar(void); void foobar(void)
extern void foobar(void); int main(void)
Make two shared libraries, libfoo.so and libbar.so :
$ gcc -c -Wall -fPIC foo.c bar.c $ gcc -shared -o libfoo.so foo.o $ gcc -shared -o libbar.so bar.o Make a third shared library, libfoobar.so that depends on the first two;
$ gcc -c -Wall -fPIC foobar.c $ gcc -shared -o libfoobar.so foobar.o -lfoo -lbar /usr/bin/ld: cannot find -lfoo /usr/bin/ld: cannot find -lbar collect2: error: ld returned 1 exit status Oops. The linker doesn’t know where to look to resolve -lfoo or -lbar .
$ gcc -shared -o libfoobar.so foobar.o -L. -lfoo -lbar The -Ldir option tells the linker that dir is one of the directories to search for libraries that resolve the -lname options it is given. It searches the -L directories first, in their commandline order; then it searches its configured default directories, in their configured order.
Now make a program that depends on libfoobar.so :
$ gcc -c -Wall main.c $ gcc -o prog main.o -L. -lfoobar /usr/bin/ld: warning: libfoo.so, needed by ./libfoobar.so, not found (try using -rpath or -rpath-link) /usr/bin/ld: warning: libbar.so, needed by ./libfoobar.so, not found (try using -rpath or -rpath-link) ./libfoobar.so: undefined reference to `bar' ./libfoobar.so: undefined reference to `foo' collect2: error: ld returned 1 exit status Oops again. The linker detects the dynamic dependencies requested by libfoobar.so but can’t satisfy them. Let’s resist its advice — try using -rpath or -rpath-link — for a bit and see what we can do with -L and -l :
$ gcc -o prog main.o -L. -lfoobar -lfoo -lbar $ ./prog ./prog: error while loading shared libraries: libfoobar.so: cannot open shared object file: No such file or directory at runtime, the loader can’t find libfoobar.so .
What about the linker’s advice then? With -rpath-link , we can do:
$ gcc -o prog main.o -L. -lfoobar -Wl,-rpath-link=$(pwd) and that linkage also succeeds. ( $(pwd) means «Print Working Directory» and just «copies» the current path.)
The -rpath-link=dir option tells the linker that when it encounters an input file that requests dynamic dependencies — like libfoobar.so — it should search directory dir to resolve them. So we don’t need to specify those dependencies with -lfoo -lbar and don’t even need to know what they are. What they are is information already written in the dynamic section of libfoobar.so :-
$ readelf -d libfoobar.so Dynamic section at offset 0xdf8 contains 26 entries: Tag Type Name/Value 0x0000000000000001 (NEEDED) Shared library: [libfoo.so] 0x0000000000000001 (NEEDED) Shared library: [libbar.so] 0x0000000000000001 (NEEDED) Shared library: [libc.so.6] . . We just need to know a directory where they can be found, whatever they are.
But does that give us a runnable prog ?
$ ./prog ./prog: error while loading shared libraries: libfoobar.so: cannot open shared object file: No such file or directory No. Same as story as before. That’s because -rpath-link=dir gives the linker the information that the loader would need to resolve some of the dynamic dependencies of prog at runtime — assuming it remained true at runtime — but it doesn’t write that information into the dynamic section of prog . It just lets the linkage succeed, without our needing to spell out all the recursive dynamic dependencies of the linkage with -l options.
At runtime, libfoo.so , libbar.so — and indeed libfoobar.so — might well not be where they are now — $(pwd) — but the loader might be able to locate them by other means: through the ldconfig cache or a setting of the LD_LIBRARY_PATH environment variable, e.g:
$ export LD_LIBRARY_PATH=.; ./prog foo bar rpath=dir provides the linker with the same information as rpath-link=dir and instructs the linker to bake that information into the dynamic section of the output file. Let’s try that:
$ export LD_LIBRARY_PATH= $ gcc -o prog main.o -L. -lfoobar -Wl,-rpath=$(pwd) $ ./prog foo bar All good. Because now, prog contains the information that $(pwd) is a runtime search path for shared libraries that it depends on, as we can see:
$ readelf -d prog Dynamic section at offset 0xe08 contains 26 entries: Tag Type Name/Value 0x0000000000000001 (NEEDED) Shared library: [libfoobar.so] 0x0000000000000001 (NEEDED) Shared library: [libc.so.6] 0x000000000000000f (RPATH) Library rpath: [/home/imk/develop/so/scrap] . ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ . That search path will be tried after the directories listed in LD_LIBRARY_PATH , if any are set, and before the system defaults — the ldconfig -ed directories, plus /lib and /usr/lib .
What’s the difference between -rpath and -L?
The GNU Compiler Collection (aka gcc) and ld provide many ways to specify a search path for libraries—among them the -rpath and -L flags. The manpages reveal no differences between these two flags, effectively saying each flag adds a library to the library search path. Yet it seems strange that both flags do exactly the same thing. What are the differences, if any, between these two options?
1 Answer 1
You must be reading some outdated copies of the manpages (emphasis added):
-rpath=dir
Add a directory to the runtime library search path. This is used
when linking an ELF executable with shared objects. All -rpath
arguments are concatenated and passed to the runtime linker, which
uses them to locate shared objects at runtime.
-L searchdir
—library-path=searchdir
Add path searchdir to the list of paths that ld will search for
archive libraries and ld control scripts.
So, -L tells ld where to look for libraries to link against when linking. You use this (for example) when you’re building against libraries in your build tree, which will be put in the normal system library paths by make install . —rpath , on the other hand, stores that path inside the executable, so that the runtime dynamic linker can find the libraries. You use this when your libraries are outside the system library search path.