- Where do executables look for shared objects at runtime?
- 4 Answers 4
- How Do so (Shared Object) Filenames Work in Linux
- What are Shared Objects (so) Files?
- How Shared Object Filenames Work
- Examples of Shared Object Filenames
- libcrypto.so.1.1
- libssl.so.1.1
- libX11.so.6
- libgtk-3.so.0
- How Shared Object Files are Found
- Benefits of Shared Objects
- Major vs. Minor Version Numbers
- Managing Shared Objects
- Conclusion
- LD_LIBRARY_PATH, the shared lib path in linux
- 2 Answers 2
Where do executables look for shared objects at runtime?
I understand how to define include shared objects at linking/compile time. However, I still wonder how do executables look for the shared object ( *.so libraries) at execution time. For instance, my app a.out calls functions defined in the lib.so library. After compiling, I move lib.so to a new directory in my $HOME . How can I tell a.out to go look for it there?
A quick solution is to use g++ -Wl,-R. this will force loader to look into the current folder for so libraries. Or -Wl,-R$HOME/path to specidy some other fixed folder.
4 Answers 4
The shared library HOWTO explains most of the mechanisms involved, and the dynamic loader manual goes into more detail. Each unix variant has its own way, but most use the same executable format (ELF) and have similar dynamic linkers¹ (derived from Solaris). Below I’ll summarize the common behavior with a focus on Linux; check your system’s manuals for the complete story.
(Terminology note: the part of the system that loads shared libraries is often called “dynamic linker”, but sometimes “dynamic loader” to be more precise. “Dynamic linker” can also mean the tool that generates instructions for the dynamic loader when compiling a program, or the combination of the compile-time tool and the run-time loader. In this answer, “linker” refers to the run-time part.)
In a nutshell, when it’s looking for a dynamic library ( .so file) the linker tries:
- directories listed in the LD_LIBRARY_PATH environment variable ( DYLD_LIBRARY_PATH on OSX);
- directories listed in the executable’s rpath;
- directories on the system search path, which (on Linux at least) consists of the entries in /etc/ld.so.conf plus /lib and /usr/lib .
The rpath is stored in the executable (it’s the DT_RPATH or DT_RUNPATH dynamic attribute). It can contain absolute paths or paths starting with $ORIGIN to indicate a path relative to the location of the executable (e.g. if the executable is in /opt/myapp/bin and its rpath is $ORIGIN/../lib:$ORIGIN/../plugins then the dynamic linker will look in /opt/myapp/lib and /opt/myapp/plugins ). The rpath is normally determined when the executable is compiled, with the -rpath option to ld , but you can change it afterwards with chrpath .
In the scenario you describe, if you’re the developer or packager of the application and intend for it to be installed in a …/bin , …/lib structure, then link with -rpath=’$ORIGIN/../lib’ . If you’re installing a pre-built binary on your system, either put the library in a directory on the search path ( /usr/local/lib if you’re the system administrator, otherwise a directory that you add to $LD_LIBRARY_PATH ), or try chrpath .
How Do so (Shared Object) Filenames Work in Linux
Linux is an open-source operating system that provides users with a wide range of features and functions. One of essential aspects of Linux is use of shared objects (so) files. Shared objects are files that are used by Linux programs to share code and data between different processes. In this article, we will discuss how shared object filenames work in Linux and provide examples to illustrate concepts.
What are Shared Objects (so) Files?
Shared objects are a type of file that contains code and data that can be shared between multiple processes in Linux. They are similar to dynamic link libraries (DLLs) in Windows systems. When a Linux program needs to use a shared object, it loads it into memory and uses code and data contained in file.
Shared objects are used to reduce size of executable files and reduce amount of memory needed to run a program. They also provide a way to share code and data between different programs, allowing for more efficient and modular programming.
How Shared Object Filenames Work
Shared object filenames in Linux follow a specific naming convention that is used to identify file and its contents. filename consists of several parts, each of which provides information about file.
The naming convention for shared object filenames is as follows −
The parts of filename are −
- lib − This is a prefix that indicates that file is a shared library.
- − This is name of library. It is typically a short, descriptive name that identifies purpose of library.
- .so − This is a suffix that indicates that file is a shared object.
- − This is a number that indicates major version of library. A change in major version number indicates a significant change in API or ABI of library.
- − This is a number that indicates minor version of library. A change in minor version number indicates a small change in API or ABI of library.
Examples of Shared Object Filenames
Let’s take a look at some examples of shared object filenames to see how they work.
libcrypto.so.1.1
This filename indicates that file is a shared library called «crypto». major version number is 1, and minor version number is 1. This indicates that this is first major version of library and that there have been some minor changes to API or ABI.
libssl.so.1.1
This filename indicates that file is a shared library called «ssl». major version number is 1, and minor version number is 1. This indicates that this is first major version of library and that there have been some minor changes to API or ABI.
libX11.so.6
This filename indicates that file is a shared library called «X11». major version number is 6, and minor version number is 0. This indicates that this is sixth major version of library and that there have been no minor changes to API or ABI.
libgtk-3.so.0
This filename indicates that file is a shared library called «gtk-3». major version number is 0, and minor version number is 0. This indicates that this is first major version of library and that there have been no minor changes to API or ABI.
How Shared Object Files are Found
Shared objects are searched for and loaded by dynamic linker at runtime. dynamic linker is responsible for resolving symbols and linking shared objects with rest of program.
The dynamic linker searches for shared objects in several directories, including −
- /lib
- /usr
- /usr/local/lib
- Directories listed in LD_LIBRARY_PATH environment variable
When a program is compiled, it includes a list of shared object dependencies that it requires to run. When program is executed, dynamic linker searches for these dependencies in directories listed above. If it finds required shared objects, it loads them into memory and links them with program.
If required shared objects are not found, program will fail to run, and an error message will be displayed. To resolve this issue, you can install missing shared objects or add directory containing missing shared objects to LD_LIBRARY_PATH environment variable.
Benefits of Shared Objects
Shared objects provide several benefits over static libraries, including −
- Reduced memory usage − Shared objects are loaded into memory only when they are needed, reducing amount of memory needed to run a program.
- Dynamic linking − Shared objects allow programs to link with libraries at runtime, enabling them to adapt to changes in environment and load libraries only when they are required.
- Code reusability − Shared objects allow developers to reuse code across different programs, reducing development time and improving code maintainability.
- Faster program startup − Shared objects are loaded only when they are needed, reducing program startup time and improving program performance.
Major vs. Minor Version Numbers
The major and minor version numbers in shared object filenames provide information about compatibility of library with other programs. A change in major version number indicates a significant change in API or ABI of library. Programs that use old version of library may not be compatible with new version.
A change in minor version number indicates a small change in API or ABI of library. Programs that use old version of library should still be compatible with new version, but may not be able to take advantage of new features or improvements in library.
Managing Shared Objects
Managing shared objects in Linux can be challenging, particularly when dealing with dependencies between libraries. Here are some tips for managing shared objects in Linux −
- Use package managers − Most Linux distributions come with package managers that allow you to install and manage shared objects and their dependencies. Using a package manager can help ensure that your system is up-to-date and that you have all necessary dependencies.
- Avoid modifying system directories − Modifying system directories such as /lib and /usr can cause issues with dynamic linker and may break other programs. Instead, install shared objects in directories such as /usr/local/lib or in your home directory.
- Use symbolic links − If you have multiple versions of a shared object, you can use symbolic links to point to current version. This can help ensure that programs that depend on shared object can still find it even if version number changes.
- Check LD_LIBRARY_PATH − If you are experiencing issues with missing shared objects, check LD_LIBRARY_PATH environment variable to ensure that it includes directories containing required shared objects.
Conclusion
Shared objects are an essential part of Linux operating system. They allow programs to share code and data between processes, reducing size of executable files and amount of memory needed to run a program. Shared object filenames in Linux follow a specific naming convention that provides information about file’s contents, including name of library, major and minor version numbers, and file type. dynamic linker searches for shared objects at runtime in several directories, including /lib, /usr, and directories listed in LD_LIBRARY_PATH environment variable. Understanding how shared object filenames work is crucial for developing and running Linux programs efficiently.
LD_LIBRARY_PATH, the shared lib path in linux
I wrote a shared object, say libsd.so , and I put libsd.so and its header file sd.h in ~/lib . Here is another program using libsd.so , say test.c , then compile it like this:
$ gcc -o test test.c -I~/lib -L~/lib -lsd $ ./test ./test_sd: error while loading shared libraries: libsd.so: cannot open shared object file: No such file or directory So I set export LD_LIBRARY_PATH=. , then it works. But if I unset LD_LIBRARY_PATH and put LD_LIBRARY_PATH=~/lib in my ~/.bashrc , then source ~/.bashrc , again it doesn’t work for ./test , WHY? export LD_LIBRARY_PATH=~/lib is difference from putting LD_LIBRARY_PATH=~/lib in ~/.bashrc ?
2 Answers 2
Without the export your declared LD_LIBRARY_PATH is only valid in the script (.bashrc). With the export it should work, but it is usually not a good idea to set your LD_LIBRARY_PATH like this.
If you don’t want to install your library in the system path (e.g. /usr/lib) you should probably use a script that sets LD_LIBARAY_PATH locally and starts your application.
You can see here, why it is not a good idea: linuxmafia.com/faq/Admin/ld-lib-path.html However, it seems, that you shouldn’t even use it in a script, but rather use the -R option
On Fedora there should be -rpath, that should do pretty much the same thing, even though I actually do not know for sure.
Try $HOME/lib instead of ~/lib — it should be the same but I’ve seen cases where ~ wasn’t expanded properly when used in an variable assignment.
To check, try echo $LD_LIBRARY_PATH which gives you the current value.
Re export : If you omit the export , then the variable is only known to the current shell process and will not be exported to child processes. So if you omit it, echo $LD_LIBRARY_PATH will get the value because the variable is expanded by the shell before the echo command/builtin has a chance to do anything. But ./test won’t see it because it’s not exported to the new subprocess.