Understanding Shared Libraries in Linux
In programming, a library is an assortment of pre-compiled pieces of code that can be reused in a program. Libraries simplify life for programmers, in that they provide reusable functions, routines, classes, data structures, and so on (written by another programmer), which they can use in their programs.
For instance, if you are building an application that needs to perform math operations, you don’t have to create a new math function for that, you can simply use existing functions in libraries for that programming language.
Examples of libraries in Linux include libc (the standard C library) or Glibc (GNU version of the standard C library), libcurl (multiprotocol file transfer library), libcrypt (library used for encryption, hashing, and encoding in C), and many more.
Linux supports two classes of libraries, namely:
- Static libraries – are bound to a program statically at compile time.
- Dynamic or shared libraries – are loaded when a program is launched and loaded into memory and binding occurs at run time.
Dynamic or shared libraries can further be categorized into:
- Dynamically linked libraries – here a program is linked with the shared library and the kernel loads the library (in case it’s not in memory) upon execution.
- Dynamically loaded libraries – the program takes full control by calling functions with the library.
Shared Library Naming Conventions
Shared libraries are named in two ways: the library name (a.k.a soname) and a “filename” (absolute path to file which stores library code).
For example, the soname for libc is libc.so.6: where lib is the prefix, c is a descriptive name, so means shared object, and 6 is the version. And its filename is: /lib64/libc.so.6. Note that the soname is actually a symbolic link to the filename.
Locating Shared Libraries in Linux
Shared libraries are loaded by ld.so (or ld.so.x) and ld-linux.so (or ld-linux.so.x) programs, where x is the version. In Linux, /lib/ld-linux.so.x searches and loads all shared libraries used by a program.
A program can call a library using its library name or filename, and a library path stores directories where libraries can be found in the filesystem. By default, libraries are located in /usr/local/lib, /usr/local/lib64, /usr/lib and /usr/lib64; system startup libraries are in /lib and /lib64. Programmers can, however, install libraries in custom locations.
The library path can be defined in /etc/ld.so.conf file which you can edit with a command-line editor.
The line(s) in this file instruct the kernel to load file in /etc/ld.so.conf.d. This way, package maintainers or programmers can add their custom library directories to the search list.
If you look into the /etc/ld.so.conf.d directory, you’ll see .conf files for some common packages (kernel, mysql, and postgresql in this case):
# ls /etc/ld.so.conf.d kernel-2.6.32-358.18.1.el6.x86_64.conf kernel-2.6.32-696.1.1.el6.x86_64.conf mariadb-x86_64.conf kernel-2.6.32-642.6.2.el6.x86_64.conf kernel-2.6.32-696.6.3.el6.x86_64.conf postgresql-pgdg-libs.conf
If you take a look at the mariadb-x86_64.conf, you will see an absolute path to package libraries.
# cat mariadb-x86_64.conf /usr/lib64/mysql
The method above sets the library path permanently. To set it temporarily, use the LD_LIBRARY_PATH environment variable on the command line. If you want to keep the changes permanent, then add this line in the shell initialization file /etc/profile (global) or ~/.profile (user-specific).
# export LD_LIBRARY_PATH=/path/to/library/file
Managing Shared Libraries in Linux
Let us now look at how to deal with shared libraries. To get a list of all shared library dependencies for a binary file, you can use the ldd utility. The output of ldd is in the form:
library name => filename (some hexadecimal value) OR filename (some hexadecimal value) #this is shown when library name can’t be read
This command shows all shared library dependencies for the ls command.
# ldd /usr/bin/ls OR # ldd /bin/ls
Sample Output
linux-vdso.so.1 => (0x00007ffebf9c2000) libselinux.so.1 => /lib64/libselinux.so.1 (0x0000003b71e00000) librt.so.1 => /lib64/librt.so.1 (0x0000003b71600000) libcap.so.2 => /lib64/libcap.so.2 (0x0000003b76a00000) libacl.so.1 => /lib64/libacl.so.1 (0x0000003b75e00000) libc.so.6 => /lib64/libc.so.6 (0x0000003b70600000) libdl.so.2 => /lib64/libdl.so.2 (0x0000003b70a00000) /lib64/ld-linux-x86-64.so.2 (0x0000561abfc09000) libpthread.so.0 => /lib64/libpthread.so.0 (0x0000003b70e00000) libattr.so.1 => /lib64/libattr.so.1 (0x0000003b75600000)
Because shared libraries can exist in many different directories, searching through all of these directories when a program is launched would be greatly inefficient: which is one of the likely disadvantages of dynamic libraries. Therefore a mechanism of caching is employed, performed by the program ldconfig.
By default, ldconfig reads the content of /etc/ld.so.conf, creates the appropriate symbolic links in the dynamic link directories, and then writes a cache to /etc/ld.so.cache which is then easily used by other programs.
This is very important especially when you have just installed new shared libraries or created your own, or created new library directories. You need to run the ldconfig command to effect the changes.
# ldconfig OR # ldconfig -v #shows files and directories it works with
After creating your shared library, you need to install it. You can either move it into any of the standard directories mentioned above and run the ldconfig command.
Alternatively, run the following command to create symbolic links from the soname to the filename:
# ldconfig -n /path/to/your/shared/libraries
To get started with creating your own libraries, check out this guide from The Linux Documentation Project(TLDP).
That’s all for now! In this article, we gave you an introduction to libraries and explained shared libraries, and how to manage them in Linux. If you have any queries or additional ideas to share, use the comment form below.
How do you link to a specific version of a shared library in GCC
I’m compiling some code which uses libcurl on a Debian Linux system. My dev machine is running Debian 5 but I want the binary to be usable on older Debian 4 systems too. I find that if I specify -lcurl it will link to libcurl.so.4 but Debian 4 systems only have libcurl.so.3 Is there some way I can tell GCC to link to either libcurl.so.3 (which exists in both Debian 4 and 5) or just libcurl.so so it will use whatever version is available ?
On the older Debian, isn’t libcurl.so a symlink to libcurl.so.3 ? I mean, it looks strange that -lcurl does not the right thing by default.
kastauyra: the versions are not, or at least cannot be assumed to be, binary compatible. So when you link it records the major version linked against in the binary: if you compile on the newer system it will require version 4 and not work on the old system. (Actually what it records is the soname, which is a string stored in the library file which conventionally but not necessarily is something «libcurl.so.3»)
4 Answers 4
Instead of using -lcurl use -l:libcurl.so.3 And of course also use -L _installed_path_
This doesn’t work for me even with the rpath. ldd shows it linking with one thing. The linked output shows the correct filename. But strace confirms it is loading file.2 instead of file.2.3 and it isn’t a symlink.
it does not really work for me for some reason, even if I replace symlink with actual file and delete all the other libraries its still linking against specific version, which it probably gets from the SONAME
You can pass the specific version shared library file using the syntax -l:libfoo.so.1 that specifies the filename instead of the syntax -lfoo that specifies the library name following the convention libfoo.so on the linker command line, and it ought to do what you want as can be seen at the linker documentation section for the option —library=namespec .
If namespec is of the form :filename, ld will search the library path for a file called filename
In order to provide more details on how to link to a specific version through an example, consider a system that contains two versions of the same library, namely libfoo.so.1.0 and libfoo.so.2.0 installed in one of the library directories, in this case /lib .
$ ls -l /lib/libfoo* lrwxrwxrwx root root /lib/libfoo.so -> /lib/libfoo.so.2 lrwxrwxrwx root root /lib/libfoo.so.1 -> /lib/libfoo.so.1.1 -rwxr-xr-x root root /lib/libfoo.so.1.0 -rwxr-xr-x root root /lib/libfoo.so.1.1 lrwxrwxrwx root root /lib/libfoo.so.2 -> /lib/libfoo.so.2.2 -rwxr-xr-x root root /lib/libfoo.so.2.0 -rwxr-xr-x root root /lib/libfoo.so.2.1 -rwxr-xr-x root root /lib/libfoo.so.2.2 # ldconfig -p | grep libfoo libfoo.so.2 (libc6,x86-64) => /lib/libfoo.so.2 libfoo.so.1 (libc6,x86-64) => /lib/libfoo.so.1 libfoo.so (libc6,x86-64) => /lib/libfoo.so A program compiled with the option -lfoo will make the linker look for a file that relies on the naming convention and thus resolve to /lib/libfoo.so (for a shared library object) or /lib/libfoo.a (for a static library object).
A special file name convention is used for libraries: A library known as foo is expected to exist as the file libfoo.so or libfoo.a.
In contrast to that, a program compiled with the option -l:libfoo.so.1 will be linked against /lib/libfoo.so.1 , that is a itself currently a symbolic link to libfoo.so.1.1 as can be seen from the listing above, a minor update from 1.0.
And finally, a program compiled with the option -l:libfoo.so.2 will be linked against /lib/libfoo.so.2 , that is itself currently a symbolic link to libfoo.so.2.2 as can be seen from the listing above, a minor update from 2.0 and 2.1.
Should you install a newer version of such library, as long as it is a minor update, there should be no need to recompile programs linked to it, since compatible versions should have the same soname and the symbolic links should be updated accordingly.
The actual library foo version X.Y exists as the file libfoo.so.x.y. Inside the library file, a soname is recorded with the value libfoo.so.x to signal the compatibility.
$ ls -l /lib/libfoo.so.2* lrwxrwxrwx root root /lib/libfoo.so.2 -> /lib/libfoo.so.2.3 [. ] -rwxr-xr-x root root /lib/libfoo.so.2.3