How to Determine and Fix Boot Issues in Linux
The Linux system boots so fast that most of the output scrolls by too quickly to read the text (showing services being started) sent to the console. Therefore observing boot issues/errors becomes a little of a challenge for us.
In this article, we will briefly explain the different stages in a Linux system boot process, then learn how to establish and get to the bottom of boot issues: in terms files to look into or commands to view system boot messages.
Summary Of Linux Boot Process
In summary, once we press the Power On button, the BIOS (Basic Input Output System) a program integrated in a motherboard performs a POST (Power on Self Test) – where hardware such as disks, RAM (Random Access Memory), keyboard, etc are scanned. In case of an error (missing/malfunctioning hardware), it is reported on the screen.
During POST, the BIOS also looks for the boot device, the disk to stat up from (usually the first hard disk, however we may configure it to be a DVD, USB, Network Card etc instead).
Then the system will connect to the disk and search for the Master Boot Record (512 bytes in size) which stores the boot loader (446 bytes in size) and the rest of the space stores info about disk partitions (four maximum) and the MBR itself.
The boot loader will identify and point to, as well as load the Kernel and the initrd file (initialization ram disk – provides the kernel access to the mounted root filesystem and modules/drivers stored in /lib directory), which are generally stored in the /boot directory of the filesystem.
After the kernel is loaded, it executes init (or systemd on newer Linux distros), the first process with PID 1, which in turn starts all other processes on the system. It is also the last process to be executed on system shutdown.
How to Find Out Linux Boot Issues or Error Messages
As we mentioned before, the Linux boot processes happens rapidly that we can’t even clearly read most of the output sent to the console.
So taking note of boot issues/errors calls for a system administrator to look into certain important files in conjunction with particular commands. And these include:
/var/log/boot.log – Logs System Boot Messages
This is probably the first file you want to look into, to view all that unfolded during the system boot.
Rather than trying so hard to follow the output on the screen during boot, we can view this file after the boot process has completed to assist us in determining and resolving boot issues/errors.
We use the cat command for that purpose as follows (below is a sample of this file):
[ OK ] Started Load/Save RF Kill Switch Status. [ TIME ] Timed out waiting for device dev-disk-by\x2duuid-53e41ce9\x2ddc18\x2d458c\x2dbc08\x2d584c208ed615.device. [DEPEND] Dependency failed for /dev/disk/by-uuid/53e41ce9-dc18-458c-bc08-584c208ed615. [DEPEND] Dependency failed for Swap. [ OK ] Reached target System Initialization. [ OK ] Listening on UUID daemon activation socket. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Listening on CUPS Scheduler. [ OK ] Started Daily apt activities. [ OK ] Reached target Timers. [ OK ] Listening on Avahi mDNS/DNS-SD Stack Activation Socket. [ OK ] Started ACPI Events Check. [ OK ] Started Trigger resolvconf update for networkd DNS. [ OK ] Started CUPS Scheduler. [ OK ] Reached target Paths. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Listening on ACPID Listen Socket. Starting Console System Startup Logging. [ OK ] Listening on Cockpit Web Service Socket. [ OK ] Reached target Sockets. [ OK ] Reached target Basic System. Starting LSB: Set the CPU Frequency Scaling governor to "ondemand". [ OK ] Started ACPI event daemon. [ OK ] Started mintsystem.service. Starting Detect the available GPUs and deal with any system changes. Starting LSB: daemon to balance interrupts for SMP systems. Starting Bluetooth service. [ OK ] Started ClamAV virus database updater. Starting LSB: Starts syslogd. [ OK ] Started Regular background program processing daemon. Starting Modem Manager. Starting Accounts Service. .
From the output above, we can see that there is a boot issues indicated by the lines below.
[DEPEND] Dependency failed for /dev/disk/by-uuid/53e41ce9-dc18-458c-bc08-584c208ed615. [DEPEND] Dependency failed for Swap.
Issue: Problem with the swap partition; the system either failed to read the swap file/device/partition or it’s not present.
Let’s check if the system is using swap space with free command.
# free total used free shared buff/cache available Mem: 3742792 2421060 433696 287376 888036 967000 Swap: 0 0 0
Alternatively, we may run the swapon command to view the system swap space usage summary (we’ll not get any output).
Note: The content of this file is cleared at system shutdown: new data is stored in it on a new boot.
/var/log/messages – General System Logs
This file stores general system messages, including the messages that are logged during system boot.
Jun 4 13:04:44 tecmint syslogd (GNU inetutils 1.9.4): restart Jun 4 13:19:55 tecmint -- MARK -- Jun 4 13:39:55 tecmint -- MARK -- Jun 4 13:59:55 tecmint -- MARK -- Jun 4 14:19:55 tecmint -- MARK -- Jun 4 14:20:17 tecmint vmunix: [ 4945.388740] i915 0000:00:02.0: BAR 6: [. 0x00000000 flags 0x2] has bogus alignment Jun 4 14:20:17 tecmint vmunix: [ 4945.388837] i915 0000:00:02.0: BAR 6: [. 0x00000000 flags 0x2] has bogus alignment Jun 4 14:20:17 tecmint vmunix: [ 4945.388903] i915 0000:00:02.0: BAR 6: [. 0x00000000 flags 0x2] has bogus alignment Jun 4 14:20:17 tecmint vmunix: [ 4945.388930] i915 0000:00:02.0: BAR 6: [. 0x00000000 flags 0x2] has bogus alignment Jun 4 14:20:17 tecmint vmunix: [ 4945.389334] i915 0000:00:02.0: BAR 6: [. 0x00000000 flags 0x2] has bogus alignment Jun 4 14:20:17 tecmint vmunix: [ 4945.389402] pcieport 0000:00:1c.0: BAR 15: assigned [mem 0xdfa00000-0xdfbfffff 64bit pref] .
Because this file may be relatively lengthy, we can view it in pages use the more command (which even shows a percentage) like this.
The contents of /var/log/messages unlike the previous file is not cleared shutdown because it does not only contain boot messages but messages concerning other system activities as well. So older files are compressed and kept on the system for later inspection as shown below.
# ls -l message* -rw-r--r-- 1 root root 1206127 Jun 10 14:20 messages -rw-r--r-- 1 root root 1419494 Jun 4 13:00 messages.1 -rw-r--r-- 1 root root 153011 May 28 09:30 messages.2.gz
dmesg – Shows Kernel Messages
The dmesg command can show operations once the boot process has completed, such as command line options passed to the kernel; hardware components detected, events when a new USB device is added, or errors like NIC (Network Interface Card) failure and the drivers report no link activity detected on the network and so much more.
[ 0.000000] Initializing cgroup subsys cpuset [ 0.000000] Initializing cgroup subsys cpu [ 0.000000] Initializing cgroup subsys cpuacct [ 0.000000] Linux version 4.4.0-21-generic (buildd@lgw01-21) (gcc version 5.3.1 20160413 (Ubuntu 5.3.1-14ubuntu2) ) #37-Ubuntu SMP Mon Apr 18 18:33:37 UTC 2016 (Ubuntu 4.4.0-21.37-generic 4.4.6) [ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-4.4.0-21-generic root=UUID=57b36d48-1938-43c2-bf85-e97bc9f423ea ro quiet splash [ 0.000000] KERNEL supported cpus: [ 0.000000] Intel GenuineIntel [ 0.000000] AMD AuthenticAMD [ 0.000000] Centaur CentaurHauls [ 0.000000] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256 [ 0.000000] x86/fpu: Supporting XSAVE feature 0x01: 'x87 floating point registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x02: 'SSE registers' [ 0.000000] x86/fpu: Supporting XSAVE feature 0x04: 'AVX registers' [ 0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format. [ 0.000000] x86/fpu: Using 'eager' FPU context switches. [ 0.000000] e820: BIOS-provided physical RAM map: [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000008ffff] usable [ 0.000000] BIOS-e820: [mem 0x0000000000090000-0x00000000000bffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000001fffffff] usable [ 0.000000] BIOS-e820: [mem 0x0000000020000000-0x00000000201fffff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000020200000-0x0000000040003fff] usable [ 0.000000] BIOS-e820: [mem 0x0000000040004000-0x0000000040004fff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000040005000-0x0000000080b2ffff] usable [ 0.000000] BIOS-e820: [mem 0x0000000080b30000-0x0000000080d31fff] reserved [ 0.000000] BIOS-e820: [mem 0x0000000080d32000-0x00000000ce3eefff] usable [ 0.000000] BIOS-e820: [mem 0x00000000ce3ef000-0x00000000ce5eefff] type 20 [ 0.000000] BIOS-e820: [mem 0x00000000ce5ef000-0x00000000daeeefff] reserved [ 0.000000] BIOS-e820: [mem 0x00000000daeef000-0x00000000daf9efff] ACPI NVS .
journalctl – Query Contents of Systemd Journal
This is useful in most current Linux distros using systemd: the newest system and service manager. Under it, system logging is done via the journal component – a centralized location for all messages logged by different components.
These messages include kernel and boot messages; messages from syslog or various services. We can use it to review boot messages and establish boot issues by reading through the output and identifying lines of interest (errors pointed out by red lines depending on terminal text color settings).
Jun 13 16:35:32 tecmint mtp-probe[963]: checking bus 2, device 5: "/sys/devices/pci0000:00/0000:00:1d.0/u Jun 13 16:35:32 tecmint mtp-probe[963]: bus: 2, device: 5 was not an MTP device Jun 13 16:35:54 tecmint systemd[1]: dev-disk-by\x2duuid-53e41ce9\x2ddc18\x2d458c\x2dbc08\x2d584c208ed615. Jun 13 16:35:54 tecmint systemd[1]: Timed out waiting for device dev-disk-by\x2duuid-53e41ce9\x2ddc18\x2d Jun 13 16:35:54 tecmint systemd[1]: Dependency failed for /dev/disk/by-uuid/53e41ce9-dc18-458c-bc08-584c2 Jun 13 16:35:54 tecmint systemd[1]: Dependency failed for Swap. Jun 13 16:35:54 tecmint systemd[1]: swap.target: Job swap.target/start failed with result 'dependency'. Jun 13 16:35:54 tecmint systemd[1]: dev-disk-by\x2duuid-53e41ce9\x2ddc18\x2d458c\x2dbc08\x2d584c208ed615. Jun 13 16:35:54 tecmint systemd[1]: dev-disk-by\x2duuid-53e41ce9\x2ddc18\x2d458c\x2dbc08\x2d584c208ed615. Jun 13 16:35:54 tecmint systemd[1]: Reached target System Initialization. Jun 13 16:35:54 tecmint systemd[1]: Started ACPI Events Check. Jun 13 16:35:54 tecmint systemd[1]: Listening on CUPS Scheduler. Jun 13 16:35:54 tecmint systemd[1]: Starting Console System Startup Logging. Jun 13 16:35:54 tecmint systemd[1]: Started Daily Cleanup of Temporary Directories.
The above is a sample of the command output showing an error we already identified by viewing /var/log/boot.log: the swap partition error. To view more output lines, simply press [Enter] button.
Also check out these useful guides about Linux system boot and service management in relation to common system and service managers:
In this article, we explained in short the different stages in the Linux system boot process as well as learned how to establish and possibly resolve boot issues. To add any thought to this guide, use the comment form below.