Enlarge a disk and partition of any Linux VM without a reboot
Some days ago, one of our Performance Analyzer customers asked in a workshop how we handle the data growth over time and how they can resize the data partition. Like many other customers, they want to make sure that the disks are not filling up anytime soon.
On the other hand our goal was, that no customer needs to oversize the data disk – and our automatic resizing solves that issue in a very convenient way:
There is only a simple resize of the virtual disk of the virtual appliance required and the scripts in our appliance handle everything else automatically.
That can be done while the virtual machine is running using pretty much any modern linux distribution – just make sure that no snapshots are active!
When following that guide for the first time on a production system – we would recommend a backup (without snapshots).
Performance Analyzer is smart enough to detect the change, and enlarges both the partition and the file system contained in the partition.
- detect the resize of either the boot or the data disk
- resize the last partition to add the new space
But what is the magic behind that?
Something we didn’t mention – all is done without LVM!
Let’s get into some details:
The Performance Analyzer appliance has a two disk configuration. The first disk holds the operating system, the second disk the data. We skipped the swap partition and have a swap file within the root file system. That is acceptable since we avoid swapping at all costs anyway.
It’s important to know that only the last or the only partition within a disks (green color) can be resized in an straightforward way.
The disk layout of Performance Analyzer looks like this:
If you want to enlarge the second disk, you simply change the VM configuration and increase the disk there. If the option is greyed out then make sure to remove existing snapshots.
Although the disk is resized, the Linux kernel is not aware of that change.
disk is resized, but the linux kernel is not aware of the new size without reboot or interaction
To make the kernel aware, one can issue the following command as root:
echo 1 > /sys/class/block/sdb/device/rescanThis triggers a rescan of the device sdb. If you resized another disk, replace sdb with the disk number you changed. You can follow the procedure in syslog – and you should see some lines similar to this:
[89741.613318] sd 2:0:1:0: [sdb] 106954752 512-byte logical blocks: (54.8 GB/51.0 GiB) [89741.613393] sdb: detected capacity change from 53687091200 to 54760833024When the kernel finishes the rescan (usually within fractions of a second), it is aware of the larger disk:
new disk size detected, partition is unchanged
The partition is not automatically adjusted and needs to be resized as well in two steps
Typically fdisk was the tool of choice for the first step and a utility like partprobe (or a reboot) for the second step.
But things changed and you can use a great software called growpart
growpart is part of the cloud-utils-package, and should be available in your distro’s repositories, in case its not already installed with your OS. Debian or Ubuntu:
sudo apt-get install -y cloud-utilsWith growpart, enlarging the partition to maximum size and informing the kernel of the changed partition size is a one-liner:
The first parameter is the disk device, the second parameter is the number of the partition to resize. After running growpart, your disks look like this:
partition is resized using growpart
The partition is resized now and the kernel is already using the new partition table.
The last missing piece is the file system. Since we use an ext4 file system on Ubuntu in Performance Analyzer, this is an one-liner as well, that can even run on a mounted file system:
In case you’re using any other file system, you need to check for the right tools to resize.
completely resized without reboot or LVM
That’s it, the disk, the partition and the filesystem are resized and you can use the newly gained disk space immediately, without any reboot or the use of LVM.
Putting the pieces together
Opvizor automated this process by creating a cron job which runs every 5 minutes. This job checks, if one of the Performance Analyzer’s disk changed in size and resizes the partitions and file systems on this disks.
/etc/cron.d/resize_disk
As an example to run the script disk_resize every 5 minutes as root, simply add the following line to /etc/cron.d/resize_disk
*/5 * * * * root /usr/local/bin/disk_resize.shdisk_resize.sh
The script /usr/local/bin/disk_resize.sh itself looks that:
#!/bin/bash LOGFILE=/data/log/speed/resize.log LOCKFILE=/tmp/enlarge.lock export PATH=/usr/sbin:/usr/bin:/sbin:/bin NEEDREBOOT=0 dotlockfile -r 0 $LOCKFILE || exit 1 echo 1 > /sys/class/block/sda/device/rescan sleep 5 GROWPART_OUT=`growpart /dev/sda 2` if [ $? -eq 0 ]; then echo `date` >> $LOGFILE echo "trying to resize fs" >> $LOGFILE echo $GROWPART_OUT >> $LOGFILE resize2fs /dev/sda2 >> $LOGFILE 2>&1 echo `date` >> $LOGFILE echo "resize done" >> $LOGFILE #TODO: need reboot NEEDREBOOT=1 fi echo 1 2>/dev/null >/sys/class/block/sdb/device/rescan sleep 5 GROWPART_OUT=`growpart /dev/sdb 1` if [ $? -eq 0 ]; then echo `date` >> $LOGFILE echo "trying to resize fs" >> $LOGFILE echo $GROWPART_OUT >> $LOGFILE resize2fs /dev/sdb1 >> $LOGFILE 2>&1 echo `date` >> $LOGFILE echo "resize done" >> $LOGFILE #TODO: need reboot NEEDREBOOT=1 fi dotlockfile -u $LOCKFILE if [ $NEEDREBOOT -eq "1" ]; then /sbin/reboot fiIntroduction
I sometimes have to add a virtual HD to an existing virtual host running in VMWare and extend the host`s root partition without having to reboot it.
This can be easily done if the root partition on my Linux system is already located on an LVM volume.
Execution
After you assigned a new disk to the virtual machine you have to add it in the guest and expand the LVM volume.
First rescan the SCSI bus to make the kernel detect the new virtual disk:
# echo "- - -" > /sys/class/scsi_host/host0/scan
Find out what the new device is called, should be displayed with ‘dmesg’. If it is too noisy, you may get what you need with
Use fdisk to create a single partition on the new device, make it type 8e (LVM).
Initialize the new partition as a physical volume:
Extend your volume group with the new PV:
# vgextend yourvolumegroup /dev/sdX1
Extend your logical volume with the free space of the VG:
# lvextend -l +100%FREE /dev/yourvolumegroup/yourlogicalvolume
Resize the filesystem online (only ext3 and ext4!):
# resize2fs /dev/mapper/yourvolumegroup-yourlogicalvolume
Verify your new partitions free space with ‘df -h‘.
Live Example
In this example we will use commands extended to extend root partition with 20Gb.
In our example the / was 16Gb and almost full:
# df -h / Filesystem Size Used Avail Use% Mounted on /dev/mapper/localvg-root 16G 15G 412M 98% /
First we add a new virtual drive to the VmWare host, reread the SCSI bus and check the dmesg to see if the process was successful:
# dmesg [1900683.028115] scsi 0:0:2:0: Direct-Access VMware Virtual disk 1.0 PQ: 0 ANSI: 2 [1900683.028127] target0:0:2: Beginning Domain Validation [1900683.031944] target0:0:2: Domain Validation skipping write tests [1900683.031946] target0:0:2: Ending Domain Validation [1900683.032163] target0:0:2: FAST-40 WIDE SCSI 80.0 MB/s ST (25 ns, offset 127) [1900683.032796] sd 0:0:2:0: [sdc] 41943040 512-byte hardware sectors (21475 MB) [1900683.033357] sd 0:0:2:0: [sdc] Test WP failed, assume Write Enabled [1900683.033536] sd 0:0:2:0: [sdc] Cache data unavailable [1900683.033538] sd 0:0:2:0: [sdc] Assuming drive cache: write through [1900683.034207] sd 0:0:2:0: [sdc] 41943040 512-byte hardware sectors (21475 MB) [1900683.034734] sd 0:0:2:0: [sdc] Test WP failed, assume Write Enabled [1900683.034911] sd 0:0:2:0: [sdc] Cache data unavailable [1900683.034912] sd 0:0:2:0: [sdc] Assuming drive cache: write through [1900683.035191] sdc: unknown partition table [1900683.070657] sd 0:0:2:0: [sdc] Attached SCSI disk [1900683.070715] sd 0:0:2:0: Attached scsi generic sg3 type 0
Then we check the LVM info:
# lvdisplay /dev/mapper/localvg-root --- Logical volume --- LV Name /dev/localvg/root VG Name localvg LV UUID joih9S-vbJj-04NM-fbld-9e7Q-h9M4-Bnpaoa LV Write Access read/write LV Status available # open 1 LV Size 15.29 GB Current LE 3915 Segments 2 Allocation inherit Read ahead sectors 0 Block device 254:0
Time to partition the newly added device:
# fdisk /dev/sdc Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel Building a new DOS disklabel with disk identifier 0xe8d635ce. Changes will remain in memory only, until you decide to write them. After that, of course, the previous content won't be recoverable. The number of cylinders for this disk is set to 2610. There is nothing wrong with that, but this is larger than 1024, and could in certain setups cause problems with: 1) software that runs at boot time (e.g., old versions of LILO) 2) booting and partitioning software from other OSs (e.g., DOS FDISK, OS/2 FDISK) Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
Let’s see what device looks like before partitioning:
Command (m for help): p Disk /dev/sdc: 21.4 GB, 21474836480 bytes 255 heads, 63 sectors/track, 2610 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Disk identifier: 0xe8d635ce Device Boot Start End Blocks Id System
As expected, no partitions here. Now let’s create new partition:
Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 1 First cylinder (1-2610, default 1): Using default value 1 Last cylinder or +size or +sizeM or +sizeK (1-2610, default 2610): Using default value 2610
Let’s see the newly created partition:
Command (m for help): p Disk /dev/sdc: 21.4 GB, 21474836480 bytes 255 heads, 63 sectors/track, 2610 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Disk identifier: 0xe8d635ce Device Boot Start End Blocks Id System /dev/sdc1 1 2610 20964793+ 83 Linux
Now we need to change the partition type and set it to Linux LVM:
Command (m for help): t Selected partition 1 Hex code (type L to list codes): 8e Changed system type of partition 1 to 8e (Linux LVM) Command (m for help): p Disk /dev/sdc: 21.4 GB, 21474836480 bytes 255 heads, 63 sectors/track, 2610 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Disk identifier: 0xe8d635ce Device Boot Start End Blocks Id System /dev/sdc1 1 2610 20964793+ 8e Linux LVM
Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks.
Now add newly created device to the LVM volume:
# pvcreate /dev/sdc1 Physical volume "/dev/sdc1" successfully created # vgextend localvg /dev/sdc1 Volume group "localvg" successfully extended # lvextend -l +100%FREE /dev/localvg/root Extending logical volume root to 35.36 GB Logical volume root successfully resized
And finally resize the / partition:
# resize2fs /dev/mapper/localvg-root resize2fs 1.40.8 (13-Mar-2008) Filesystem at /dev/mapper/localvg-root is mounted on /; on-line resizing required old desc_blocks = 1, new_desc_blocks = 3 Performing an on-line resize of /dev/mapper/localvg-root to 9270272 (4k) blocks. The filesystem on /dev/mapper/localvg-root is now 9270272 blocks long.
# df -h / Filesystem Size Used Avail Use% Mounted on /dev/mapper/localvg-root 36G 15G 20G 43% /
scalple said .
Naveeka said .
Hobett said .
This is exactly what I needed. I’d read the LVM2 tutorials but was not certain that I could do a resize of root live from what I’d read, until I found this.