Change logical sector size to 4k
Many times asked, but without a conclusive answer: Can you change the logical block size from 512e to 4k (physical block size)? A solution using hdparm —set-sector-size 4096 doesn’t work under qemu/kvm so i can’t really test it, without using a spare device which i don’t have. Can the linux kernel be convinced to switch to 4k sector sizes instead of 512e? Optimally in a way, which can be tested in a VM.
It is the drive you will have to convince to change sector size. Whether this is possible or not is very much dependent on the drive.
As soon as I saw the hdparm solution, I was dreading something like that. A quick glance at the kernel source blk-settings.c gave me some hope, because the kernel seems to set logical, physical and io_min to a default 512. So if the kernel determines these sizes, i hoped .
Mainly trying to get rid of UEFI/GPT on larger HDDs. But giving an explanation to «WHY» rarely helps with the «HOW». It will most probably escalate into bikeshading and explaining why someone wants to paint his bike red instead of how it’s best done. 😉
If you don’t need to boot from the drive, and you’re using LVM or Btrfs or ZFS, you can eschew partitions entirely, thus avoiding GPT altogether.
4 Answers 4
Changing a HDD to native 4k sectors works at least with WD Red Plus 14 TB drives but LOSES ALL DATA. The data is not actually wiped but partition tables and filesystems cannot be found after the change because of their now incorrect LBA locations.
hdparm --set-sector-size 4096 --please-destroy-my-drive /dev/sdX This command changes your drive to native 4k sectors. The change persists on drive over reboots but you can revert it by setting 512 at some later time. REBOOT IMMEDIATELY after adjusting your disks. Attempt partitioning the drives and adding data only after a reboot (gdisk will then show 4096/4096 sector size).
- The kernel was getting read errors trying to access the drive directly after adjustment. A controller rescan or hotplugging can be used instead, but for most it is easiest to reboot the machine.
- Expect problems if you try to dd any disk images created in 512 byte sector size (e.g. .iso files meant for installing distros).
- For NVME SSDs the LBA sector size can be changed with the nvme utility (in package nvme-cli on Debian based ditros).
Can you change the logical block size from 512e to 4k (physical block size)?
You can ask a drive to change its logical block size. This is what hdparm —set-sector-size 4096 does.
The fact that this is implemented in the drive has two practical consequences:
- it only works if the drive supports it (as Johan Myréensaid);
- it can only be done by accessing the drive itself, so testing in a VM would only work if the VM is given full access to the target drive.
Yes you can. But you need to modify your system on hard drive level and not on operating system level. The operating system will take care of it.
The next issue you have to keep in mind is that such an operation will break your partition table both MBR and GPT if that drive contains already data.
A simple example to verify what I said is to just duplicate a Samsung S1 Mini drive (1k sectors) to a standard drive (usual 512 byte emulation).
As today’s drive mostly run on a 4k native size you will have to patch the firmware (I can’t do that) to generate the desired behaviour if your drive’s manufacturer doesn’t provides you with that option.
When searching for such drives myself I noticed that the manufacturers cash in nice markups for drives that translate physical 4k into logical 4k without «downsizing» to 512. The same applies to the ability to quickly return a read error if a sector read fails (important for RAID arrays). This is just a firmware issue. On typical end user drives the drives read lots of time repeatedly before finally returning an error condition.
Maybe putting in a controller card in between that allows for such a translation could solve the issue — just an idea.
Please let us know how you solved the problem — that is an interesting one!
From the comments it appears you want to use msdos style partitions («to not use UEFI/GPT»), and presumably you want to use 4K sector sizes to allow the maximum partition sizes and disk usage by the msdos partition. One way of doing this in linux is to use the a loopback device to provide a view of the underlying block device that has a different sector size.
# truncate -s 20T disk.20T.img # parted -s disk.20T.img mklabel msdos p mkpart primary 16M 10TB p Model: (file) Disk /home/ubuntu/disk.20T.img: 22.0TB Sector size (logical/physical): 512B/512B Partition Table: msdos Disk Flags: Number Start End Size Type File system Flags Error: partition length of 19531218944 sectors exceeds the msdos-partition-table-imposed maximum of 4294967295 Here we can see that creating a 10Tb msdos partition fails on a 20Tb disk with 512 byte sector size. However, when using a loopback device on the disk and setting the sector size to 4K, a 10Tb msdos partition is created and an EXT4 partition can be created using the full partition.
# losetup --sector-size 4096 --show -f disk.20T.img /dev/loop18 # blockdev --getss --getpbsz /dev/loop18 4096 4096 # parted -s /dev/loop18 mklabel msdos mkpart primary 16M 10TB p Model: Loopback device (loopback) Disk /dev/loop18: 22.0TB Sector size (logical/physical): 4096B/4096B Partition Table: msdos Disk Flags: Number Start End Size Type File system Flags 1 15.7MB 10000GB 10000GB primary lba # kpartx -av /dev/loop18 add map loop18p1 (253:4): 0 19531218944 linear 7:18 30720 # mkfs.ext4 /dev/mapper/loop18p1 mke2fs 1.46.5 (30-Dec-2021) Discarding device blocks: done Creating filesystem with 2441402368 4k blocks and 305176576 inodes Filesystem UUID: 6aa698fa-1cb7-4aa6-9ba6-ebc493d40808 Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968, 102400000, 214990848, 512000000, 550731776, 644972544, 1934917632 Allocating group tables: done Writing inode tables: done Creating journal (262144 blocks): done Writing superblocks and filesystem accounting information: done # mount /dev/mapper/loop18p1 /mnt # df -h /mnt Filesystem Size Used Avail Use% Mounted on /dev/mapper/loop18p1 9.1T 28K 8.6T 1% /mnt If you want to use this in a QEMU virtual machine you can add a disk specifying physical and logical sector sizes of the disk using the properties physical_block_size=4096 and logical_block_size=4096 .
For convenience, here’s a command I used to test this. You can use any bootable iso you’d like:
qemu-system-x86_64 -name 'test live cd: bios' -m 2048 -serial stdio -rtc base=localtime \ -vga std --enable-kvm -cdrom /media/ubuntu/STORAGE/isos/lubuntu-18.04-desktop-i386.iso \ -boot d -device virtio-scsi-pci,id=scsi1,bus=pci.0 \ -drive file=/tmp/disk.20T.img,if=none,id=drive-virtio-disk1 \ -device scsi-hd,bus=scsi1.0,drive=drive-virtio-disk1,id=virtio-scsi-pci1,physical_block_size=4096,logical_block_size=4096 Now in the VM you can install do what ever tests you’re wanting to do with a 4K drive. Again, if you want to test how an OS handles the live changing of a block device from 512 byte to 4K sector sizes, you’re out of luck unless you want to write that emulation into QEMU. Regardless, I think it would be helpful to clarify the question.
how to set block size using blockdev command
Anyone can provide me any test case for the below command: blockdev —setbsz BYTES to set/change the block size. I have tried like below but no luck.
$ blockdev --setbsz 2048 /dev/sda5 blockdev: 2048: No such file or directory I tried to use —setbsz option for blockdev; but giving the above error. What will be the correct syntax for this option?
from man: Note that the block size is specific to the current file descriptor opening the block device, so the change of block size only persists for as long as blockdev has the device open, and is lost once blockdev exits.
3 Answers 3
Check the block size of current device.
$ blockdev --getbsz /dev/vdb1 512 Unmount filesystem to change block size.
Create filesystem to change new block size.
$ mkfs -t ext4 -b 4096 /dev/vdb1 Mount to check the changed block size.
$ mount /dev/vdb1 /test/ $ blockdev --getbsz /dev/vdb1 4096 You have to do it as root user after unmounting that device.
# this is as root user [root@pse-linvm ~]# blockdev --setbsz 4096 /dev/vda2 BLKBSZSET: Device or resource busy [root@pse-linvm ~]# blockdev --setbsz 4096 /dev/vda6 /dev/vda6: No such file or directory # this is normal user [root@pse-linvm ~]# su - raja [raja@pse-linvm ~]$ blockdev --getbsz /dev/vda2 /dev/vda2: Permission denied # Error same as yours [raja@pse-linvm ~]$ blockdev --getbsz 4096 /dev/vda2 4096: No such file or directory Not an answer — just posting in answer box for clarity
[dani@localhost ~]$ blockdev --getbsz /dev/sda1 blockdev: cannot open /dev/sda1: Permission denied [dani@localhost ~]$ [dani@localhost ~]$ su - Password: [root@localhost ~]# blockdev --getbsz /dev/sda1 1024 [root@localhost ~]# blockdev --setbsz 1024 /dev/sda1 BLKBSZSET: Device or resource busy [root@localhost ~]# umount /boot [root@localhost ~]# blockdev --setbsz 1024 /dev/sda1 works well on fd20 with linux-utils 2.24.2
$ mount /dev/sda1 on / type ext3 (rw,errors=remount-ro) proc on /proc type proc (rw,noexec,nosuid,nodev) none on /sys type sysfs (rw,noexec,nosuid,nodev) . binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,noexec,nosuid,nodev) $ sudo blockdev —getbsz /dev/sda1 4096 $ sudo blockdev —getbsz /dev/sdb1 512 $ sudo blockdev —setbsz 2048 /dev/sdb1 $ sudo blockdev —getbsz /dev/sdb1 512 Here i mount and tried to change the bsz but no change. And i didn’t get any error.
How To Find the Default Block Size in Unix
The questions about default block sizes used in your Unix system are always popular. Today I’d like to show you a few ways to answer them.
Default block size in Linux
If you ever want to confirm the block size of any filesystem of Ubuntu or any other Linux OS, tune2fs command is here to help:
ubuntu# tune2fs -l /dev/sda1 | grep Block Block count: 4980736 Block size: 4096 Blocks per group: 32768From this example, you can see that the default block size for the filesystem on /dev/sda1 partition is 4096 bytes, or 4k. That’s the default block size for ext3 filesystem.
Default block size in Solaris
The default block size in Solaris is 8192 bytes, or 8k. However, some architectures allow you to use 4k size as well, by specifying it as a command line option for the newfs command. To be absolutely sure, you can use one of the commands: df -g (takes a filesystem mount point name as the parameter – / or /usr for example) or use fstyp -v command (needs a character device of the filesystem you’re interested in).
Using df -g to confirm the filesystem block size
This command can be used as any user, so to confirm a block size for any of the filesystems you don’t have to be root. However, it works only for mounted filesystems.
bash-3.00$ df -g / / (/dev/dsk/c1t0d0s0 ): 8192 block size 1024 frag size 12405898 total blocks 4399080 free blocks 4275022 available 751296 total files 603544 free files 30932992 filesys id ufs fstype 0x00000004 flag 255 filename lengthUsing fstyp -v to confirm the filesystem block size
Because this command accesses the character device of a particular filesystem, you have to be root to run it. But as a bonus compared to df -g, you can use fstyp -v on an unmounted filesystem:
bash-3.00# fstyp -v /dev/dsk/c1t0d0s0 | grep ^bsize bsize 8192 shift 13 mask 0xffffe000