Hdparm: Difference between revisions

From Leo's Notes
This page was last edited on 20 January 2024, at 22:44.
 
(8 intermediate revisions by the same user not shown)
Line 1: Line 1:
hdparm is a disk utility that allows for low level commands to be sent to the storage device.
hdparm is a disk utility that allows for low level commands to be sent to the storage device.


==Repair a bad sector==
==Usage==
If you have a bad sector that the disk is unable to read, you can use hdparm to write to a specific sector which will hopefully cause the disk to remap the bad sector to a good sector. This process is done transparently by the disk's firmware and the number of bad sectors that have been reallocated can be seen under the <code>Reallocated_Sector_Ct</code> parameter on SMART.
hdparm can be used to on any mass storage device.


Given the following message, we know that LBA 4005685760 is bad:{{Highlight|lang=text|code=SMART Self-test log structure revision number 1
===Get disk information===
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
Use <code>hdparm -I /dev/disk</code> to show hardware properties of the device.
1  Extended offline    Completed: read failure      40%    12786        4005685760}}We can confirm this by attempting to read that sector with dd or hdparm:
{{Highlight
{{Highlight
| code = # dd if=/dev/sdb of=/dev/null skip=4005685760 bs=512 conv=sync
| code = # hdparm -I /dev/sdb
dd: error reading '/dev/sdb': Input/output error


# hdparm --read-sector 4005685760  /dev/sdb
/dev/sdb:
/dev/sdb:
reading sector 4005685760: FAILED: Input/output error
 
ATA device, with non-removable media
Model Number:      ST3000DM001-9YN166                     
Serial Number:      W1F0SJ27
Firmware Revision:  CC4B   
Transport:          Serial, SATA Rev 3.0
Standards:
Used: unknown (minor revision code 0x0029)
Supported: 8 7 6 5
Likely used: 8
Configuration:
Logical max current
cylinders 16383 16383
heads 16 16
sectors/track 63 63
--
CHS current addressable sectors:    16514064
LBA    user addressable sectors:  268435455
LBA48  user addressable sectors:  5860533168
Logical  Sector size:                  512 bytes
Physical Sector size:                  4096 bytes
Logical Sector-0 offset:                  0 bytes
device size with M = 1024*1024:    2861588 MBytes
device size with M = 1000*1000:    3000592 MBytes (3000 GB)
cache/buffer size  = unknown
Nominal Media Rotation Rate: 7200
Capabilities:
LBA, IORDY(can be disabled)
Queue depth: 32
Standby timer values: spec'd by Standard, no device specific minimum
R/W multiple sector transfer: Max = 16 Current = ?
Advanced power management level: 128
Recommended acoustic management value: 208, current value: 0
DMA: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 udma5 *udma6
    Cycle time: min=120ns recommended=120ns
PIO: pio0 pio1 pio2 pio3 pio4
    Cycle time: no flow control=120ns  IORDY flow control=120ns
Commands/features:
Enabled Supported:
  * SMART feature set
    Security Mode feature set
  * Power Management feature set
  * Write cache
  * Look-ahead
  * Host Protected Area feature set
  * WRITE_BUFFER command
  * READ_BUFFER command
  * DOWNLOAD_MICROCODE
  * Advanced Power Management feature set
    SET_MAX security extension
  * 48-bit Address feature set
  * Device Configuration Overlay feature set
  * Mandatory FLUSH_CACHE
  * FLUSH_CACHE_EXT
  * SMART error logging
  * SMART self-test
  * General Purpose Logging feature set
  * WRITE_{DMA{{!}}MULTIPLE}_FUA_EXT
  * 64-bit World wide name
    Write-Read-Verify feature set
  * WRITE_UNCORRECTABLE_EXT command
  * {READ,WRITE}_DMA_EXT_GPL commands
  * Segmented DOWNLOAD_MICROCODE
  * Gen1 signaling speed (1.5Gb/s)
  * Gen2 signaling speed (3.0Gb/s)
  * Gen3 signaling speed (6.0Gb/s)
  * Native Command Queueing (NCQ)
  * Phy event counters
  * READ_LOG_DMA_EXT equivalent to READ_LOG_EXT
    DMA Setup Auto-Activate optimization
    Device-initiated interface power management
  * Software settings preservation
  * SMART Command Transport (SCT) feature set
  * SCT Write Same (AC2)
    unknown 206[7]
    unknown 206[12] (vendor specific)
    unknown 206[13] (vendor specific)
Security:
Master password revision code = 65534
supported
not enabled
not locked
not frozen
not expired: security count
supported: enhanced erase
330min for SECURITY ERASE UNIT. 330min for ENHANCED SECURITY ERASE UNIT.
Logical Unit WWN Device Identifier: 5000c500529df4f3
NAA : 5
IEEE OUI : 000c50
Unique ID : 0529df4f3
Checksum: correct
| lang = terminal
| lang = terminal
}}
}}
Using hdparm, we can write to this specific sector. As per hdparm's man page, the write-sector issues a low-level write command directly to the disk. ('''Question''': Does this mean the write-sector will ignore remapped sectors by the disk firmware?) We may also try the same with dd. In fact, dd'ing the entire disk with zeros is a sure way of finding and remapping all bad sectors on a drive.
 
===Disk Timing and Benchmark===
Use <code>hdparm -t</code> to benchmark how fast the disk can be read.  The documentation suggests running this a few times on an idle system for the best result.
 
Because disks start from the outside of the platter, an offset could be used to test disk performance in another area with the <code>--offset <GiB></code> parameter. For example, a 8TB disk could be benchmarked with an offset of 4000 GB to see the read speed somewhere near the middle of the platter.
 
===Tuning===
You can tune how many sectors the disk is to return for each data transmission to help speed up disk reads.  According to the manual, this can help with the OS I/O overhead by 30-50% and may improve throughput between 5%-50%. Eg: <code>hdparm -m16</code>
 
You can improve sequential reads of large files by setting a hread-ahead value. This specifies how many sectors to read-ahead by in anticipation of them being read later on. Set this with the <code>-a</code> option: <code>hdparm -a16</code>.
 
Acoustic modes can be set with <code>-M</code>, passing either 128 (for quiet mode), or 254 (fastest).
 
==Tasks==
 
=== Secure Erase ===
See: [[Securely_Erasing_a_Disk#ATA_Secure_Erase]]
 
===Fix a damaged sector by forcing reallocation===
All modern hard drives have a small set of reserved sectors that can be reallocated (or mapped to) by bad sectors as they are detected by the disk firmware. This is usually a transparent process and only happens when the sector is written to. However, because of how filesystems (and possibly the kernel?) functions, it may require some manual intervention with <code>hdparm</code> which we'll go into details below.
 
Typically, bad sectors will present itself as a device that can't be read or written to at a specific location which results in the filesystem being mounted as read-only or the disk being dropped from a RAID array. Another good clue would be having <code>dd</code> result in an "Input/output error". The Linux kernel would also complain similar to the following (from <code>dmesg</code>):
{{Highlight
{{Highlight
| code = # dd if=/dev/zero of=/dev/sdb skip=4005685760 bs=512 count=1 conv=sync
| code = [Fri Apr  2 20:47:18 2021] sd 2:0:0:0: [sdb] tag#11 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
 
[Fri Apr  2 20:47:18 2021] sd 2:0:0:0: [sdb] tag#11 Sense Key : Medium Error [current]
# hdparm --write-sector 4005685760 --yes-i-know-what-i-am-doing /dev/sdb
[Fri Apr  2 20:47:18 2021] sd 2:0:0:0: [sdb] tag#11 Add. Sense: Unrecovered read error - auto reallocate failed
## A very long time later:
[Fri Apr  2 20:47:18 2021] sd 2:0:0:0: [sdb] tag#11 CDB: Read(16) 88 00 00 00 00 00 00 00 02 08 00 00 01 f8 00 00
[Fri Apr  2 20:47:18 2021] print_req_error: I/O error, dev sdb, sector 1000 flags 0
| lang = text
}}
Bad sectors can also be found with SMART tests. The LBA of any problematic sectors will be displayed with the <code>smartctl -a</code> output and will look something similar to the following under the self-test section:{{Highlight|lang=text|code=SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
1  Extended offline    Completed: read failure      40%    12786        4005685760}}Knowing which sectors are bad allows us to try making the drive firmware be aware of the issue and allow it to reallocate the bad sector. We can force a low level sector read or write using the <code>hdparm</code> utility. We can try reading the bad sector with <code>hdparm</code> to confirm the issue:{{Highlight
| code = # hdparm --read-sector 4005685760 /dev/sdb
/dev/sdb:
/dev/sdb:
re-writing sector 4005685760: succeeded
reading sector 4005685760: FAILED: Input/output error
| lang = terminal
| lang = terminal
}}
}}
hdparm took a very long time before showing the success message. <code>dmesg</code> shows that the disk is still having I/O errors on that logical block:
W can try to remap the bad sector by writing to this specific sector with <code>hdparm</code>. The drive may take some time to reallocate. If this fails with an error, something else far worse than bad sectors is probably wrong with the drive. Usually, bad sectors come consecutively and it may help to write to a range of sectors. <code>hdparm</code> doesn't appear to support this, but can be worked around using a loop in bash:
{{Highlight
{{Highlight
| code = [Fri Jul 31 11:22:36 2020] blk_update_request: I/O error, dev sdb, sector 0 op 0x1:(WRITE) flags 0x800 phys_seg 0 prio class 0
| code = # hdparm --write-sector 4005685760 --yes-i-know-what-i-am-doing /dev/sdb
[Fri Jul 31 11:22:38 2020] sd 2:0:0:0: [sdb] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE cmd_age=1s
## This could take some time if the disk needs to reallocate the sector.
[Fri Jul 31 11:22:38 2020] sd 2:0:0:0: [sdb] tag#0 Sense Key : Medium Error [current]
/dev/sdb:
[Fri Jul 31 11:22:38 2020] sd 2:0:0:0: [sdb] tag#0 Add. Sense: Unrecovered read error
re-writing sector 4005685760: succeeded
[Fri Jul 31 11:22:38 2020] sd 2:0:0:0: [sdb] tag#0 CDB: Read(16) 88 00 00 00 00 00 ee c1 ea 00 00 00 00 08 00 00
[Fri Jul 31 11:22:38 2020] blk_update_request: critical medium error, dev sdb, sector 4005685760 op 0x0:(READ) flags 0x80700 phys_seg 1 prio class 0
[Fri Jul 31 11:28:38 2020] sd 2:0:0:0: [sdb] tag#0 timing out command, waited 360s
[Fri Jul 31 11:28:38 2020] sd 2:0:0:0: [sdb] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE cmd_age=360s
[Fri Jul 31 11:28:38 2020] sd 2:0:0:0: [sdb] tag#0 Sense Key : Hardware Error [current]
[Fri Jul 31 11:28:38 2020] sd 2:0:0:0: [sdb] tag#0 Add. Sense: Internal target failure
[Fri Jul 31 11:28:38 2020] sd 2:0:0:0: [sdb] tag#0 CDB: Synchronize Cache(10) 35 00 00 00 00 00 00 00 00 00
[Fri Jul 31 11:28:38 2020] blk_update_request: I/O error, dev sdb, sector 0 op 0x1:(WRITE) flags 0x800 phys_seg 0 prio class 0
[Fri Jul 31 11:28:38 2020] sd 2:0:0:0: [sdb] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE cmd_age=0s
[Fri Jul 31 11:28:38 2020] sd 2:0:0:0: [sdb] tag#0 Sense Key : Medium Error [current]
[Fri Jul 31 11:28:38 2020] sd 2:0:0:0: [sdb] tag#0 Add. Sense: Unrecovered read error
[Fri Jul 31 11:28:38 2020] sd 2:0:0:0: [sdb] tag#0 CDB: Read(16) 88 00 00 00 00 00 ee c1 ea 00 00 00 00 08 00 00
[Fri Jul 31 11:28:38 2020] blk_update_request: critical medium error, dev sdb, sector 4005685760 op 0x0:(READ) flags 0x0 phys_seg 8 prio class 0
[Fri Jul 31 11:28:38 2020] Buffer I/O error on dev sdb, logical block 4005685760, async page read
[Fri Jul 31 11:28:38 2020] Buffer I/O error on dev sdb, logical block 4005685761, async page read
[Fri Jul 31 11:28:38 2020] Buffer I/O error on dev sdb, logical block 4005685762, async page read
[Fri Jul 31 11:28:38 2020] Buffer I/O error on dev sdb, logical block 4005685763, async page read
[Fri Jul 31 11:28:38 2020] Buffer I/O error on dev sdb, logical block 4005685764, async page read
[Fri Jul 31 11:28:38 2020] Buffer I/O error on dev sdb, logical block 4005685765, async page read
[Fri Jul 31 11:28:38 2020] Buffer I/O error on dev sdb, logical block 4005685766, async page read
[Fri Jul 31 11:28:38 2020] Buffer I/O error on dev sdb, logical block 4005685767, async page read
| lang = text
}}
However, the hdparm --read-sector on the sector now works, though dd still fails.
{{Highlight
| code = # dd if=/dev/sdb of=/dev/null skip=4005685760 bs=512 conv=sync
dd: error reading '/dev/sdb': Input/output error


# hdparm --read-sector 4005685760 /dev/sdb
## In cases where you think there's a range of bad sectors, you can do something like:
/dev/sdb:
# for i in {1000..1010} ; do echo $i ; hdparm --write-sector $i  --yes-i-know-what-i-am-doing  /dev/sdb ; done
reading sector 4005685760: succeeded
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
| lang = terminal
| lang = terminal
}}But the reallocate sector count didn't increase...
}}
 
If the corrupt sector was reallocated, you should see both the <code>Reallocated_Sector_Ct</code> and <code>Reallocated_Event_Count</code> value increase and the <code>Current_Pending_Sector</code> decrease in the SMART report (ie. from <code>smartctl -a /dev/sdb</code>). Your goal is to try writing to the disk using <code>hdparm</code> until <code>Current_Pending_Sector</code> becomes 0.<br /> {{Navbox Linux}}
Something's wonky with this...
[[Category:Linux]]
[[Category:LinuxUtilities]]

Latest revision as of 22:44, 20 January 2024

hdparm is a disk utility that allows for low level commands to be sent to the storage device.

Usage

hdparm can be used to on any mass storage device.

Get disk information

Use hdparm -I /dev/disk to show hardware properties of the device.

# hdparm -I /dev/sdb

/dev/sdb:

ATA device, with non-removable media
	Model Number:       ST3000DM001-9YN166                      
	Serial Number:      W1F0SJ27
	Firmware Revision:  CC4B    
	Transport:          Serial, SATA Rev 3.0
Standards:
	Used: unknown (minor revision code 0x0029) 
	Supported: 8 7 6 5 
	Likely used: 8
Configuration:
	Logical		max	current
	cylinders	16383	16383
	heads		16	16
	sectors/track	63	63
	--
	CHS current addressable sectors:    16514064
	LBA    user addressable sectors:   268435455
	LBA48  user addressable sectors:  5860533168
	Logical  Sector size:                   512 bytes
	Physical Sector size:                  4096 bytes
	Logical Sector-0 offset:                  0 bytes
	device size with M = 1024*1024:     2861588 MBytes
	device size with M = 1000*1000:     3000592 MBytes (3000 GB)
	cache/buffer size  = unknown
	Nominal Media Rotation Rate: 7200
Capabilities:
	LBA, IORDY(can be disabled)
	Queue depth: 32
	Standby timer values: spec'd by Standard, no device specific minimum
	R/W multiple sector transfer: Max = 16	Current = ?
	Advanced power management level: 128
	Recommended acoustic management value: 208, current value: 0
	DMA: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 udma5 *udma6 
	     Cycle time: min=120ns recommended=120ns
	PIO: pio0 pio1 pio2 pio3 pio4 
	     Cycle time: no flow control=120ns  IORDY flow control=120ns
Commands/features:
	Enabled	Supported:
	   *	SMART feature set
	    	Security Mode feature set
	   *	Power Management feature set
	   *	Write cache
	   *	Look-ahead
	   *	Host Protected Area feature set
	   *	WRITE_BUFFER command
	   *	READ_BUFFER command
	   *	DOWNLOAD_MICROCODE
	   *	Advanced Power Management feature set
	    	SET_MAX security extension
	   *	48-bit Address feature set
	   *	Device Configuration Overlay feature set
	   *	Mandatory FLUSH_CACHE
	   *	FLUSH_CACHE_EXT
	   *	SMART error logging
	   *	SMART self-test
	   *	General Purpose Logging feature set
	   *	WRITE_{DMA|MULTIPLE}_FUA_EXT
	   *	64-bit World wide name
	    	Write-Read-Verify feature set
	   *	WRITE_UNCORRECTABLE_EXT command
	   *	{READ,WRITE}_DMA_EXT_GPL commands
	   *	Segmented DOWNLOAD_MICROCODE
	   *	Gen1 signaling speed (1.5Gb/s)
	   *	Gen2 signaling speed (3.0Gb/s)
	   *	Gen3 signaling speed (6.0Gb/s)
	   *	Native Command Queueing (NCQ)
	   *	Phy event counters
	   *	READ_LOG_DMA_EXT equivalent to READ_LOG_EXT
	    	DMA Setup Auto-Activate optimization
	    	Device-initiated interface power management
	   *	Software settings preservation
	   *	SMART Command Transport (SCT) feature set
	   *	SCT Write Same (AC2)
	    	unknown 206[7]
	    	unknown 206[12] (vendor specific)
	    	unknown 206[13] (vendor specific)
Security: 
	Master password revision code = 65534
		supported
	not	enabled
	not	locked
	not	frozen
	not	expired: security count
		supported: enhanced erase
	330min for SECURITY ERASE UNIT. 330min for ENHANCED SECURITY ERASE UNIT.
Logical Unit WWN Device Identifier: 5000c500529df4f3
	NAA		: 5
	IEEE OUI	: 000c50
	Unique ID	: 0529df4f3
Checksum: correct

Disk Timing and Benchmark

Use hdparm -t to benchmark how fast the disk can be read. The documentation suggests running this a few times on an idle system for the best result.

Because disks start from the outside of the platter, an offset could be used to test disk performance in another area with the --offset <GiB> parameter. For example, a 8TB disk could be benchmarked with an offset of 4000 GB to see the read speed somewhere near the middle of the platter.

Tuning

You can tune how many sectors the disk is to return for each data transmission to help speed up disk reads. According to the manual, this can help with the OS I/O overhead by 30-50% and may improve throughput between 5%-50%. Eg: hdparm -m16

You can improve sequential reads of large files by setting a hread-ahead value. This specifies how many sectors to read-ahead by in anticipation of them being read later on. Set this with the -a option: hdparm -a16.

Acoustic modes can be set with -M, passing either 128 (for quiet mode), or 254 (fastest).

Tasks

Secure Erase

See: Securely_Erasing_a_Disk#ATA_Secure_Erase

Fix a damaged sector by forcing reallocation

All modern hard drives have a small set of reserved sectors that can be reallocated (or mapped to) by bad sectors as they are detected by the disk firmware. This is usually a transparent process and only happens when the sector is written to. However, because of how filesystems (and possibly the kernel?) functions, it may require some manual intervention with hdparm which we'll go into details below.

Typically, bad sectors will present itself as a device that can't be read or written to at a specific location which results in the filesystem being mounted as read-only or the disk being dropped from a RAID array. Another good clue would be having dd result in an "Input/output error". The Linux kernel would also complain similar to the following (from dmesg):

[Fri Apr  2 20:47:18 2021] sd 2:0:0:0: [sdb] tag#11 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[Fri Apr  2 20:47:18 2021] sd 2:0:0:0: [sdb] tag#11 Sense Key : Medium Error [current]
[Fri Apr  2 20:47:18 2021] sd 2:0:0:0: [sdb] tag#11 Add. Sense: Unrecovered read error - auto reallocate failed
[Fri Apr  2 20:47:18 2021] sd 2:0:0:0: [sdb] tag#11 CDB: Read(16) 88 00 00 00 00 00 00 00 02 08 00 00 01 f8 00 00
[Fri Apr  2 20:47:18 2021] print_req_error: I/O error, dev sdb, sector 1000 flags 0

Bad sectors can also be found with SMART tests. The LBA of any problematic sectors will be displayed with the smartctl -a output and will look something similar to the following under the self-test section:

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
 1  Extended offline    Completed: read failure       40%     12786         4005685760

Knowing which sectors are bad allows us to try making the drive firmware be aware of the issue and allow it to reallocate the bad sector. We can force a low level sector read or write using the hdparm utility. We can try reading the bad sector with hdparm to confirm the issue:

# hdparm --read-sector 4005685760  /dev/sdb
/dev/sdb:
reading sector 4005685760: FAILED: Input/output error

W can try to remap the bad sector by writing to this specific sector with hdparm. The drive may take some time to reallocate. If this fails with an error, something else far worse than bad sectors is probably wrong with the drive. Usually, bad sectors come consecutively and it may help to write to a range of sectors. hdparm doesn't appear to support this, but can be worked around using a loop in bash:

# hdparm --write-sector 4005685760 --yes-i-know-what-i-am-doing /dev/sdb
## This could take some time if the disk needs to reallocate the sector.
/dev/sdb:
re-writing sector 4005685760: succeeded

## In cases where you think there's a range of bad sectors, you can do something like:
# for i in {1000..1010} ; do echo $i ; hdparm --write-sector $i  --yes-i-know-what-i-am-doing  /dev/sdb ; done

If the corrupt sector was reallocated, you should see both the Reallocated_Sector_Ct and Reallocated_Event_Count value increase and the Current_Pending_Sector decrease in the SMART report (ie. from smartctl -a /dev/sdb). Your goal is to try writing to the disk using hdparm until Current_Pending_Sector becomes 0.