dd
dd is a utility that lets you copy data from one device/file to another while letting you change the block size and perform conversions if desired.
Tasks
Wiping a Disk
Since UEFI, GPT, and RAID information is typically saved to both the start and end of a disk, in order to clear the metadata, you'll have to write to both ends of the disk.
To wipe the first and last 1MB of a disk:
## Wipe first 1MB
# dd bs=512 if=/dev/zero of=/dev/sdb count=2048
## Wipe last 1MB
# dd bs=512 if=/dev/zero of=/dev/sdb count=2048 seek=$((`blockdev --getsz /dev/sdb` - 2048))
There are other ways to wipe a disk on the Securely Erasing a Disk page.
Copy while ignoring IO errors
To make dd ignore errors, such as when cloning a failing drive, add the conv=noerror option. See also my other page on copying files from a failing disk.
The conv option specifies the type of conversion that dd needs to do to the input data. In this case, specifying noerror 'converts' errors from the input medium into nulls in the output medium.
# dd conv=noerror,sync if=/dev/sda1 > /mnt/rescue.img
There are lots of other conversions that dd can handle such as case conversion, EBCDIC, byte swapping, and more. (see https://www.gnu.org/software/coreutils/manual/html_node/dd-invocation.html#dd-invocation)
Manipulating data byte-by-byte
Use the iflag=skip_bytes,count_bytes flags when working with data in bytes. The bs block size value will be ignored when these flags are used as the skip and offset values will be in bytes.
| If you need... | Example command |
|---|---|
| everything past byte 199232000 from a file: | $ dd if=blah.dat iflag=skip_bytes,count_bytes skip=199232000 | xxd
|
| only 200 bytes of data starting at 199232000: | $ dd if=blah.dat iflag=skip_bytes,count_bytes skip=199232000 count=200 | xxd
|
| only 2 kilobytes (2048 bytes) at 199232000: | $ dd if=blah.dat iflag=skip_bytes,count_bytes skip=199232000 count=2k | xxd
|
Use-Case Example
I had a broken multi-archive .rar archive which was incomplete. rar didn't like the file and refused to extract the file past this broken archive:
$ rar e -kb something.iso.part001.rar something.iso
...
something.iso - checksum error
Unexpected end of archive
something.iso : packed data checksum error in volume something.iso.part009.rar
I needed the file, even if it was corrupt in the middle. rar's repair tool didn't seem to help either. So, I appended the difference from the archive before it so that the file is 'complete' (ie. it has the same size as the other part archives) and then retried extracting the file again.
This time, rar complained packed data checksum error in volume but continued on with the extraction and while the file is 'broken', the -kb option kept the file.
Write to a specific LBA
You may wish to write to a specific LBA to force a hard drive to reallocate a bad sector. You can either use hdparm --write-sector ### or you can use this method using dd (though, hdparm is probably better suited for this task).
If you know the bad LBA (such as from a SMART test, as shown below), you can write to the specific LBA by using its offset. For example:
SMART Self-test log structure revision number 1
Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error
1 Extended offline Completed: read failure 40% 12786 4005685760
Knowing that each sector is 512 bytes and knowing the LBA of the first error, we can tell dd to write to the LBA location with 512 block sizes which should overwrite this bad block:
# dd if=/dev/zero of=/dev/sdb skip=4005685760 bs=512 conv=sync
Re-read this block back afterwards and see if it's relocated.
Extracting resources from a binary
See: Extract .exe Resources with dd
Patching an executable with dd
See: Patching a binary file with dd