m
 
Line 1: Line 1:
{{DISPLAYTITLE:dd}}{{code|dd}} is a utility that lets you copy data from one device/file to another while letting you change the block size and perform conversions if desired.
+
{{DISPLAYTITLE:dd}}
  
== Wiping a Disk ==
+
{{code|dd}} is a utility that lets you copy data from one device/file to another while letting you change the block size and perform conversions if desired.
 +
 
 +
==Wiping a Disk==
  
 
Since UEFI, GPT, and RAID information is typically saved to both the start and end of a disk, in order to clear the metadata, you'll have to write to both ends of the disk.
 
Since UEFI, GPT, and RAID information is typically saved to both the start and end of a disk, in order to clear the metadata, you'll have to write to both ends of the disk.
Line 14: Line 16:
 
}}
 
}}
  
== Ignoring Errors ==
+
==Ignoring Errors==
 
To make dd ignore errors, such as when cloning a failing drive, add the {{code|1=conv=noerror}} option.
 
To make dd ignore errors, such as when cloning a failing drive, add the {{code|1=conv=noerror}} option.
  
Line 25: Line 27:
 
There are lots of other conversions that dd can handle such as case conversion, EBCDIC, byte swapping, and more. (see https://www.gnu.org/software/coreutils/manual/html_node/dd-invocation.html#dd-invocation)
 
There are lots of other conversions that dd can handle such as case conversion, EBCDIC, byte swapping, and more. (see https://www.gnu.org/software/coreutils/manual/html_node/dd-invocation.html#dd-invocation)
  
== Manipulating by Bytes ==
+
==Manipulating by Bytes==
 
Use the {{code|1=iflag=skip_bytes,count_bytes}} flags when working with data in bytes. The {{code|bs}} block size value will be ignored when these flags are used.
 
Use the {{code|1=iflag=skip_bytes,count_bytes}} flags when working with data in bytes. The {{code|bs}} block size value will be ignored when these flags are used.
  
 
Question: When using the {{code|count_bytes}} flag, is this equivalent to {{code|1=bs=1}}? (It looks like it...)
 
Question: When using the {{code|count_bytes}} flag, is this equivalent to {{code|1=bs=1}}? (It looks like it...)
  
=== Offset / Skipping Bytes ===
+
===Offset / Skipping Bytes===
 
If you need to extract a chunk of data, say everything past byte 199232000 from a file:
 
If you need to extract a chunk of data, say everything past byte 199232000 from a file:
  
Line 47: Line 49:
 
}}
 
}}
  
==== Use-Case Example ====
+
====Use-Case Example====
 
I had a broken multi-archive .rar archive which was incomplete. {{code|rar}} didn't like the file and refused to extract the file past this broken archive:
 
I had a broken multi-archive .rar archive which was incomplete. {{code|rar}} didn't like the file and refused to extract the file past this broken archive:
  
Line 62: Line 64:
 
This time, {{code|rar}} complained {{code|packed data checksum error in volume}} but continued on with the extraction and while the file is 'broken', the {{code|-kb}} option kept the file.
 
This time, {{code|rar}} complained {{code|packed data checksum error in volume}} but continued on with the extraction and while the file is 'broken', the {{code|-kb}} option kept the file.
  
== See Also ==
+
==== Write to a specific LBA ====
* https://www.gnu.org/software/coreutils/manual/html_node/dd-invocation.html
+
If you don't have [[hdparm]], you can use this method to write to a specific LBA. For example, if SMART is reporting an error on a specific LBA:
 +
{{Highlight
 +
| code = SMART Self-test log structure revision number 1
 +
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
 +
1  Extended offline    Completed: read failure      40%    12786        4005685760
 +
| lang = text
 +
}}
 +
You can write to that specific LBA by using it as the offset and using 512 block sizes. You might want to do this if (for example) you want to see if the disk can relocate the bad block elsewhere.
 +
{{Highlight
 +
| code = # dd if=/dev/zero of=/dev/sdb skip=4005685760 bs=512 conv=sync
 +
| lang = terminal
 +
}}
 +
Re-read this block back afterwards and see if it's relocated.
 +
 
 +
==See Also==
 +
 
 +
*https://www.gnu.org/software/coreutils/manual/html_node/dd-invocation.html
  
{{Navbox Linux}}[[Category:Linux]]
+
{{Navbox Linux}}
 +
[[Category:Linux]]
 
[[Category:LinuxUtilities]]
 
[[Category:LinuxUtilities]]

Latest revision as of 17:46, 31 July 2020


dd is a utility that lets you copy data from one device/file to another while letting you change the block size and perform conversions if desired.

Wiping a Disk[edit | edit source]

Since UEFI, GPT, and RAID information is typically saved to both the start and end of a disk, in order to clear the metadata, you'll have to write to both ends of the disk.

To wipe the first and last 1MB of a disk:

## Wipe first 1MB
# dd bs=512 if=/dev/zero of=/dev/sdb count=2048

## Wipe last 1MB
# dd bs=512 if=/dev/zero of=/dev/sdb count=2048 seek=$((`blockdev --getsz /dev/sdb` - 2048))

Ignoring Errors[edit | edit source]

To make dd ignore errors, such as when cloning a failing drive, add the conv=noerror option.

The conv option specifies the type of conversion that dd needs to do to the input data. In this case, specifying noerror 'converts' errors from the input medium into nulls in the output medium.

# dd conv=noerror,sync if=/dev/sda1 > /mnt/rescue.img

There are lots of other conversions that dd can handle such as case conversion, EBCDIC, byte swapping, and more. (see https://www.gnu.org/software/coreutils/manual/html_node/dd-invocation.html#dd-invocation)

Manipulating by Bytes[edit | edit source]

Use the iflag=skip_bytes,count_bytes flags when working with data in bytes. The bs block size value will be ignored when these flags are used.

Question: When using the count_bytes flag, is this equivalent to bs=1? (It looks like it...)

Offset / Skipping Bytes[edit | edit source]

If you need to extract a chunk of data, say everything past byte 199232000 from a file:

$ dd if=blah.dat iflag=skip_bytes,count_bytes skip=199232000 | xxd

To get only 200 bytes of data instead:

$ dd if=blah.dat iflag=skip_bytes,count_bytes skip=199232000 count=200 | xxd

Or 2 kilobytes (2048 bytes):

$ dd if=blah.dat iflag=skip_bytes,count_bytes skip=199232000 count=2k | xxd

Use-Case Example[edit | edit source]

I had a broken multi-archive .rar archive which was incomplete. rar didn't like the file and refused to extract the file past this broken archive:

$ rar e -kb something.iso.part001.rar something.iso
...
something.iso - checksum error
Unexpected end of archive
something.iso : packed data checksum error in volume something.iso.part009.rar

I needed the file, even if it was corrupt in the middle. rar's repair tool didn't seem to help either. So, I appended the difference from the archive before it so that the file is 'complete' (ie. it has the same size as the other part archives) and then retried extracting the file again.

This time, rar complained packed data checksum error in volume but continued on with the extraction and while the file is 'broken', the -kb option kept the file.

Write to a specific LBA[edit | edit source]

If you don't have hdparm, you can use this method to write to a specific LBA. For example, if SMART is reporting an error on a specific LBA:

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
 1  Extended offline    Completed: read failure       40%     12786         4005685760

You can write to that specific LBA by using it as the offset and using 512 block sizes. You might want to do this if (for example) you want to see if the disk can relocate the bad block elsewhere.

# dd if=/dev/zero of=/dev/sdb skip=4005685760 bs=512 conv=sync

Re-read this block back afterwards and see if it's relocated.

See Also[edit | edit source]