Linux dd Command: A Complete Guide

Linux is an amazing operating system that provides many tools and utilities for various purposes. One of these tools is the dd command, which can be used for copying and converting files, disks, partitions, and more. In this article, we will explain what the dd command is, how to use its syntax and options, and show some practical examples of its usage.

What is the dd command?

The dd command in Linux is a utility for copying and converting files and has many practical uses. It has been suggested that the name is derived from an older IBM Job Control Language function where dd stood for “Data Definition”. In Linux, the abbreviation stands for “Data Duplicator” or “Disk Dump” or a variety of other alliterations depending on your source. It may have even earned the poignant nickname “data destroyer”.

The reason for this is that it is capable of causing some serious damage. Hence, be sure to double check your syntax to avoid a costly mistake. You don’t want to be the person that confuses partition names and watches in agony as their root partition is destroyed and replaced with a blank file.

The dd command can be used for backing up and restoring disks or partitions, creating ISO files from CDs or DVDs, creating bootable USBs, wiping data, reading random or zero data, converting text cases, and more. It can also handle different formats of data, such as ASCII, EBCDIC, binary, etc.

Syntax and options

The basic syntax of the dd command is as follows:

dd [OPERAND]...
dd OPTION

The most common operands are if and of, which stand for input file and output file respectively. These operands specify the source and destination of the data that will be copied or converted by the dd command. For example, if we want to copy the contents of /dev/sda (a hard disk) to /dev/sdb (another hard disk), we can use the following command:

dd if=/dev/sda of=/dev/sdb

The input file and output file can be any file, device, or standard input/output. For example, we can use stdin (standard input) from our keyboard to collect input and point it to a file. We can also use stdout (standard output) to display the output on our terminal.

There are also many other options that can be used with the dd command to modify its behavior and performance. Some of the frequently used options are:

bs=BYTES: Set both input and output block size to BYTES bytes.
count=N: Copy only N input blocks.
status=LEVEL: Set verbosity level of information displayed on stderr.
conv=CONVS: Apply one or more conversions to the data.
seek=N: Skip N output blocks before copying.
skip=N: Skip N input blocks before copying.

For a complete list of options, you can refer to the man page of the dd command by typing man dd in your terminal.

Examples

Now that we have learned the basics of the dd command syntax and options, let’s see some real-world examples of how we can use it for various purposes.

Creating disk or partition image

One of the most common uses of the Linux dd command is to create an image of a disk or a partition. This can be useful for backup and restore purposes, or for cloning a disk or a partition to another device.

To create an image of a disk or a partition, you need to specify the device name as the input file and the image name as the output file. For example, if you want to create an image of /dev/sda (the first hard disk) and save it as sda.img in your home directory, you can use this command:

sudo dd if=/dev/sda of=~/sda.img status=progress

The sudo prefix is needed because you need root privileges to access /dev/sda. The status=progress option shows the progress of the copying process.

Note that this command will copy the entire disk, including free space and partitions. If you want to copy only a specific partition, such as /dev/sda1 (the first partition on /dev/sda), you can use this command:

sudo dd if=/dev/sda1 of=~/sda1.img status=progress

To restore an image to a disk or a partition, you need to reverse the input and output files. For example, if you want to restore the sda.img image to /dev/sda, you can use this command:

sudo dd if=~/sda.img of=/dev/sda status=progress

Warning: Be very careful when using dd to copy or restore disks or partitions. Make sure you specify the correct device names and avoid overwriting your data. Always double-check your syntax before executing the command.

Restoring a disk or partition image

We can also use the dd command to restore a disk or partition image that we have created earlier. For example, if we want to restore the image file sda.img that we created in the previous example to /dev/sda, we can use the following command:

sudo dd if=~/sda.img of=/dev/sda

In this command, we used the sudo command to obtain the required permissions to access /dev/sda. We then used the if operand to specify the sda.img file as input and the of operand to specify the /dev/sda partition as output.

Reading zero or random data

The dd command can also be used to read zero or random data and write it to a file or a device. This can be useful for testing, benchmarking, or wiping data. For example, if we want to create a file named zero.dat with 100 megabytes of zero data, we can use the following command:

dd if=/dev/zero of=~/zero.dat bs=1M count=100

In this command, we used the special device /dev/zero, which provides an endless stream of zero bytes, as input. We then used the of operand to specify the output file name, and the bs and count options to specify that we want 100 blocks of 1 megabyte each.

Similarly, if we want to create a file named random.dat with 100 megabytes of random data, we can use the following command:

dd if=/dev/urandom of=~/random.dat bs=1M count=100

In this command, we used the special device /dev/urandom, which provides an endless stream of pseudo-random bytes, as input. The rest of the options are similar to the previous command.

Wiping a disk or partition

The dd command can also be used to wipe a disk or partition by overwriting it with zero or random data. This can be useful for securely erasing data or preparing a disk for a new installation. For example, if we want to wipe /dev/sdb1 (a partition) with zero data, we can use the following command:

sudo dd if=/dev/zero of=/dev/sdb1 bs=4M status=progress

In this command, we used the sudo command to obtain the required permissions to access /dev/sdb1. We then used the /dev/zero device as input and /dev/sdb1 as output. We also used the bs option to set the block size to 4 megabytes for faster wiping, and the status option to show the progress on stderr.

Similarly, if we want to wipe /dev/sdb1 with random data, we can use the following command:

sudo dd if=/dev/urandom of=/dev/sdb1 bs=4M status=progress

In this command, we used the /dev/urandom device as input instead of /dev/zero. The rest of the options are similar to the previous command

Tips and Tricks

The dd command is a versatile and powerful tool that can be used for many purposes. However, there are some tips and tricks that can make it even more useful and efficient. Here are some of them:

Monitoring the progress of dd

One of the drawbacks of the dd command is that it does not show any progress indicator by default. This can make it hard to know how long the operation will take or if it is stuck. However, there are some ways to overcome this limitation. One of them is to use the status option with the value progress, as we have seen in the previous examples. This will show the amount of data transferred, the transfer rate, and the elapsed time on stderr.

Another way is to use the kill command with the signal USR1 to send a message to the dd process. This will make it print the current status on stderr without interrupting the operation. For example, if we want to monitor the progress of a dd process with the PID 1234, we can use the following command in another terminal:

kill -USR1 1234

This will show something like this on stderr:

123+0 records in
123+0 records out
129761280 bytes (130 MB, 124 MiB) copied, 10.0869 s, 12.9 MB/s

We can also use a loop to send this signal periodically. For example, we can use the following command to send the signal every 10 seconds:

while true; do kill -USR1 1234; sleep 10; done

Speeding up dd with block size

Another drawback of the dd command is that it can be slow if the block size is not optimal for the input and output devices. The block size is the amount of data that is read or written at a time by the dd command. By default, the block size is 512 bytes, which may be too small for modern devices. A small block size means more system calls and more overhead, which can reduce the performance.

To speed up the dd command, we can use a larger block size that matches the physical or logical sector size of the devices. For example, if we are copying from or to a hard disk that has a sector size of 4096 bytes, we can use bs=4K or bs=4096 to set the block size accordingly. This will reduce the number of system calls and increase the transfer rate.

However, there is no universal rule for choosing the best block size for every situation. It may depend on various factors such as the device type, speed, cache, alignment, etc. Therefore, it is advisable to experiment with different values and measure the performance with tools such as time or pv. For example, we can use the following command to measure how long it takes to copy 1 gigabyte of data from /dev/zero to /dev/null with different block sizes:

time dd if=/dev/zero of=/dev/null bs=VALUE count=1024

We can replace VALUE with different values such as 512, 4K, 1M, etc., and compare the results.

Conclusion

The dd command is a versatile and powerful tool that can be used for many purposes. However, it also comes with some risks and limitations that require caution and attention. Therefore, it is important to always double check your syntax and backup your data before using it.