DRBD

DRBD (Distributed Replicated Block Device) is a virtual block device that is distributed across two or more machines.

Installation

On a clean installation of RHEL 7 or equivalent, run updates, and ensure that you have NTP and a recent kernel installed.

From ElRepo

Get the DRBD packages from ELRepo:

rpm -Uvh http://www.elrepo.org/elrepo-release-7.0-2.el7.elrepo.noarch.rpm
yum install drbd84-utils kmod-drbd84

You should be able to load the drbd module:

# modprobe drbd
# lsmod|grep drbd
drbd                  392583  0 
libcrc32c              12644  1 drbd

From Source

Install prerequisites:

yum install gcc flex make autoconf m4

Get the source from their git repository. Releases from tarballs may fail depending on which kernel version you're using.

git clone git://git.drbd.org/drbd-8.4.git
cd drbd-8.4
make
make install

The make install should have loaded the module and you can verify this:

lsmod | grep drbd

If it isn't loaded, load it using modprobe

modprobe drbd

Compile the userspace utilities:

git clone git://git.drbd.org/drbd-utils.git
cd drbd-utils
./aclocal.sh
./configure --prefix=/usr --localstatedir=/var --sysconfdir=/etc
make
make install

Note: I got errors about man pages not being found. Ugh. Use the RPMs instead (it worked after a clean install...)

Configuration

This section will configure DRBD for use in a redundant MySQL setup.

DRBD syncs volumes. If you did not configure a partition or have a separate disk to mirror, you can create a file volume instead.

On all nodes:

mkdir /var/drbd
cd /var/drbd

# Create a 50GB volume
dd if=/dev/zero of=data_volume.dat bs=1024 count=52428800

# Create a partition
echo -e "n\n\n\n\nw\nq\n" | fdisk data_volume.dat

# Create a device
losetup /dev/loop0 /var/drbd/data_volume.dat
kpartx -v -a /var/drbd/data_volume.dat
ls /dev/mapper/

mkdir -p /mnt/drbd
echo "/dev/drbd0 /mnt/drbd ext4 noauto 0 0 " >>  /etc/fstab

On all nodes, configure the DRBD config in /etc/drbd.d/*conf. You may want to disable the survey.

On all nodes, configure DRBD by creating a file in /etc/drbd.d/mysql.res with:

resource mysql {
	disk {
		on-io-error detach;
	}

	device    /dev/drbd0;
	disk      /dev/mapper/loop0p1;
	meta-disk internal;

	on $hostname {
		address 172.20.10.10:7700;
	}

	on $hostname {
		address 172.20.10.9:7700;
	}
}

TODO: Can you just use /dev/loop0 instead?!

Be sure to rename the hostnames so that it matches the actual machine's hostname. Otherwise, when you try creating the resource, you will get an error like:

'mysql' not defined in your config (for this host).

On all nodes, create the resource:

drbdadm -v create-md mysql

If it complains about finding a filesystem, zero out the first few blocks using dd and try again or ignore and continue.

On all nodes, start the resource:

drbdadm up mysql

You can check the status of the drbd volume:

drbd-overview
 0:mysql/0  Connected Secondary/Secondary Inconsistent/Inconsistent

The two volumes will not sync until one of them is set to be the primary.

On one of the nodes, force it to be the primary node.

drbdadm -- --force primary mysql

On the primary node, create a file system:

mkfs -t xfs /dev/drbd0

You can now mount the device on the primary node:

mount /dev/drbd0 /mnt/drbd

Setting up MySQL

Install MySQL (or MariaDB)

yum -y install mariadb mariadb-server

If you mounted the drbd volume to /mnt/drbd, you could make a symlink for /var/lib/mysql.

ln -s /mnt/drbd/mysql /var/lib/mysql

Initialize the database:

mysql_install_db 

Optionally, set the bind address in my.cnf to the shared cluster IP address. It should default to 0.0.0.0 if you don't do this which is also fine.

bind-address = 172.20.10.8

You'll need crmsh if you want to follow some other guides. Install it using the repository at:

http://download.opensuse.org/repositories/network:/ha-clustering:/Stable/CentOS_CentOS-7/x86_64/

cd
wget http://download.opensuse.org/repositories/network:/ha-clustering:/Stable/CentOS_CentOS-7/x86_64/python-pssh-2.3.1-4.2.x86_64.rpm http://download.opensuse.org/repositories/network:/ha-clustering:/Stable/CentOS_CentOS-7/x86_64/pssh-2.3.1-4.2.x86_64.rpm http://download.opensuse.org/repositories/network:/ha-clustering:/Stable/CentOS_CentOS-7/x86_64/crmsh-2.1.2-1.1.x86_64.rpm
yum install *.rpm

crm configure

primitive p_ip_mysql ocf:heartbeat:IPaddr2 \
  params ip="172.20.10.8" cidr_netmask="24" \
  op monitor interval="30s"
primitive p_drbd_mysql ocf:linbit:drbd \
  params drbd_resource="mysql" \
  op start timeout="90s" \
  op stop timeout="180s" \
  op promote timeout="180s" \
  op demote timeout="180s" \
  op monitor interval="30s" role="Slave" \
  op monitor interval="29s" role="Master"
primitive p_fs_mysql ocf:heartbeat:Filesystem \
  params device="/dev/drbd1" \
    directory="/mnt/drbd" \
    fstype="xfs" \
    options="relatime" \
  op start timeout="60s" \
  op stop timeout="180s" \
  op monitor interval="60s" timeout="60s"
primitive p_mysql ocf:heartbeat:mysql \
  params additional_parameters="--bind-address=172.20.10.8" \
    config="/etc/my.cnf" \
    pid="/var/run/mariadb/mariadb.pid" \
    socket="/var/lib/mysql/mysql.sock" \
    log="/var/log/mariadb/mariadb.log" \
  op monitor interval="20s" timeout="10s" \
  op start timeout="120s" \
  op stop timeout="120s"
group g_mysql p_ip_mysql p_fs_mysql p_mysql
ms ms_drbd_mysql p_drbd_mysql \
  meta notify="true" clone-max="2"
colocation c_mysql_on_drbd inf: g_mysql ms_drbd_mysql:Master
order o_drbd_before_mysql inf: ms_drbd_mysql:promote g_mysql:start
commit

Other Commands

Change a node to a secondary on a specific resource (eg: mysql):

drbdadm secondary mysql

To show the current role of a node on a specific resource:

drbdadm role mysql

See Also

• https://drbd.linbit.com/users-guide/s-build-from-source.html
• http://unix-linux-server.blogspot.ca/2014/10/drbd-on-centos-70-oracle-linux-70-and.html
• http://wiki.centos.org/HowTos/Ha-Drbd (CentOS 5)
• http://wiki.openvz.org/HA_cluster_with_DRBD_and_Heartbeat