DRBD

From Leo's Notes
This page was last edited on 1 April 2015, at 17:31.

DRBD (Distributed Replicated Block Device) is a virtual block device that is distributed across two or more machines.

Installation

On a clean installation of RHEL 7 or equivalent, run updates, and ensure that you have NTP and a recent kernel installed.

From ElRepo

Get the DRBD packages from ELRepo: 

rpm -Uvh http://www.elrepo.org/elrepo-release-7.0-2.el7.elrepo.noarch.rpm
yum install drbd84-utils kmod-drbd84

From Source

Install prerequisites: 

yum install gcc flex make autoconf m4

Get the source from their git repository. Releases from tarballs may fail depending on which kernel version you're using. 

git clone git://git.drbd.org/drbd-8.4.git
cd drbd-8.4
make
make install

The make install should have loaded the module and you can verify this: 

lsmod | grep drbd

If it isn't loaded, load it using modprobe 

modprobe drbd

Compile the userspace utilities: 

git clone git://git.drbd.org/drbd-utils.git
cd drbd-utils
./aclocal.sh
./configure --prefix=/usr --localstatedir=/var --sysconfdir=/etc
make
make install

Note: I got errors about man pages not being found. Ugh.


Configuration

This section will configure DRBD for use in a redundant MySQL setup.

DRBD syncs volumes. If you did not configure a partition or have a separate disk to mirror, you can create a file volume instead.

On all nodes: 

mkdir /var/drbd
cd /var/drbd

# Create a 50GB volume
dd if=/dev/zero of=data_volume.dat bs=1024 count=52428800

# Create a partition
echo -e "n\n\n\n\nw\nq\n" | fdisk data_volume.dat

# Create a device
losetup /dev/loop0 /var/drbd/data_volume.dat
kpartx -v -a /var/drbd/data_volume.dat
ls /dev/mapper/

mkdir -p /mnt/drbd
echo "/dev/drbd0 /mnt/drbd ext4 noauto 0 0 " >>  /etc/fstab

On all nodes, configure the DRBD config in /etc/drbd.d/*conf. You may want to disable the survey.

On all nodes, configure DRBD by creating a file in /etc/drbd.d/mysql.res with: 

resource mysql {
	disk {
		on-io-error detach;
	}

	device    /dev/drbd0;
	disk      /dev/mapper/loop0p1;
	meta-disk internal;

	on $hostname {
		address 172.20.10.10:7700;
	}

	on $hostname {
		address 172.20.10.9:7700;
	}
}

TODO: Can you just use /dev/loop0 instead?!

Be sure to rename the hostnames so that it matches the actual machine's hostname. Otherwise, when you try creating the resource, you will get an error like: 

'mysql' not defined in your config (for this host).

On all nodes, create the resource: 

drbdadm -v create-md mysql

If it complains about finding a filesystem, zero out the first few blocks using dd and try again.

On all nodes, start the resource: 

drbdadm up mysql

On one of the nodes, force it to be the primary node for the resource: 

drbdadm -- --force primary mysql

On the primary node, create a file system: 

mkfs -t xfs /dev/drbd0

You can now mount the device on the primary node: 

mount /dev/drbd0 /mnt/drbd

Setting up MySQL

yum -y install mariadb mariadb-server

mount the drbd volume to /mnt/drbd/mysql/data, then run 

mysql_install_db --datadir=/mnt/drbd/mysql/data

Set the bind address in my.cnf 

bind-address = 172.20.10.8



You'll need crmsh if you want to follow some other guides. Install it using the repository at: 

http://download.opensuse.org/repositories/network:/ha-clustering:/Stable/CentOS_CentOS-7/x86_64/

cd
wget http://download.opensuse.org/repositories/network:/ha-clustering:/Stable/CentOS_CentOS-7/x86_64/python-pssh-2.3.1-4.2.x86_64.rpm http://download.opensuse.org/repositories/network:/ha-clustering:/Stable/CentOS_CentOS-7/x86_64/pssh-2.3.1-4.2.x86_64.rpm http://download.opensuse.org/repositories/network:/ha-clustering:/Stable/CentOS_CentOS-7/x86_64/crmsh-2.1.2-1.1.x86_64.rpm
yum install *.rpm





crm configure

primitive p_ip_mysql ocf:heartbeat:IPaddr2 \
  params ip="172.20.10.8" cidr_netmask="24" \
  op monitor interval="30s"
primitive p_drbd_mysql ocf:linbit:drbd \
  params drbd_resource="mysql" \
  op start timeout="90s" \
  op stop timeout="180s" \
  op promote timeout="180s" \
  op demote timeout="180s" \
  op monitor interval="30s" role="Slave" \
  op monitor interval="29s" role="Master"
primitive p_fs_mysql ocf:heartbeat:Filesystem \
  params device="/dev/drbd1" \
    directory="/mnt/drbd" \
    fstype="xfs" \
    options="relatime" \
  op start timeout="60s" \
  op stop timeout="180s" \
  op monitor interval="60s" timeout="60s"
primitive p_mysql ocf:heartbeat:mysql \
  params additional_parameters="--bind-address=172.20.10.8" \
    config="/etc/my.cnf" \
    pid="/var/run/mariadb/mariadb.pid" \
    socket="/var/lib/mysql/mysql.sock" \
    log="/var/log/mariadb/mariadb.log" \
  op monitor interval="20s" timeout="10s" \
  op start timeout="120s" \
  op stop timeout="120s"
group g_mysql p_ip_mysql p_fs_mysql p_mysql
ms ms_drbd_mysql p_drbd_mysql \
  meta notify="true" clone-max="2"
colocation c_mysql_on_drbd inf: g_mysql ms_drbd_mysql:Master
order o_drbd_before_mysql inf: ms_drbd_mysql:promote g_mysql:start
commit


Other Commands

Change a node to a secondary on a specific resource (eg: mysql): 

drbdadm secondary mysql

To show the current role of a node on a specific resource: 

drbdadm role mysql
Various Linux Notes
Linux Tools and Utilites
Package Management
Linux Distributions
Networking
Containers

See Also

Various Linux Notes
Linux Tools and Utilites
Package Management
Linux Distributions
Networking
Containers