DRBD(Distributed Replicated Block Device)分布式复制块设备，这是一种基于软件、无共享、复制的解决方案。在服务器之间的块设备（包括硬盘、分区、逻辑卷）进行镜像。也
.
DRBD(Distributed Replicated Block Device) 分布式复制块设备，这是一种基于软件、无共享、复制的解决方案。在服务器之间的块设备（包括硬盘、分区、逻辑卷）进行镜像。也就是说当某一个应用程序完成写操作后，它提交的数据不仅仅会保存在本地块设备上，DRBD也会将这份数据复制一份，通过网络传输到另一个节点的块设备上，这样，两个节点上的块设备上的数据将会保存一致，这就是镜像功能。
.
环境说明
| 主机1 | 主机2 | |
|---|---|---|
| hostname | s1 | s2 |
| IP | 192.168.1.202 | 192.168.1.203 |
| DRDB |
基础环境配置
设置主机名
[root@s1 /]# hostname s1[root@s1 /]#vim /etc/sysconfig/network HOSTNAME=s1[root@s2 /]# hostname s2[root@s2 /]#vim /etc/sysconfig/network HOSTNAME=s2 关闭防火墙和selinux
s1,s2:
service iptables stopsetenforce 0sed -i 's/SELINUX=enforcing/SELINUX=disabled/' /etc/selinux/confighost设置
s1,s2:
cat /etc/hosts192.168.1.202 s1192.168.1.203 s2内核验证
验证内核是否可用，执行下面命令，如果返回非0，需要升级，如果为0，跳过下一步。
s1,s2:
ls /usr/src/kernels/`uname -r`/ >/dev/null &&echo $?内核升级
s1,s2:
yum clean all && yum makecacheyum groupinstall "Development tools" "Server Platform Development" yum update yum install kernel-devel kernel-headers flex升级完执行[内核验证]的命令，必要情况需要重启。
安装
官方下载仓库地址： http://oss.linbit.com/drbd/
本文使用 8.4.4 版本 http://oss.linbit.com/drbd/8.4/drbd-8.4.4.tar.gz
s1,s2:
tar xf drbd-8.4.3.tar.gzcd drbd-8.4.3./configure --prefix=/usr/local/drbd --sysconfdir=/etc/drbd /--datarootdir=/usr/local/share/ --with-utils --with-km make KDIR=/usr/src/kernels/`uname -r`/make installmkdir -p /usr/local/drbd/var/run/drbdcp -a /etc/drbd/rc.d/init.d/drbd /etc/rc.d/init.d/chkconfig --add drbdchkconfig drbd oncd drbdmake cleanmake KDIR=/usr/src/kernels/`uname -r`/cp drbd.ko /lib/modules/`uname -r`/kernel/lib/加载模块
s1,s2:
[root@s1 drbd]# modprobe drbd[root@s1 drbd]# lsmod |grep drbddrbd 326986 0 libcrc32c 1246 1 drbd[root@s1 drbd]# ln -sd /usr/local/drbd/sbin/drbd-overview /usr/local/sbin/准备同步所用的磁盘
因为测试，我直接从虚拟机加载了一块.（因为我已经加好了一块，此文新加一个/dev/sdc仅作为演示，后面配置还是使用/dev/sdb）
#fdisk -lDisk /dev/sdc: 321 MB, 321912832 bytes255 heads, 63 sectors/track, 39 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk identifier: 0x00000000分区
s1,s2:
[root@s1 drbd]# fdisk /dev/sdc <------------------命令在此Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabelBuilding a new DOS disklabel with disk identifier 0x2e8e99bb.Changes will remain in memory only, until you decide to write them.After that, of course, the previous content won't be recoverable.Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)WARNING: DOS-compatible mode is deprecated. It's strongly recommended to switch off the mode (command 'c') and change display units to sectors (command 'u').Command (m for help): m <------------------命令在此Command action a toggle a bootable flag b edit bsd disklabel c toggle the dos compatibility flag d delete a partition l list known partition types m print this menu n add a new partition o create a new empty DOS partition table p print the partition table q quit without saving changes s create a new empty Sun disklabel t change a partitions system id u change display/entry units v verify the partition table w write table to disk and exit x extra functionality (experts only)Command (m for help): n <------------------命令在此Command action e extended p primary partition (1-4)pPartition number (1-4): 1 <------------------命令在此First cylinder (1-39, default 1): <------------------命令在此Using default value 1Last cylinder, +cylinders or +size{K,M,G} (1-39, default 39): <------------------命令在此Using default value 39Command (m for help): p <------------------命令在此Disk /dev/sdc: 321 MB, 321912832 bytes255 heads, 63 sectors/track, 39 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk identifier: 0x2e8e99bb Device Boot Start End Blocks Id System/dev/sdc1 1 39 313236 83 LinuxCommand (m for help): w <------------------命令在此The partition table has been altered!Calling ioctl() to re-read partition table.Syncing disks.[root@s1 drbd]# fdisk -l <------------------命令在此Disk /dev/sdc: 321 MB, 321912832 bytes255 heads, 63 sectors/track, 39 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk identifier: 0x2e8e99bb Device Boot Start End Blocks Id System/dev/sdc1 1 39 313236 83 Linux[root@s2 drbd.d]# partx /dev/sdC <------------------命令在此# 1: 63- 417689 ( 417627 sectors, 213 MB)# 2: 0- -1 ( 0 sectors, 0 MB)# 3: 0- -1 ( 0 sectors, 0 MB)# 4: 0- -1 ( 0 sectors, 0 MB)DRBD配置
配置DRBD配置文件
drbd的主配置文件为 /etc/drbd/drbd.conf；为了管理的便捷性，目前通常会将些配置文件分成多个部分，且都保存至/etc/drbd/drbd.d目录中，主配置文件中仅使用"include"指令将这些配置文件片断整合起来。通常，/etc/drbd/drbd.d目录中的配置文件为global_common.conf和所有以.res结尾的文件。其中global_common.conf中主要定义global段和common段，而每一个.res的文件用于定义一个资源。
在配置文件中，global段仅能出现一次，且如果所有的配置信息都保存至同一个配置文件中而不分开为多个文件的话，global段必须位于配置文件的最开始处。目前global段中可以定义的参数仅有minor-count, dialog-refresh, disable-ip-verification和usage-count。
common段则用于定义被每一个资源默认继承的参数，可以在资源定义中使用的参数都可以在common段中定义。实际应用中，common段并非必须，但建议将多个资源共享的参数定义为common段中的参数以降低配置文件的复杂度。
s1:
vim /etc/drbd/drbd.d/global_common.conf global { usage-count no;}common { protocol C; handlers { pri-on-incon-degr "/usr/lib/drbd/notify-pri-on-incon-degr.sh; /usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ; reboot -f"; pri-lost-after-sb "/usr/lib/drbd/notify-pri-lost-after-sb.sh; /usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ; reboot -f"; local-io-error "/usr/lib/drbd/notify-io-error.sh; /usr/lib/drbd/notify-emergency-shutdown.sh; echo o > /proc/sysrq-trigger ; halt -f"; } startup { wfc-timeout 120; degr-wfc-timeout 120; } disk { on-io-error detach; fencing resource-only; } net { cram-hmac-alg "sha1"; shared-secret "mydrbdlab"; } syncer { rate 100M; }}vi myres1.res resource myres1 { #定义资源的名字 on s1 { #主机名 device /dev/drbd0; #drbd设备名称 disk /dev/sdb1; #drbd0使用的磁盘 address 192.168.1.202:7789; #drbd监听地址与端口 meta-disk internal; } on s2 { device /dev/drbd0; disk /dev/sdb1; address 192.168.1.203:7789; meta-disk internal; }}将配置文件拷贝到另一台机器完全对端配置
scp /etc/drbd/drbd.d/myres1.res /etc/drbd/drbd.d/global_common.conf root@s2:/etc/drbd/drbd.d/启动
s1,s2:
资源初始化
[root@s1 drbd.d]# drbdadm create-md myres1 Writing meta data...initializing activity logNOT initializing bitmapNew drbd meta data block successfully created.success启动服务
s1,s2:
[root@s1 drbd.d]# service drbd startStarting DRBD resources: [ create res: myres1 prepare disk: myres1 adjust disk: myres1 adjust net: myres1]outdated-wfc-timeout has to be shorter than degr-wfc-timeoutoutdated-wfc-timeout implicitly set to degr-wfc-timeout (120s)[root@s1 drbd.d]# service drbd status #或 drbdadm role myres1 或 cat /proc/drbd 或 drbd-overviewdrbd driver loaded OK; device status:version: 8.4.4 (api:1/proto:86-101)GIT-hash: 74402fecf24da8e5438171ee8c19e28627e1c98a build by root@s1, 2017-05-07 00:05:41m:res cs ro ds p mounted fstype0:myres1 Connected Secondary/Secondary UpToDate/UpToDate C此时，默认两端都是Secondary状态。接下来需要设置两端的主次关系.
置主
s1：
将s1设为主节点。第一次设置主节点时，需要使用--force参数来初始化设备同步，之后切换节点则不在需要
[root@s1 drbd.d]# drbdadm -- --overwrite-data-of-peer primary myres1[root@s1 drbd.d]# service drbd status drbd driver loaded OK; device status:version: 8.4.4 (api:1/proto:86-101)GIT-hash: 74402fecf24da8e5438171ee8c19e28627e1c98a build by root@s1, 2017-05-07 00:05:41m:res cs ro ds p mounted fstype0:myres1 Connected Primary/Secondary UpToDate/UpToDate C查看日志可看到
May 7 01:32:15 s1 kernel: block drbd0: role( Secondary -> Primary )
对方日志也接收到消息
May 7 11:43:55 s2 kernel: block drbd0: peer( Secondary -> Primary )
创建文件系统
s1:
注意，只需要在当前主节点操作。
[root@s1 drbd.d]# mke2fs -t ext4 -L drbdmyres1 /dev/drbd0 mke2fs 1.41.12 (17-May-2010)文件系统标签=drbdmyres1操作系统:Linux块大小=1024 (log=0)分块大小=1024 (log=0)Stride=0 blocks, Stripe width=0 blocks52208 inodes, 208768 blocks10438 blocks (5.00%) reserved for the super user第一个数据块=1Maximum filesystem blocks=6737100826 block groups8192 blocks per group, 8192 fragments per group2008 inodes per groupSuperblock backups stored on blocks: 8193, 24577, 40961, 57345, 73729, 204801正在写入inode表: 完成 Creating journal (4096 blocks): 完成Writing superblocks and filesystem accounting information: 完成This filesystem will be automatically checked every 24 mounts or180 days, whichever comes first. Use tune2fs -c or -i to override.[root@s1 drbd.d]# mkdir /myres1 #创建同步文件夹 [root@s1 drbd.d]# mount /dev/drdrbd0 dri/ [root@s1 drbd.d]# mount /dev/drbd0 /myres1/ #挂载[root@s1 drbd.d]# df -h Filesystem Size Used Avail Use% Mounted on/dev/mapper/vg_localhost-lv_root 18G 2.9G 14G 18% /tmpfs 186M 0 186M 0% /dev/shm/dev/sda1 477M 54M 398M 12% /boot/dev/drbd0 194M 1.8M 182M 1% /myres1[root@s1 drbd.d]# cd /myres1/ && touch IamS1 #创建测试数据[root@s1 myres1]# ll /myres1/总用量 12-rw-r--r-- 1 root root 0 5月 7 01:43 IamS1drwx------ 2 root root 12288 5月 7 01:39 lost+found[root@s1 myres1]# df -h Filesystem Size Used Avail Use% Mounted on/dev/mapper/vg_localhost-lv_root 18G 2.9G 14G 18% /tmpfs 186M 0 186M 0% /dev/shm/dev/sda1 477M 54M 398M 12% /boot/dev/drbd0 194M 1.8M 182M 1% /myres1将主节点置从
[root@s1 ~]# umount /dev/drbd0 [root@s1 ~]# drbdadm secondary myres1注：一定不能在 /myres1 当前目录操作，会报错
umount: /myres1: device is busy.
(In some cases useful info about processes that use
the device is found by lsof(8) or fuser(1))
坑了我好长时间。
将从节点置主
[root@s2 drbd.d]# drbdadm primary myres1[root@s2 drbd.d]# mount /dev/drbd0 /myres1/[root@s2 myres1]# cd /myres1/ && lltotal 12-rw-r--r-- 1 root root 0 May 7 01:43 IamS1drwx------ 2 root root 12288 May 7 01:39 lost+found
s1的数据已经同步到s2了。
