Heartbeat+DRBD+NFS高可用案例

Heartbeat+DRBD+NFS高可用案例

9.4 部署DRBD 的需求描述

9.4.1業(yè)務(wù)需求描述

  假設(shè)兩臺服務(wù)器Rserver-1/Lserver-1, 其實(shí)際IP分別為192.168.236.143（Rserver）和192.168.236.192（Lserver）

  配置目標(biāo)：兩臺服務(wù)器分別配置好DRBD服務(wù)后、實(shí)現(xiàn)在Rserver-1機(jī)器上/dev/sdb 分區(qū)上寫入數(shù)據(jù)、數(shù)據(jù)會時(shí)時(shí)的同步到Lserver-1機(jī)器上面、一旦服務(wù)器Rserver-1機(jī)器宕機(jī)或硬盤損壞導(dǎo)致數(shù)據(jù)不可用、Lserver-1機(jī)器上的數(shù)據(jù)此時(shí)是picdata-1機(jī)器的一個(gè)完整備份、當(dāng)然、不光是一個(gè)完整的備份、還可以瞬間接替壞數(shù)據(jù)或宕機(jī)的Rserver-1機(jī)器上數(shù)據(jù)的異機(jī)時(shí)時(shí)同步、從而達(dá)到數(shù)據(jù)高可用無業(yè)務(wù)影響的目的

9.4.2 DRBD部署結(jié)構(gòu)圖

1、   Drbd服務(wù)通過直連或以太網(wǎng)實(shí)時(shí)互相數(shù)據(jù)同步、

2、   兩臺存儲服務(wù)器互相備份、正常情況下兩端各提供一個(gè)主分區(qū)提供NFS使用

3、   存儲服務(wù)器之間、存儲服務(wù)器和交換機(jī)之間都是雙千兆網(wǎng)卡綁定（bonding）

4、   應(yīng)用服務(wù)器通過NFS訪問存儲

9.4.3服務(wù)主機(jī)資源規(guī)劃

9.4.5 drbd的環(huán)境配置

設(shè)置hosts文件兩臺都配置注意這里是主機(jī)名也需要改成picadata-1-1 是主機(jī)名需要改

例如：hostname picadata-1-1 如果這步?jīng)]有操作啟動服務(wù)的時(shí)候會出現(xiàn)報(bào)錯(cuò)。

echo '172.16.1.1 Rserver-1'>>/etc/hosts

echo '172.16.1.2 Lserver-1'>>/etc/hosts

[root@Lserver-1 ~]# tail -2 /etc/hosts

172.16.1.2 Rserver-1

172.16.1.1 Lserver-1

8.3.3配置服務(wù)器間心跳連接：

192.168.1.1 和192.168.1.2 兩塊網(wǎng)卡之間是通過普通網(wǎng)線直連連接的、即不通過交換機(jī)、直接把兩塊網(wǎng)卡連接在一起用于做心跳檢測

Master:

ifconfig eth3 192.168.1.1 netmask255.255.255.0

Backup:

ifconfig eth3 192.168.1.2 netmask255.255.255.0

Rserver-1 server 上添加如下主機(jī)路由

route add –host 192.168.1.2 dev eth3

####這條命令是：從picdata-1-1server 訪問192.168.1.2 走網(wǎng)卡eth3出去、作為心跳線路

echo 'route add -host 192.168.1.2 deveth3' >>/etc/rc.local

##-à加入開機(jī)自啟動配置里、這樣下次啟動后就會自動加載這個(gè)路由配置。

route -n

Lserver-1 server 上添加如下主機(jī)路由

route add –host 192.168.1.2 dev eth3

####這條命令是：從picdata-1-2server 訪問192.168.1.2 走網(wǎng)卡eth3出去、作為心跳線路

echo 'route add -host 192.168.1.1 deveth3' >>/etc/rc.local

##-à加入開機(jī)自啟動配置里、這樣下次啟動后就會自動加載這個(gè)路由配置。

9.5 開始實(shí)施部署

9.5.1硬盤進(jìn)行分區(qū)

   首先，通過fdisk，mkfs，ext3，tune2fs 等命令、對硬盤進(jìn)行分區(qū)、分區(qū)信息如下表

提示：如果生產(chǎn)環(huán)境中單個(gè)硬盤和raid的硬盤大于2Tfdisk 命令是查看不到的。

在虛擬機(jī)中添加兩塊硬盤。后面查看一下

Rserver-1查看

[root@Rserver-1 ~]# fdisk -l

Disk /dev/sda: 21.5 GB, 21474836480 bytes

255 heads, 63 sectors/track, 2610 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk identifier: 0x000486f5

   DeviceBoot      Start         End      Blocks  Id  System

/dev/sda1  *           1         64      512000   83 Linux

Partition 1 does not end on cylinder boundary.

/dev/sda2             64        2611    20458496  8e  Linux LVM

Disk /dev/sdb: 21.5 GB, 21474836480 bytes

Lserver-1查看

[root@Lserver-1 ~]# fdisk -l

Disk /dev/sda: 21.5 GB, 21474836480 bytes

255 heads, 63 sectors/track, 2610 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk identifier: 0x00087dae

   DeviceBoot      Start         End      Blocks  Id  System

/dev/sda1  *           1          64      512000  83  Linux

Partition 1 does not end on cylinder boundary.

/dev/sda2             64        2611   20458496   8e  Linux LVM

Disk /dev/sdb: 21.5 GB, 21474836480 bytes

255 heads, 63 sectors/track, 2610 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

9.5.2在master和backup中做分區(qū)操作（注：兩臺一樣）

因此、我們需要做的就是對/dev/sdb 進(jìn)行分區(qū)、需要分區(qū)具體內(nèi)容見下表

[root@Lserver-1 ~]# fdisk /dev/sdb

Device contains neither a valid DOSpartition table, nor Sun, SGI or OSF disklabel

Building a new DOS disklabel with diskidentifier 0x95767900.

Changes will remain in memory only,until you decide to write them.

After that, of course, the previouscontent won't be recoverable.

Warning: invalid flag 0x0000 of partitiontable 4 will be corrected by w(rite)

WARNING: DOS-compatible mode isdeprecated. It's strongly recommended to

        switch off the mode (command 'c') and change display units to

        sectors (command 'u').

Command (m for help): n

Command action

  e   extended

  p   primary partition (1-4)

p   ####新建一個(gè)主分區(qū)

Partition number (1-4): 1

First cylinder (1-2610, default 1):

Using default value 1

Last cylinder, +cylinders or+size{K,M,G} (1-2610, default 2610): +500M  ####大小為500M

Command (m for help): p

Disk /dev/sdb: 21.5 GB, 21474836480bytes

255 heads, 63 sectors/track, 2610cylinders

Units = cylinders of 16065 * 512 =8225280 bytes

Sector size (logical/physical): 512bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes /512 bytes

Disk identifier: 0x95767900

  Device Boot      Start         End      Blocks  Id  System

/dev/sdb1               1          65      522081  83  Linux

Command (m for help): n

Command action

  e   extended

  p   primary partition (1-4)

p

Partition number (1-4): 2

First cylinder (66-2610, default 66):

Using default value 66

Last cylinder, +cylinders or+size{K,M,G} (66-2610, default 2610): +200M ####新建一個(gè)200M

Command (m for help): p

Disk /dev/sdb: 21.5 GB, 21474836480bytes

255 heads, 63 sectors/track, 2610cylinders

Units = cylinders of 16065 * 512 =8225280 bytes

Sector size (logical/physical): 512bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes /512 bytes

Disk identifier: 0x95767900

  Device Boot      Start         End      Blocks  Id  System

/dev/sdb1               1          65      522081  83  Linux

/dev/sdb2              66          91      208845  83  Linux

Command (m for help):w  ######表示保存

如果提示

the kernel still uses the old table

The new table will be used at next reboot

上面這句話的意思是內(nèi)核還不知道你做了分區(qū)，需要重啟才能讓內(nèi)核知道，可以用如下命令讓內(nèi)核知曉

partprobe

現(xiàn)在查看一下分區(qū)的結(jié)果

[root@Lserver-1 ~]# fdisk -l

Disk /dev/sda: 21.5 GB, 21474836480bytes

255 heads, 63 sectors/track, 2610cylinders

Units = cylinders of 16065 * 512 =8225280 bytes

Sector size (logical/physical): 512bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes /512 bytes

Disk identifier: 0x00087dae

  Device Boot      Start         End      Blocks  Id  System

/dev/sda1   *          1          64      512000  83  Linux

Partition 1 does not end on cylinderboundary.

/dev/sda2              64        2611   20458496   8e  Linux LVM

Disk /dev/sdb: 21.5 GB, 21474836480bytes

255 heads, 63 sectors/track, 2610cylinders

Units = cylinders of 16065 * 512 =8225280 bytes

Sector size (logical/physical): 512bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes /512 bytes

Disk identifier: 0x95767900

  Device Boot      Start         End      Blocks  Id  System

/dev/sdb1               1          65      522081  83  Linux

/dev/sdb2              66          91      208845  83  Linux

現(xiàn)在對數(shù)據(jù)分區(qū)格式化

[root@Rserver-1~]# mkfs.ext4 /dev/sdb1

[root@Lserver-1~]# mkfs.ext4 /dev/sdb1

[root@Rserver-1~]# tune2fs -c -1 /dev/sdb1

tune2fs 1.41.12(17-May-2010)

Setting maximal mount count to -1   ####設(shè)置最大掛載次數(shù)為-1

[root@Lserver-1~]# tune2fs -c -1 /dev/sdb1

tune2fs 1.41.12(17-May-2010)

Setting maximal mount count to -1  ####設(shè)置最大掛載次數(shù)為-1

9.6、安裝前準(zhǔn)備：(Rserver-1,Lserver-1)
1、關(guān)閉iptables和SELINUX，避免安裝過程中報(bào)錯(cuò)。

# service iptables stop

# chkconfig iptables off

# setenforce 0

# vi /etc/selinux/config

---------------

SELINUX=disabled

---------------

9.6.1時(shí)間同步：

ntpdate -u asia.pool.ntp.org

9.6.2 DRBD的安裝配置：

# yum install gcc gcc-c++ make glibcflex kernel-develkernel-headers  這兩個(gè)的安裝包一定要和uname –r 的版本一定是需要一樣的。不然后面不能把drbd 加入到內(nèi)核當(dāng)中?？梢杂帽镜貀um的方式安裝。

9.6.3安裝DRBD：(Rserver-1主,Lserver-1備)

# wget http://oss.linbit.com/drbd/8.4/drbd-8.4.2.tar.gz

# tar zxvf drbd-8.4.3.tar.gz

# cd drbd-8.4.3

# ./configure --prefix=/usr/local/drbd--with-km --with-heartbeat --sysconfdir=/etc/

# make KDIR=/usr/src/kernels/2.6.32-504.16.2.el6.x86_64/

# make install

# mkdir -p /usr/local/drbd/var/run/drbd

# chkconfig --add drbd

# chkconfig drbd on

2、加載DRBD模塊:(Rserver-1主,Lserver-1備)

# modprobe drbd

查看DRBD模塊是否加載到內(nèi)核：

# lsmod |grep drbd

drbd 310172 4

libcrc32c 1246 1 drbd

3、參數(shù)配置：(Rserver-1主,Lserver-1備)

vi /etc/drbd.conf

清空文件內(nèi)容，并添加如下配置：

resource r0{

protocol C;

startup { wfc-timeout 0;degr-wfc-timeout 120;}

disk { on-io-error detach;}

net{

timeout 60;

connect-int 10;

ping-int 10;

max-buffers 2048;

max-epoch-size 2048;

}

syncer { rate 200M;} 

on Rserver-1{     #######on 后面是主機(jī)名

device /dev/drbd0;        #####指定的是一個(gè)drbd一個(gè)盤

disk /dev/sdb1;         #####本地磁盤。就是上面分區(qū)好的硬盤

address 172.16.1.1:7788;       ######內(nèi)網(wǎng)IP

meta-disk internal;

}

on Lserver-1{

device /dev/drbd0;

disk /dev/sdb1;

address 172.16.1.2:7788;

meta-disk internal;

}

}

注：請修改上面配置中的主機(jī)名、IP、和disk為自己的具體配置

4，創(chuàng)建DRBD設(shè)備并激活r0資源：(Rserver-1主,Lserver-1備)

# mknod /dev/drbd0 b 147 0

# drbdadm create-md r0

等待片刻，顯示success表示drbd塊創(chuàng)建成功

Writing meta data...

initializing activity log

NOT initializing bitmap

New drbd meta data block successfullycreated.

--== Creating metadata ==--

As with nodes, we count the total numberof devices mirrored by DRBD

at http://usage.drbd.org. 

The counter works anonymously. Itcreates a random number to identify

the device and sends that random number,along with the kernel and

DRBD version, to usage.drbd.org.

http://usage.drbd.org/cgi-bin/insert_usage.pl?

nu=716310175600466686&ru=15741444353112217792&rs=1085704704

* If you wish to opt out entirely,simply enter 'no'.

* To continue, just press [RETURN]

success

再次輸入該命令：

# drbdadm create-md r0

成功激活r0

[need to type 'yes' to confirm] yes

Writing meta data...

initializing activity log

NOT initializing bitmap

New drbd meta data block successfullycreated.

5、啟動DRBD服務(wù)：(Rserver-1主,Lserver-1備)

servicedrbd start

注：需要主從共同啟動方能生效

6、查看狀態(tài)：(Rserver-1主,Lserver-1備)

# service drbd status

drbd driver loaded OK; device status:

version: 8.4.3(api:1/proto:86-101)

GIT-hash:89a294209144b68adb3ee85a73221f964d3ee515 build by root@drbd1.corp.com,2015-05-12 21:05:41

m:res cs         ro                   ds                         p  mounted    fstype

0:r0  Connected Secondary/Secondary Inconsistent/Inconsistent  C

這里ro:Secondary/Secondary表示兩臺主機(jī)的狀態(tài)都是備機(jī)狀態(tài)，ds是磁盤狀態(tài)，顯示的狀態(tài)內(nèi)容為“Inconsistent不一致”，這是因?yàn)镈RBD無法判斷哪一方為主機(jī)，應(yīng)以哪一方的磁盤數(shù)據(jù)作為標(biāo)準(zhǔn)。
7、將drbd1.example.com主機(jī)配置為主節(jié)點(diǎn)(Rserver-1)

# drbdsetup /dev/drbd0 primary --force

8、分別查看主從DRBD狀態(tài)：(Rserver-1)主

# service drbd status

drbd driver loaded OK; device status:

version: 8.4.3(api:1/proto:86-101)

GIT-hash:89a294209144b68adb3ee85a73221f964d3ee515 build by root@Rserver-1, 2017-05-1813:40:26

m:res cs         ro                 ds                 p  mounted    fstype

0:r0  Connected  Primary/Secondary  UpToDate/UpToDate  C

（Lserver-1)備

# service drbd status

drbd driver loaded OK; device status:

version: 8.4.3(api:1/proto:86-101)

GIT-hash:89a294209144b68adb3ee85a73221f964d3ee515 build by root@Lserver-1, 2017-05-1813:38:57

m:res cs         ro                 ds                 p  mounted    fstype

0:r0  Connected  Secondary/Primary  UpToDate/UpToDate  C

ro在主從服務(wù)器上分別顯示 Primary/Secondary和Secondary/Primary
ds顯示UpToDate/UpToDate
表示主從配置成功。
9、掛載DRBD：(Rserver-1)主
從剛才的狀態(tài)上看到mounted和fstype參數(shù)為空，所以我們這步開始掛載DRBD到系統(tǒng)目錄/store

# mkfs.ext4 /dev/drbd0

# mkdir /data

# mount /dev/drbd0 /data

注：Secondary節(jié)點(diǎn)上不允許對DRBD設(shè)備進(jìn)行任何操作，包括掛載；所有的讀寫操作只能在Primary節(jié)點(diǎn)上進(jìn)行，只有當(dāng)Primary節(jié)點(diǎn)掛掉時(shí)，Secondary節(jié)點(diǎn)才能提升為Primary節(jié)點(diǎn)，并自動掛載DRBD繼續(xù)工作。
成功掛載后的DRBD狀態(tài)：（Rserver-1主）

[root@Rserver-1~]# service drbd status

drbddriver loaded OK; device status:

version:8.4.2 (api:1/proto:86-101)

GIT-hash:7ad5f850d711223713d6dcadc3dd48860321070c build by root@Rserver-1, 2017-05-1813:40:26

m:res  cs        ro                 ds                 p  mounted fstype

0:r0   Connected Primary/Secondary  UpToDate/UpToDate  C /data    ext4

9.7配置heartbeat服務(wù)

yum install heartbeat –y

9.7.1配置ha.cf

cd/usr/share/doc/heartbeat-3.0.4

ll|grep ha.cfauthkeys haresources

8.4.2.1配置ha.cf 文件

debugfile/var/log/ha-debug

logfile/var/log/ha-log

logfacility     local0

####-à以上三行為日志的配置、在你配置時(shí)一般不需要改動、

keepalive 2

deadtime 30

warntime 10

initdead 120

###-à以上四行為一些基礎(chǔ)的參數(shù)、在你配置時(shí)一般不需要改動

#       serial serialportname ...

mcast eth3225.0.0.219 694 1 0

##-à此行表示使用多播的方式、需要改動的僅有eth3 改成你的心跳線的網(wǎng)卡

auto_failbackon

node    Rserver-1 ##-à兩臺存儲server的主機(jī)名

node    Lserver-1  ##-à兩臺存儲server的主機(jī)名

crm     no

9.7.2配置authkeys

auth 3

#1 crc

#2 sha1 HI!

3 md5 Hello!

authkey文件必須為600 權(quán)限。Authkey文件中已經(jīng)說明了需要配置600 權(quán)限

#       Authentication file.  Must be mode 600

9.7.3配置haresources

添加一行文件

Rserver-1IPaddr::172.16.1.10/24/eth2 drbddisk::r0 Filesystem::/dev/drbd0::/data::ext4 killnfsd

注：該文件內(nèi)IPaddr,Filesystem等腳本存放路徑在/etc/ha.d/resource.d/下，也可在該目錄下存放服務(wù)啟動腳本（例如：mysql,www），將相同腳本名稱添加到/etc/ha.d/haresources內(nèi)容中，從而跟隨heartbeat啟動而啟動該腳本。
IPaddr::192.168.0.190/24/eth0：用IPaddr腳本配置對外服務(wù)的浮動虛擬IP
drbddisk::r0：用drbddisk腳本實(shí)現(xiàn)DRBD主從節(jié)點(diǎn)資源組的掛載和卸載
Filesystem::/dev/drbd0::/store::ext4：用Filesystem腳本實(shí)現(xiàn)磁盤掛載和卸載

Killnfsd這個(gè)為控制nfs啟動的腳本

9.7.4、編輯腳本文件killnfsd,用來重啟NFS服務(wù)：(Rserver-1,Lserver-1)

# vi/etc/ha.d/resource.d/killnfsd

killall -9nfsd; /etc/init.d/nfs restart;exit 0

賦予755執(zhí)行權(quán)限：

# chmod 755/etc/ha.d/resource.d/killnfsd

9.7.5、啟動HeartBeat服務(wù)
在兩個(gè)節(jié)點(diǎn)上啟動HeartBeat服務(wù)，先啟動（Rserver-1）：(Rserver-1,Lserver-1)

# serviceheartbeat start

# chkconfigheartbeat on

現(xiàn)在從其他機(jī)器能夠ping通虛IP 172.16.1.10，表示配置成功

9.7.6、配置NFS: (Rserver-1,Lserver-1)

編輯exports配置文件，添加以下配置：

# vi/etc/exports

/data        *(rw,no_root_squash)

9.7.7重啟NFS服務(wù)：

# servicerpcbind restart

# service nfsrestart

# chkconfigrpcbind on

# chkconfig nfsoff

注：這里設(shè)置NFS開機(jī)不要自動運(yùn)行，因?yàn)?etc/ha.d/resource.d/killnfsd 該腳本會控制NFS的啟動。

9.8、測試高可用
9.8.1、正常熱備切換
在客戶端掛載NFS共享目錄

# mount -t nfs 172.16.1.10:/store/tmp

模擬將主節(jié)點(diǎn)的heartbeat Rserver-1主節(jié)點(diǎn)服務(wù)停止

，則備節(jié)點(diǎn)Lserver-1備節(jié)點(diǎn)會立即無縫接管；

測試客戶端掛載的NFS共享讀寫正常。

此時(shí)備機(jī)（Lserver-1備）上的DRBD狀態(tài)：

如果備上面的狀態(tài)成為primary 就表示已經(jīng)切換成功。

9.8.2、異常宕機(jī)切換

首先把服務(wù)和IP全部切換回主上去。后面直接關(guān)閉主的電源

[root@Rserver-1ha.d]# /etc/init.d/heartbeat start

Starting High-Availabilityservices: INFO:  Resource is stopped

Done.

[root@Rserver-1ha.d]# ip addr list

1: lo:<LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN

    link/loopback 00:00:00:00:00:00 brd00:00:00:00:00:00

    inet 127.0.0.1/8 scope host lo

    inet6::1/128 scope host

       valid_lft forever preferred_lft forever

2: eth0:<BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen1000

    link/ether 00:0c:29:20:dc:da brdff:ff:ff:ff:ff:ff

    inet 192.168.236.143/24 brd 192.168.236.255scope global eth0

    inet 192.168.236.10/24 brd 192.168.236.255scope global secondary eth0

    inet6 fe80::20c:29ff:fe20:dcda/64 scopelink

       valid_lft forever preferred_lft forever

3: eth2:<BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen1000

    link/ether 00:0c:29:20:dc:e4 brdff:ff:ff:ff:ff:ff

    inet 172.16.1.1/24 brd 172.16.1.255 scopeglobal eth2

    inet 172.16.1.10/24 brd 172.16.1.255 scopeglobal secondary eth2

    inet6 fe80::20c:29ff:fe20:dce4/64 scopelink

       valid_lft forever preferred_lft forever

4: eth3:<BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen1000

    link/ether 00:0c:29:20:dc:ee brdff:ff:ff:ff:ff:ff

    inet 192.168.1.1/24 brd 192.168.1.255 scopeglobal eth3

    inet6 fe80::20c:29ff:fe20:dcee/64 scopelink

       valid_lft forever preferred_lft forever

5: pan0:<BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN

    link/ether 6e:5d:75:f7:48:77 brdff:ff:ff:ff:ff:ff

[root@Rserver-1ha.d]# df

Filesystem                      1K-blocks    Used Available Use% Mounted on

/dev/mapper/vg_rserver1-lv_root  18650424 4093320  13609700 24% /

tmpfs                              372156      76   372080   1% /dev/shm

/dev/sda1                          495844   34853   435391   8% /boot

/dev/sr0                          4363088 4363088         0 100% /media/CentOS_6.5_Final

/dev/drbd0                         505552   10521   468930   3% /data

[root@Rserver-1ha.d]#

已經(jīng)切換成功。現(xiàn)在測試一下直接宕機(jī)看看能不能轉(zhuǎn)換

已經(jīng)關(guān)閉了電源。查看一下備的情況吧。

[root@Lserver-1ha.d]# ip addr list

1:lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN

    link/loopback 00:00:00:00:00:00 brd00:00:00:00:00:00

    inet 127.0.0.1/8 scope host lo

    inet6 ::1/128 scope host

       valid_lft forever preferred_lft forever

2:eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast stateUP qlen 1000

    link/ether 00:0c:29:4d:f6:92 brdff:ff:ff:ff:ff:ff

    inet 192.168.236.192/24 brd 192.168.236.255scope global eth0

    inet6 fe80::20c:29ff:fe4d:f692/64 scopelink

       valid_lft forever preferred_lft forever

3:eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast stateUP qlen 1000

    link/ether 00:0c:29:4d:f6:9c brdff:ff:ff:ff:ff:ff

    inet 172.16.1.2/24 brd 172.16.1.255 scopeglobal eth2

    inet 172.16.1.10/24 brd 172.16.1.255 scopeglobal secondary eth2

    inet6 fe80::20c:29ff:fe4d:f69c/64 scopelink

       valid_lft forever preferred_lft forever

4:eth3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast stateUP qlen 1000

    link/ether 00:0c:29:4d:f6:a6 brdff:ff:ff:ff:ff:ff

    inet 192.168.1.2/24 brd 192.168.1.255 scopeglobal eth3

    inet6 fe80::20c:29ff:fe4d:f6a6/64 scopelink

       valid_lft forever preferred_lft forever

5:pan0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN

    link/ether 92:be:67:20:6e:b6 brdff:ff:ff:ff:ff:ff

[root@Lserver-1ha.d]# df

Filesystem                      1K-blocks    Used Available Use% Mounted on

/dev/mapper/vg_lserver1-lv_root  18650424 3966516  13736504 23% /

tmpfs                              372156     224   371932   1% /dev/shm

/dev/sda1                          495844   34856   435388   8% /boot

/dev/sr0                          4363088 4363088         0 100% /media/CentOS_6.5_Final

/dev/drbd0                        505552   10521   468930   3% /data

[root@Lserver-1ha.d]#

客戶端檢查一下

如上圖顯示heartbeat+DRBD+NFS已經(jīng)搭建成功。