HACMP NFS CROSS MOUNT

一.NFS CROSS MOUNT实战
一次一套系统搭建HA配置中需要配置NFS CROSS MOUNT,那么有人会问了,什么是NFS CROSS MOUNT?NFS CROSS MOUNT到底实现并解决了什么功能呢?传统的NFS分为NFS SERVER和CLIENT,server端提供NFS挂载目录抛出转换成ip形式供客户端挂载,客户端则是直接启动服务挂载ip:文件系统形式即可,那 么NFS CROSS MOUNT就可以理解为双NFS SERVER主备模式的提供NFS服务的HACMP中的冗余NFS模式。

通过smit hacmp方式我们可以看到:

Change/Show All Resources and Attributes for a Resource Group

Type or select values in entry fields.
Press Enter AFTER making all desired changes.

[Entry Fields]
Resource Group Name rg_sap_ppi_nfs
Participating Nodes (Default Node Priority) MPIPRDA MPIPRDB

Startup Policy Online On Home Node Only
Fallover Policy Fallover To Next Priority Node In The List
Fallback Policy Never Fallback

Service IP Labels/Addresses [MPINFS] +
Application Controllers [] +

Volume Groups [ppinfsvg ] +
Use forced varyon of volume groups, if necessary false +
Automatically Import Volume Groups false +

Filesystems (empty is ALL for VGs specified) [ ] +
Filesystems Consistency Check fsck +
Filesystems Recovery Method parallel +
Filesystems mounted before IP configured false +

Filesystems/Directories to Export (NFSv2/3) [/export/sapmnt /export/usr/sap/trans /export/forsap] +
Filesystems/Directories to Export (NFSv4) [] +
Stable Storage Path (NFSv4) [] +
Filesystems/Directories to NFS Mount [/sapmnt;/export/sapmnt /usr/sap/trans;/export/usr/sap/trans /forsap;/export/forsap]
Network For NFS Mount [net_ether_01] +

Tape Resources [] +
Raw Disk PVIDs [] +

Primary Workload Manager Class [] +
Secondary Workload Manager Class [] +

Miscellaneous Data []
WPAR Name [] +
F1=Help F2=Refresh F3=Cancel F4=List
Esc+5=Reset Esc+6=Command Esc+7=Edit Esc+8=Image
Resource Groups
再次用df -g查看:
MPIPRDA:/dev#df -g
Filesystem GB blocks Free %Used Iused %Iused Mounted on
/dev/hd4 5.00 4.90 3% 6779 1% /
/dev/hd2 12.00 9.48 21% 60445 3% /usr
/dev/hd9var 10.00 9.87 2% 1022 1% /var
/dev/hd3 10.00 9.96 1% 742 1% /tmp
/dev/hd1 10.00 10.00 1% 25 1% /home
/dev/hd11admin 0.50 0.50 1% 9 1% /admin
/proc – – – – – /proc
/dev/hd10opt 4.00 3.81 5% 2484 1% /opt
/dev/livedump 0.50 0.50 1% 4 1% /var/adm/ras/livedump
/dev/lvcmbc_admin 20.00 19.81 1% 15 1% /cmbc_admin
/dev/lvdb2 20.00 19.92 1% 13 1% /db2
/dev/lvopenv 20.00 11.83 41% 4201 1% /usr/openv
/dev/lvprecise 5.00 4.98 1% 4 1% /precise
/aha – – – 70 1% /aha
/dev/lvforsap 300.00 299.45 1% 4 1% /export/forsap
/dev/lvtrans 100.00 99.59 1% 4 1% /export/usr/sap/trans
/dev/lvsapmnt 80.00 79.68 1% 4 1% /export/sapmnt
MPINFS:/export/forsap 300.00 299.45 1% 4 1% /forsap
MPINFS:/export/usr/sap/trans 100.00 99.59 1% 4 1% /usr/sap/trans
MPINFS:/export/sapmnt 80.00 79.68 1% 4 1% /sapmnt
197.0.88.110:/vol/instmedia 1000.00 3.54 100% 633950 1% /InstMedia
/dev/lvSMDA94 10.00 9.96 1% 4 1% /usr/sap/DAA/SMDA94
/dev/lvSCS20 20.00 19.92 1% 4 1% /usr/sap/PPI/SCS20
/dev/lvASCS10 20.00 19.92 1% 4 1% /usr/sap/PPI/ASCS10
/dev/lvSMDA96 10.00 9.96 1% 4 1% /usr/sap/DAA/SMDA96

二节点df -g状态:
MPIPRDB:/#df -g
Filesystem GB blocks Free %Used Iused %Iused Mounted on
/dev/hd4 5.00 4.87 3% 6874 1% /
/dev/hd2 12.00 9.39 22% 60569 3% /usr
/dev/hd9var 10.00 9.81 2% 1038 1% /var
/dev/hd3 10.00 8.52 15% 2764 1% /tmp
/dev/hd1 10.00 9.28 8% 4416 1% /home
/dev/hd11admin 0.50 0.50 1% 9 1% /admin
/proc – – – – – /proc
/dev/hd10opt 4.00 3.81 5% 2484 1% /opt
/dev/livedump 0.50 0.50 1% 4 1% /var/adm/ras/livedump
/dev/lvcmbc_admin 20.00 19.81 1% 15 1% /cmbc_admin
/dev/lvdb2 20.00 17.71 12% 5728 1% /db2
/dev/lvopenv 10.00 3.43 66% 4212 1% /usr/openv
/dev/lvprecise 5.00 4.98 1% 4 1% /precise
/aha – – – 70 1% /aha
197.0.88.110:/vol/instmedia 1000.00 2.33 100% 636547 1% /InstMedia
MPINFS:/export/forsap 300.00 271.37 10% 2739 1% /forsap
MPINFS:/export/sapmnt 80.00 76.49 5% 10171 1% /sapmnt
MPINFS:/export/usr/sap/trans 100.00 99.46 1% 21 1% /usr/sap/trans
/dev/lvdb2PPI 20.00 15.77 22% 55 1% /db2/PPI
/dev/lvarc_log 200.00 149.36 26% 307 1% /db2/PPI/arc_log
/dev/lvlog_dir 100.00 89.35 11% 59 1% /db2/PPI/log_dir
/dev/lvsapdata1 500.00 385.52 23% 127 1% /db2/PPI/sapdata1
/dev/lvsapdata2 500.00 385.52 23% 127 1% /db2/PPI/sapdata2
/dev/lvsapdata3 500.00 385.52 23% 127 1% /db2/PPI/sapdata3
/dev/lvsapdata4 500.00 385.52 23% 127 1% /db2/PPI/sapdata4
/dev/lvSMDA95 10.00 9.96 1% 4 1% /usr/sap/DAA/SMDA95
MPIPRDB:/#

MPIPRDA:/#ifconfig -a
en1: flags=1e084863,c0<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT,CHECKSUM_OFFLOAD(ACTIVE),LARGESEND,CHAIN>
inet 2.56.0.161 netmask 0xffffff00 broadcast 2.56.0.255
inet 40.56.0.161 netmask 0xffffff00 broadcast 40.56.0.255
inet 40.56.0.164 netmask 0xffffff00 broadcast 40.56.0.255
tcp_sendspace 131072 tcp_recvspace 65536 rfc1323 0
en2: flags=1e080863,100c0<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT,CHECKSUM_OFFLOAD(ACTIVE),LARGESEND,CHAIN>
inet 40.56.0.163 netmask 0xffffff00 broadcast 40.56.0.255
inet 3.56.0.161 netmask 0xffffff00 broadcast 3.56.0.255
inet 40.56.0.166 netmask 0xffffff00 broadcast 40.56.0.255
tcp_sendspace 131072 tcp_recvspace 65536 rfc1323 0
lo0: flags=e08084b,c0<UP,BROADCAST,LOOPBACK,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT,LARGESEND,CHAIN>
inet 127.0.0.1 netmask 0xff000000 broadcast 127.255.255.255
inet6 ::1%1/0
tcp_sendspace 131072 tcp_recvspace 131072 rfc1323 1
MPIPRDA:/#cat /etc/hosts
# IBM_PROLOG_BEGIN_TAG
# This is an automatically generated prolog.
#
# bos61D src/bos/usr/sbin/netstart/hosts 1.2
#
# Licensed Materials – Property of IBM
#
# COPYRIGHT International Business Machines Corp. 1985,1989
# All Rights Reserved
#
# US Government Users Restricted Rights – Use, duplication or
# disclosure restricted by GSA ADP Schedule Contract with IBM Corp.
#
# @(#)47 1.2 src/bos/usr/sbin/netstart/hosts, cmdnet, bos61D, d2007_49A2 10/1/07 13:57:52
# IBM_PROLOG_END_TAG
#
# COMPONENT_NAME: TCPIP hosts
#
# FUNCTIONS: loopback
#
# ORIGINS: 26 27
#
# (C) COPYRIGHT International Business Machines Corp. 1985, 1989
# All Rights Reserved
# Licensed Materials – Property of IBM
#
# US Government Users Restricted Rights – Use, duplication or
# disclosure restricted by GSA ADP Schedule Contract with IBM Corp.
#
# /etc/hosts
#
# This file contains the hostnames and their address for hosts in the
# network. This file is used to resolve a hostname into an Internet
# address.
#
# At minimum, this file must contain the name and address for each
# device defined for TCP in your /etc/net file. It may also contain
# entries for well-known (reserved) names such as timeserver
# and printserver as well as any other host name and address.
#
# The format of this file is:
# Internet Address Hostname # Comments
# Internet Address can be either IPv4 or IPv6 address.
# Items are separated by any number of blanks and/or tabs. A ‘#’
# indicates the beginning of a comment; characters up to the end of the
# line are not interpreted by routines which search this file. Blank
# lines are allowed.

# Internet Address Hostname # Comments
# 192.9.200.1 net0sample # ethernet name/address
# 128.100.0.1 token0sample # token ring name/address
# 10.2.0.2 x25sample # x.25 name/address
# 2000:1:1:1:209:6bff:feee:2b7f ipv6sample # ipv6 name/address
127.0.0.1 loopback localhost # loopback (lo0) name/address
::1 loopback localhost # IPv6 loopback (lo0) name/address

40.43.192.6 NIMPBAC1
197.0.83.32 SZNIM
197.3.137.241 zwnim
197.3.137.228 TAIX

40.56.0.161 MPIPRDA
40.56.0.162 MPIPRDB

40.56.0.163 MPINFS
40.56.0.164 MPIPRD
40.56.0.165 MPIDB
40.56.0.166 MPIERS

2.56.0.161 MPIPRDA_bt1
2.56.0.162 MPIPRDB_bt1
3.56.0.161 MPIPRDA_bt2
3.56.0.162 MPIPRDB_bt2

#nbu master server
197.0.86.13 SBNBU

40.56.0.167 MPIAPPA
40.56.0.168 MPIAPPB
40.56.0.169 MPIAPPC

MPIPRDB:/#ifconfig -a
en1: flags=1e080863,c0<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT,CHECKSUM_OFFLOAD(ACTIVE),LARGESEND,CHAIN>
inet 2.56.0.162 netmask 0xffffff00 broadcast 2.56.0.255
inet 40.56.0.162 netmask 0xffffff00 broadcast 40.56.0.255
tcp_sendspace 131072 tcp_recvspace 65536 rfc1323 0
en2: flags=1e080863,100c0<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT,CHECKSUM_OFFLOAD(ACTIVE),LARGESEND,CHAIN>
inet 40.56.0.165 netmask 0xffffff00 broadcast 40.56.0.255
inet 3.56.0.162 netmask 0xffffff00 broadcast 3.56.0.255
tcp_sendspace 131072 tcp_recvspace 65536 rfc1323 0
lo0: flags=e08084b,c0<UP,BROADCAST,LOOPBACK,RUNNING,SIMPLEX,MULTICAST,GROUPRT,64BIT,LARGESEND,CHAIN>
inet 127.0.0.1 netmask 0xff000000 broadcast 127.255.255.255
inet6 ::1%1/0
tcp_sendspace 131072 tcp_recvspace 131072 rfc1323 1
MPIPRDB:/#

system mirror7.1 redbook上对于nfs mount_system参数的解释:

RECOVERY_METHOD

parallel

Parallel preferred as the recovery method for this resource group. (The default is sequential.)

EXPORT_FILESYSTEM

/nfsdir

The file system for NFS to export.

MOUNT_FILESYSTEM

“/sap;/nfsdir”

same syntax because we used it in smit to define the NFS cross mount.

Filesystems recovery method

确定文件系统恢复方法:parallel(并行)(用于快速恢复)或 sequential(串行)(缺省)。如果您有共享的嵌套文件系统,请不要将此字段设置为 parallel。这些文件系统必须按串行方式进行恢复。

注意:集群验证实用程序 clverify 不会报告文件系统和快速恢复的不一致性。

二. NFS 简介

nfs cross mount 1

NFS (network filesystem )是一种客户端/服务器端(client/server)应用,它可以在tcp/ip的网络基础上提供文件共享。

任何支持nfs的系统本身都可以做nfs客户端或服务器端,把自己本地硬盘资源分享给其他机器或从其他机器上获得硬盘资源。

NFS 服务器端通过编辑/etc/exports 文件来输出本地硬盘资源和光驱资源,并且运行MOUNTD 和NFSD 进程。任何本地硬盘上的文件,目录,文件系统都可以以读/写权限或只读权限输出。

通过mount 命令,/etc/filesystems 和 biod 进程,NFS客户端可以通过网络远程mount NFS 服务器端的硬盘或者 是光驱资源。

NFS 具有很灵活的配置参数,比如在服务器端的 /etc/exports文件和 客户端的/etc/filesystems 文件。
在服务器端可以输出各种文件,目录和文件系统;
1.read only (所有主机都只有读权限)
2.read/write (所有主机获得读写权限)
3.read mostly (在一个特殊列表里的主机获得读写权限,其他主机只有只读权限)
4.root equivalency (只有列出节点享有root访问权限)
5.host access list (可以mount 资源的主机列表)

客户端有以下mount 选项:
1.foreground (前台mount)
2.background (如果第一次mount 失败,在后台继续mount )
3.hard (一直继续试图mount ,没有过期时间)
4.soft (在进行了指定次数的mount 尝试后放弃mount)

三. 如何使nfs具有高可靠性

当NFS 与hacmo结合起来后,可以使NFS 高可用。把NFS 放入到HACMP资源组中,当hacmp接管时,允许NFS客户端
重新连接。因为NFS mount 可以设置重新试图mount的次数,所以对NFS客户端来说试透明的,客户端在HACMP接管
过程中是感觉不到的。

四. 两种nfs 在hacmp 中的应用及配置

a. NFS mount

nfs cross mount 2

当servera 宕机后, HACMP进行正常的接管过程,serverb 重新varyon sharedvg , 并且mount sharedfs 并把它输出,客户端重新mount sharedfs .

nfs cross mount 3

配置:
Change/Show Resources/Attributes for a Resource Group

Type or select values in entry fields.
Press Enter AFTER making all desired changes.

Resource Group Name shardvg
Node Relationship cascading
Site Relationship ignore
Participating Node Names / Default Node Priority servera serverb
Dynamic Node Priority []

Service IP label []
Filesystems (default is All) [/sharedfs]
Filesystems Consistency Check fsck
Filesystems Recovery Method sequential
Filesystems/Directories to Export [/sharedfs]
Filesystems/Directories to NFS mount []
Network For NFS Mount []

b. Cross -mounting

通过cross-mount 机制,一个集群(cluster)中的其他节点(node)可以通过NFS来共享文件系统。
这种机制被用在当两个节点上都有高可用性应用在运行而又要求要共享某一个文件系统,如下例:
servera 和serverb 都要共享sharevg 上的 /sharedfs ,因此server b 在本地mount /sharedfs并
把它export 出去,这样客户端都可以nfs mount serverb 上的 /sharedfs .再同时server a
也cross-mount (NFS mount) /sharedfs 文件系统。

nfs cross mount 4

当server b 发生宕机时,server a 首先umount NFS /sharedfs 文件系统,然后把sharedvg
varyon 到本地,并且把/sharedfs mount 到本地,同时把它export 出去,这样客户端又可以
访问/sharedfs 了, 过程如下:

nfs cross mount 5

配置方法:
Change/Show Resources/Attributes for a Resource Group

Type or select values in entry fields.
Press Enter AFTER making all desired changes.

Resource Group Name shardvg
Node Relationship cascading
Site Relationship ignore
Participating Node Names / Default Node Priority serverb servera
Dynamic Node Priority []

Service IP label []
Filesystems (default is All) [/sharedfs]
Filesystems Consistency Check fsck
Filesystems Recovery Method sequential
Filesystems/Directories to Export [/sharedfs]
Filesystems/Directories to NFS mount [/sharedfs]
Filesystem mountd before ip configued [ true]

五.注意事项:
a. Cross-mount 这种配置方式不支持cllockd , 因此在两个节点都对/sharedfs 进行读写访问时,无法保持数据的一致性,所以如果要使用这种方式,/shardfs 的属性就只能是只读方式(read- only )

b. cross -mount 这种配置方式只适用于casading 的资源组,而且该资源组必须要设置成IPAT方式。

c. 如果要设置cross-mount, Filesystem mountd before ip configued 必须设置为 true。

If you enjoyed this post, make sure you subscribe to my RSS feed!
此条目发表在AIX, 操作系统分类目录,贴了标签。将固定链接加入收藏夹。

发表评论

电子邮件地址不会被公开。 必填项已用*标注

*
To prove you're a person (not a spam script), type the security word shown in the picture. Click on the picture to hear an audio file of the word.
Anti-spam image