HACMP7.1修改SERVICE IP LABEL出现ST_RP_FAILED状态

smittty hacmp -> problem detection tools -> view current state

ST_INIT: cluster configured and down 二边cluster都停止时候 虽然subsystem启动但是状态是init
ST_JOINING: node joining the cluster 节点走在加入cluster
ST_VOTING: Inter-node decision state for an event
ST_RP_RUNNING: cluster running recovery program
ST_BARRIER: clstrmgr waiting at the barrier statement 正在启动操作中
ST_CBARRIER: clstrmgr is exiting recovery program
ST_UNSTABLE: cluster unstable
NOT_CONFIGURED: HA installed but not configured 没有配置ha资源组
ST_RP_FAILED: event script failed
ST_STABLE: cluster services are running with managed resources (stable cluster) or cluster services have been “forced” down with resource groups potentially in the UNMANAGED state (HACMP 5.4 only)

根据需求需要修改service ip label MFNBU_SRV为MFNBU,在没有先停集群的情况下,我先修改了/etc/hosts的主机名,再停止的HA发现clRGinfo节点1的状态为error:
MFNBU01:/#cat /etc/hosts
# IBM_PROLOG_BEGIN_TAG
# This is an automatically generated prolog.
#
# bos61D src/bos/usr/sbin/netstart/hosts 1.2
#
# Licensed Materials – Property of IBM
#
# COPYRIGHT International Business Machines Corp. 1985,1989
# All Rights Reserved
#
# US Government Users Restricted Rights – Use, duplication or
# disclosure restricted by GSA ADP Schedule Contract with IBM Corp.
#
# @(#)47 1.2 src/bos/usr/sbin/netstart/hosts, cmdnet, bos61D, d2007_49A2 10/1/07 13:57:52
# IBM_PROLOG_END_TAG
#
# COMPONENT_NAME: TCPIP hosts
#
# FUNCTIONS: loopback
#
# ORIGINS: 26 27
#
# (C) COPYRIGHT International Business Machines Corp. 1985, 1989
# All Rights Reserved
# Licensed Materials – Property of IBM
#
# US Government Users Restricted Rights – Use, duplication or
# disclosure restricted by GSA ADP Schedule Contract with IBM Corp.
#
# /etc/hosts
#
# This file contains the hostnames and their address for hosts in the
# network. This file is used to resolve a hostname into an Internet
# address.
#
# At minimum, this file must contain the name and address for each
# device defined for TCP in your /etc/net file. It may also contain
# entries for well-known (reserved) names such as timeserver
# and printserver as well as any other host name and address.
#
# The format of this file is:
# Internet Address Hostname # Comments
# Internet Address can be either IPv4 or IPv6 address.
# Items are separated by any number of blanks and/or tabs. A ‘#’
# indicates the beginning of a comment; characters up to the end of the
# line are not interpreted by routines which search this file. Blank
# lines are allowed.

# Internet Address Hostname # Comments
# 192.9.200.1 net0sample # ethernet name/address
# 128.100.0.1 token0sample # token ring name/address
# 10.2.0.2 x25sample # x.25 name/address
# 2000:1:1:1:209:6bff:feee:2b7f ipv6sample # ipv6 name/address
127.0.0.1 loopback localhost # loopback (lo0) name/address
#::1 loopback localhost # IPv6 loopback (lo0) name/address

40.43.192.6 NIMPBAC1
197.0.83.32 SZNIM
197.3.137.241 zwnim
197.3.137.228 TAIX

40.43.192.136 MFNBU01
40.43.192.137 MFNBU02

40.43.192.138 MFNBU_SVR

2.43.192.136 MFNBU01_bt1
2.43.192.137 MFNBU02_bt1
3.43.192.136 MFNBU01_bt2
3.43.192.137 MFNBU02_bt2

#opscenter
40.43.192.50 NBURMAN2

“/etc/hosts” 74 lines, 2301 characters
MFNBU01:/#smit hacmp
lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk
lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk
lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk
lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk
PowerHA SystemMirror

Move cursor to desired item and press Enter.

Cluster Nodes and Networks
Cluster Applications and Resources

System Management (C-SPOC)
Problem Determination Tools
Custom Cluster Configuration

Can’t find what you are looking for ?

F1=Help F2=Refresh F3=Cancel Esc+8=Image
PowerHA SystemMirror

Move cursor to desired item and press Enter.

Cluster Nodes and Networks
Cluster Applications and Resources

F1=Help F2=Refresh Esc+8=Image
lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk
System Management (C-SPOC)

Move cursor to desired item and press Enter.

Storage
PowerHA SystemMirror Services
Communication Interfaces
Resource Group and Applications
PowerHA SystemMirror Logs
PowerHA SystemMirror File Collection Management
Security and Users
LDAP
Configure GPFS

System Management (C-SPOC)

Move cursor to desired item and press Enter.

Storage
PowerHA SystemMirror Services
PowerHA SystemMirror Services
F1=Help F2=Refresh Esc+8=Image
lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk
PowerHA SystemMirror Services

Move cursor to desired item and press Enter.

Start Cluster Services
Stop Cluster Services
Show Cluster Services
Show Cluster Release Level

PowerHA SystemMirror Services

Move cursor to desired item and press Enter.

Start Cluster Services
Stop Cluster Services
Stop Cluster Services
F1=Help F2=Refresh Esc+8=Image
lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk
Stop Cluster Services

Type or select values in entry fields.
Press Enter AFTER making all desired changes.

[Entry Fields]
* Stop now, on system restart or both now +
Stop Cluster Services on these nodes [MFNBU01] +
BROADCAST cluster shutdown? true +
F1=Help F2=Refresh F3=Cancel F4=List
* Stop now, on system restart or both now +
F1=Help F2=Refresh F4=List
Esc+5=Reset Esc+6=Command Esc+8=Image
F1=Help F2=Refresh lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk F4=List
F1=Help F2=Refresh lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk F4=List
F1=Help F2=R F4=List
Esc+5=Reset Esc+ Esc+8=Image
lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk
x x
x x
x x
x x
x x
x x
x x
x x
x x
x x
F1=Help F2=Rx x F4=List
Esc+5=Reset Esc+x x Esc+8=Image
lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk
x Stop Cluster Services on these nodes x
x x
x F1=Help F2=Refresh F3=Cancel x
F1=Help F2=Rx Esc+7=Select Esc+8=Image Esc+0=Exit x F4=List
lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk
x Stop Cluster Services on these nodes x
x x
x Move cursor to desired item and press Esc+7. x
x ONE OR MORE items can be selected. x
x Press Enter AFTER making all selections. x
x x
x MFNBU01 x
x > MFNBU01 x

COMMAND STATUS

Command: running stdout: no stderr: no

Before command completion, additional instructions may appear below.

Broadcast message from root@MFNBU01 (tty) at 09:50:10 …

PowerHA SystemMirror on MFNBU01 shutting down. Please exit any cluster applications…
COMMAND STATUS

Command: running stdout: yes stderr: no

Before command completion, additional instructions may appear below.

MFNBU01: 0513-044 The clevmgrdES Subsystem was requested to stop.
MFNBU01: Mar 25 2015 09:50:10 /usr/es/sbin/cluster/utilities/clstop: called with flags -N -g
MFNBU02: 0513-044 The clevmgrdES Subsystem was requested to stop.
MFNBU02: Mar 25 2015 09:50:11 /usr/es/sbin/cluster/utilities/clstop: called with flags -N -g
lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk
COMMAND STATUS

Command: OK stdout: yes stderr: no

Before command completion, additional instructions may appear below.

MFNBU01: 0513-044 The clevmgrdES Subsystem was requested to stop.
MFNBU01: Mar 25 2015 09:50:10 /usr/es/sbin/cluster/utilities/clstop: called with flags -N -g
MFNBU02: 0513-044 The clevmgrdES Subsystem was requested to stop.
F1=Help F2=Refresh F3=Cancel Esc+6=Command
Esc+8=Image Esc+9=Shell Esc+0=Exit /=Find
COMMAND STATUS

Command: OK stdout: yes stderr: no

Before command completion, additional instructions may appear below.

MFNBU01:/#vi /etc/hosts
# IBM_PROLOG_BEGIN_TAG
# This is an automatically generated prolog.
#
# bos61D src/bos/usr/sbin/netstart/hosts 1.2
#
# Licensed Materials – Property of IBM
#
# COPYRIGHT International Business Machines Corp. 1985,1989
# All Rights Reserved
#
# US Government Users Restricted Rights – Use, duplication or
# disclosure restricted by GSA ADP Schedule Contract with IBM Corp.
#
# @(#)47 1.2 src/bos/usr/sbin/netstart/hosts, cmdnet, bos61D, d2007_49A2 10/1/07 13:57:52
# IBM_PROLOG_END_TAG
#
# COMPONENT_NAME: TCPIP hosts
#
# FUNCTIONS: loopback
#
# ORIGINS: 26 27
#
# (C) COPYRIGHT International Business Machines Corp. 1985, 1989
# All Rights Reserved
# Licensed Materials – Property of IBM
#
# US Government Users Restricted Rights – Use, duplication or
# disclosure restricted by GSA ADP Schedule Contract with IBM Corp.
#
# /etc/hosts
#
# This file contains the hostnames and their address for hosts in the
# network. This file is used to resolve a hostname into an Internet
# address.
#
# At minimum, this file must contain the name and address for each
# device defined for TCP in your /etc/net file. It may also contain
# entries for well-known (reserved) names such as timeserver
# and printserver as well as any other host name and address.
#
# The format of this file is:
# Internet Address Hostname # Comments
# Internet Address can be either IPv4 or IPv6 address.
# Items are separated by any number of blanks and/or tabs. A ‘#’
# indicates the beginning of a comment; characters up to the end of the
# line are not interpreted by routines which search this file. Blank
# lines are allowed.

# Internet Address Hostname # Comments
# 192.9.200.1 net0sample # ethernet name/address
# 128.100.0.1 token0sample # token ring name/address
# 10.2.0.2 x25sample # x.25 name/address
# 2000:1:1:1:209:6bff:feee:2b7f ipv6sample # ipv6 name/address
127.0.0.1 loopback localhost # loopback (lo0) name/address

40.43.192.6 NIMPBAC1

197.0.83.32 SZNIM

197.3.137.241 zwnim

197.3.137.228 TAIX

40.43.192.136 MFNBU01

40.43.192.137 MFNBU02

40.43.192.138 MFNBU

2.43.192.136 MFNBU01_bt1

2.43.192.137 MFNBU02_bt1

3.43.192.136 MFNBU01_bt2

3.43.192.137 MFNBU02_bt2

#opscenter

#opscenter
“/etc/hosts” 74 lines, 2301 characters
MFNBU01:/#lsha
Current state: ST_RP_FAILED
sccsid = “@(#)36 1.135.7.2 src/43haes/usr/sbin/cluster/hacmprd/main.C, hacmp.pe, 61haes_r711, 1225A_hacmp711 5/22/12 11:46:10”
build = “Dec 5 2012 11:50:45 1241C_hacmp711”
i_local_nodeid 0, i_local_siteid -1, my_handle 1
ml_idx[1]=0 ml_idx[2]=1
tp is 20433718
Events on event queue:
te_type 4, te_nodeid 1, te_network -1
te_type 4, te_nodeid 2, te_network -1
te_type 36, te_nodeid 1, te_network 1
te_type 11, te_nodeid 1, te_network -1
te_type 11, te_nodeid 2, te_network -1
There are 0 events on the Ibcast queue
There are 0 events on the RM Ibcast queue
CLversion: 13
local node vrmf is 7114
cluster fix level is “4”
The following timer(s) are currently active:
Event error node list: MFNBU01
Current DNP values
DNP Values for NodeId – 1 NodeName – MFNBU01
PgSpFree = 4190764 PvPctBusy = 0 PctTotalTimeIdle = 99.886272
DNP Values for NodeId – 2 NodeName – MFNBU02
PgSpFree = 4190430 PvPctBusy = 0 PctTotalTimeIdle = 99.854695
trcOn 0, kTraceOn 0, stopTraceOnExit 0, cdNodeOn 0
Last event run was FAIL_NODE on node 2
MFNBU01:/#lsha
Current state: ST_RP_FAILED
sccsid = “@(#)36 1.135.7.2 src/43haes/usr/sbin/cluster/hacmprd/main.C, hacmp.pe, 61haes_r711, 1225A_hacmp711 5/22/12 11:46:10”
build = “Dec 5 2012 11:50:45 1241C_hacmp711”
i_local_nodeid 0, i_local_siteid -1, my_handle 1
ml_idx[1]=0 ml_idx[2]=1
tp is 20433718
Events on event queue:
te_type 4, te_nodeid 1, te_network -1
te_type 4, te_nodeid 2, te_network -1
te_type 36, te_nodeid 1, te_network 1
te_type 11, te_nodeid 1, te_network -1
te_type 11, te_nodeid 2, te_network -1
There are 0 events on the Ibcast queue
There are 0 events on the RM Ibcast queue
CLversion: 13
local node vrmf is 7114
cluster fix level is “4”
The following timer(s) are currently active:
Event error node list: MFNBU01
Current DNP values
DNP Values for NodeId – 1 NodeName – MFNBU01
PgSpFree = 4190764 PvPctBusy = 0 PctTotalTimeIdle = 99.886272
DNP Values for NodeId – 2 NodeName – MFNBU02
PgSpFree = 4190430 PvPctBusy = 0 PctTotalTimeIdle = 99.854695
trcOn 0, kTraceOn 0, stopTraceOnExit 0, cdNodeOn 0
Last event run was FAIL_NODE on node 2
MFNBU01:/#lsha
Current state: ST_RP_FAILED
sccsid = “@(#)36 1.135.7.2 src/43haes/usr/sbin/cluster/hacmprd/main.C, hacmp.pe, 61haes_r711, 1225A_hacmp711 5/22/12 11:46:10”
build = “Dec 5 2012 11:50:45 1241C_hacmp711”
i_local_nodeid 0, i_local_siteid -1, my_handle 1
ml_idx[1]=0 ml_idx[2]=1
tp is 20433718
Events on event queue:
te_type 4, te_nodeid 1, te_network -1
te_type 4, te_nodeid 2, te_network -1
te_type 36, te_nodeid 1, te_network 1
te_type 11, te_nodeid 1, te_network -1
te_type 11, te_nodeid 2, te_network -1
There are 0 events on the Ibcast queue
There are 0 events on the RM Ibcast queue
CLversion: 13
local node vrmf is 7114
cluster fix level is “4”
The following timer(s) are currently active:
Event error node list: MFNBU01
Current DNP values
DNP Values for NodeId – 1 NodeName – MFNBU01
PgSpFree = 4190764 PvPctBusy = 0 PctTotalTimeIdle = 99.886272
DNP Values for NodeId – 2 NodeName – MFNBU02
PgSpFree = 4190430 PvPctBusy = 0 PctTotalTimeIdle = 99.854695
trcOn 0, kTraceOn 0, stopTraceOnExit 0, cdNodeOn 0
Last event run was FAIL_NODE on node 2
MFNBU01:/#lsha
Current state: ST_RP_FAILED
sccsid = “@(#)36 1.135.7.2 src/43haes/usr/sbin/cluster/hacmprd/main.C, hacmp.pe, 61haes_r711, 1225A_hacmp711 5/22/12 11:46:10”
build = “Dec 5 2012 11:50:45 1241C_hacmp711”
i_local_nodeid 0, i_local_siteid -1, my_handle 1
ml_idx[1]=0 ml_idx[2]=1
tp is 20433718
Events on event queue:
te_type 4, te_nodeid 1, te_network -1
te_type 4, te_nodeid 2, te_network -1
te_type 36, te_nodeid 1, te_network 1
te_type 11, te_nodeid 1, te_network -1
te_type 11, te_nodeid 2, te_network -1
There are 0 events on the Ibcast queue
There are 0 events on the RM Ibcast queue
CLversion: 13
local node vrmf is 7114
cluster fix level is “4”
The following timer(s) are currently active:
Event error node list: MFNBU01
Current DNP values
DNP Values for NodeId – 1 NodeName – MFNBU01
PgSpFree = 4190764 PvPctBusy = 0 PctTotalTimeIdle = 99.886272
DNP Values for NodeId – 2 NodeName – MFNBU02
PgSpFree = 4190430 PvPctBusy = 0 PctTotalTimeIdle = 99.854695
trcOn 0, kTraceOn 0, stopTraceOnExit 0, cdNodeOn 0
Last event run was FAIL_NODE on node 2

Command: failed stdout: yes stderr: no

Before command completion, additional instructions may appear below.

cl_clstop: ERROR: Node MFNBU01 has 5 event(s) outstanding as reported by command ‘lssrc -ls clstrmgrES’ and cannot be stopped until all outstanding events have completed
F1=Help F2=Refresh F3=Cancel Esc+6=Command
Esc+8=Image Esc+9=Shell Esc+0=Exit /=Find
COMMAND STATUS

Command: failed stdout: yes stderr: no

Before command completion, additional instructions may appear below.

MFNBU01:/#lsha
Current state: ST_RP_FAILED
sccsid = “@(#)36 1.135.7.2 src/43haes/usr/sbin/cluster/hacmprd/main.C, hacmp.pe, 61haes_r711, 1225A_hacmp711 5/22/12 11:46:10”
build = “Dec 5 2012 11:50:45 1241C_hacmp711”
i_local_nodeid 0, i_local_siteid -1, my_handle 1
ml_idx[1]=0 ml_idx[2]=1
tp is 20433718
Events on event queue:
te_type 4, te_nodeid 1, te_network -1
te_type 4, te_nodeid 2, te_network -1
te_type 36, te_nodeid 1, te_network 1
te_type 11, te_nodeid 1, te_network -1
te_type 11, te_nodeid 2, te_network -1
There are 0 events on the Ibcast queue
There are 0 events on the RM Ibcast queue
CLversion: 13
local node vrmf is 7114
cluster fix level is “4”
The following timer(s) are currently active:
Event error node list: MFNBU01
Current DNP values
DNP Values for NodeId – 1 NodeName – MFNBU01
PgSpFree = 4190764 PvPctBusy = 0 PctTotalTimeIdle = 99.886272
DNP Values for NodeId – 2 NodeName – MFNBU02
PgSpFree = 4190430 PvPctBusy = 0 PctTotalTimeIdle = 99.854695
trcOn 0, kTraceOn 0, stopTraceOnExit 0, cdNodeOn 0
Last event run was FAIL_NODE on node 2
MFNBU01:/#df -g
Filesystem GB blocks Free %Used Iused %Iused Mounted on
/dev/hd4 5.00 4.90 3% 6506 1% /
/dev/hd2 12.00 9.48 22% 60381 3% /usr
/dev/hd9var 10.00 9.87 2% 802 1% /var
/dev/hd3 10.00 9.99 1% 97 1% /tmp
/dev/hd1 10.00 9.28 8% 3634 1% /home
/dev/hd11admin 0.50 0.50 1% 9 1% /admin
/proc – – – – – /proc
/dev/hd10opt 4.00 3.84 4% 2422 1% /opt
/dev/livedump 0.50 0.50 1% 4 1% /var/adm/ras/livedump
/dev/lvcmbc_admin 10.00 9.80 2% 16 1% /cmbc_admin
/dev/lvopenv 50.00 49.80 1% 4 1% /usr/openv
/aha – – – 48 1% /aha
MFNBU01:/#lsha
Current state: ST_RP_FAILED
sccsid = “@(#)36 1.135.7.2 src/43haes/usr/sbin/cluster/hacmprd/main.C, hacmp.pe, 61haes_r711, 1225A_hacmp711 5/22/12 11:46:10”
build = “Dec 5 2012 11:50:45 1241C_hacmp711”
i_local_nodeid 0, i_local_siteid -1, my_handle 1
ml_idx[1]=0 ml_idx[2]=1
tp is 20433718
Events on event queue:
te_type 4, te_nodeid 1, te_network -1
te_type 4, te_nodeid 2, te_network -1
te_type 36, te_nodeid 1, te_network 1
te_type 11, te_nodeid 1, te_network -1
te_type 11, te_nodeid 2, te_network -1
There are 0 events on the Ibcast queue
There are 0 events on the RM Ibcast queue
CLversion: 13
local node vrmf is 7114
cluster fix level is “4”
The following timer(s) are currently active:
Event error node list: MFNBU01
Current DNP values
DNP Values for NodeId – 1 NodeName – MFNBU01
PgSpFree = 4190764 PvPctBusy = 0 PctTotalTimeIdle = 99.886272
DNP Values for NodeId – 2 NodeName – MFNBU02
PgSpFree = 4190430 PvPctBusy = 0 PctTotalTimeIdle = 99.854695
trcOn 0, kTraceOn 0, stopTraceOnExit 0, cdNodeOn 0
Last event run was FAIL_NODE on node 2
MFNBU01:/#cd /var/hacmp
MFNBU01:/var/hacmp#ls
adm clcomd clverify log odmcache
MFNBU01:/var/hacmp#cd log
MFNBU01:/var/hacmp/log#ls
autoclstrcfgmonitor.out clevmgrdevents.1 clstrmgr.debug.long.1 dnssa.log migration.log
autoverify.log clinfo.log clstrmgr.debug.long.2 domino_server.log oraappsa.log
autoverify.log.1 clinfo.rc.out clstrmgr.debug.long.3 emuhacmp.out oraclesa.log
cl2siteconfig_assist.log clstrmgr.debug clstrmgr.debug.long.4 filenetsa.log printServersa.log
cl_event_summaries.txt clstrmgr.debug.1 clstrmgr.debug.long.5 hacmp.out sa.log
cl_testtool.log clstrmgr.debug.2 clstrmgr.debug.long.6 hacmp.out.1 sapsa.log
clavan.log clstrmgr.debug.3 clstrmgr.debug.long.7 hacmprd_run_rcovcmd.debug sax.log
clconfigassist.log clstrmgr.debug.4 clutils.log hacmprd_run_rcovcmd.debug.1 tsm_admin.log
clevents clstrmgr.debug.5 cspoc.log hswizard.log tsm_client.log
clevents.1 clstrmgr.debug.6 cspoc.log.long ihssa.log tsm_server.log
clevents.2 clstrmgr.debug.7 cspoc.log.remote lsvg.err wmqsa.log
clevmgrdevents clstrmgr.debug.long dhcpsa.log maxdbsa.log
MFNBU01:/var/hacmp/log#tail -f hacmp.out
:cl_sel[144] wc -l
:cl_sel[144] 2> /dev/null
:cl_sel[144] FFDC_COUNT=’ 1′
:cl_sel[145] [ ‘ 1’ -gt 5 ]
:cl_sel[155] dspmsg scripts.cat 10059 ‘FFDC event log collection saved to /tmp/ibmsupt/hacmp/eventlogs.2015.03.25.09.50\n’ /tmp/ibmsupt/hacmp/eventlogs.2015.03.25.09.50
FFDC event log collection saved to /tmp/ibmsupt/hacmp/eventlogs.2015.03.25.09.50
:cl_sel[157] exit 0
:event_error[+137] exit 0
Mar 25 09:50:20 EVENT COMPLETED: event_error 1 TE_RG_MOVE 0

^CMFNBU01:/var/hacmp/log#ls -l
total 38120
-rw-r–r– 1 root system 16316811 Mar 25 00:00 autoclstrcfgmonitor.out
-rw-r–r– 1 root system 225013 Mar 24 15:52 autoverify.log
-rw——- 1 root system 224200 Mar 24 15:51 autoverify.log.1
-rw-r–r– 1 root system 0 Feb 16 00:00 cl2siteconfig_assist.log
-rw-r–r– 1 root system 0 Jan 24 2013 cl_event_summaries.txt
-rw-r–r– 1 root system 0 Feb 16 00:00 cl_testtool.log
-rw-r–r– 1 root system 5642 Mar 25 09:50 clavan.log
-rw-r–r– 1 root system 0 Feb 16 00:00 clconfigassist.log
-rw-r–r– 1 root system 1796 Mar 25 09:50 clevents
-rw-r–r– 1 root system 2994 Mar 24 15:51 clevents.1
-rw-r–r– 1 root system 10585 Mar 24 15:38 clevents.2
-rw-rw-rw- 1 root system 893 Mar 24 15:52 clevmgrdevents
-rw-rw-rw- 1 root system 893 Mar 24 15:46 clevmgrdevents.1
-rw-r–r– 1 root system 20484 Mar 25 09:50 clinfo.log
-rw-r–r– 1 root system 233 Mar 25 09:49 clinfo.rc.out
-rw-r–r– 1 root system 466291 Mar 25 09:50 clstrmgr.debug
-rw-r–r– 1 root system 510075 Mar 24 15:49 clstrmgr.debug.1
-rw-r–r– 1 root system 2681 Mar 24 09:56 clstrmgr.debug.2
-rw-r–r– 1 root system 2086 Mar 18 15:46 clstrmgr.debug.3
-rw-r–r– 1 root system 1931 Mar 18 15:31 clstrmgr.debug.4
-rw-r–r– 1 root system 2085 Mar 18 13:31 clstrmgr.debug.5
-rw-r–r– 1 root system 2085 Mar 18 11:28 clstrmgr.debug.6
-rw-r–r– 1 root system 1600 Oct 20 13:19 clstrmgr.debug.7
-rw-r–r– 1 root system 757279 Mar 25 09:50 clstrmgr.debug.long
-rw-r–r– 1 root system 116785 Mar 24 15:49 clstrmgr.debug.long.1
-rw-r–r– 1 root system 1437 Mar 24 00:00 clstrmgr.debug.long.2
-rw-r–r– 1 root system 815 Mar 18 15:34 clstrmgr.debug.long.3
-rw-r–r– 1 root system 815 Mar 18 13:35 clstrmgr.debug.long.4
-rw-r–r– 1 root system 815 Mar 18 11:32 clstrmgr.debug.long.5
-rw-r–r– 1 root system 3667 Mar 18 11:03 clstrmgr.debug.long.6
-rw-r–r– 1 root system 815 Oct 20 13:19 clstrmgr.debug.long.7
-rw——- 1 root system 58969 Mar 25 00:00 clutils.log
-rw-r–r– 1 root system 9934 Mar 25 09:51 cspoc.log
-rw-r–r– 1 root system 0 Feb 16 00:00 cspoc.log.long
-rw-r–r– 1 root system 0 Feb 16 00:00 cspoc.log.remote
-rw-r–r– 1 root system 0 Feb 16 00:00 dhcpsa.log
-rw-r–r– 1 root system 0 Feb 16 00:00 dnssa.log
-rw-r–r– 1 root system 0 Feb 16 00:00 domino_server.log
-rw-r–r– 1 root system 0 Feb 16 00:00 emuhacmp.out
-rw-r–r– 1 root system 0 Feb 16 00:00 filenetsa.log
-rw-r–r– 1 root system 639346 Mar 25 09:50 hacmp.out
-rw-r–r– 1 root system 870 Jan 23 2013 hacmp.out.1
-rw-r–r– 1 root system 23493 Mar 25 09:50 hacmprd_run_rcovcmd.debug
-rw-r–r– 1 root system 28095 Mar 24 15:49 hacmprd_run_rcovcmd.debug.1
-rw-r–r– 1 root system 0 Feb 16 00:00 hswizard.log
-rw-r–r– 1 root system 0 Feb 16 00:00 ihssa.log
-rw-r–r– 1 root system 0 Mar 24 15:52 lsvg.err
-rw-r–r– 1 root system 0 Feb 16 00:00 maxdbsa.log
-rw-r–r– 1 root system 0 Mar 24 15:49 migration.log
-rw-r–r– 1 root system 0 Feb 16 00:00 oraappsa.log
-rw-r–r– 1 root system 0 Feb 16 00:00 oraclesa.log
-rw-r–r– 1 root system 0 Feb 16 00:00 printServersa.log
-rw-r–r– 1 root system 0 Feb 16 00:00 sa.log
-rw-r–r– 1 root system 0 Feb 16 00:00 sapsa.log
-rw-r–r– 1 root system 0 Feb 16 00:00 sax.log
-rw-r–r– 1 root system 0 Feb 16 00:00 tsm_admin.log
-rw-r–r– 1 root system 0 Feb 16 00:00 tsm_client.log
-rw-r–r– 1 root system 0 Feb 16 00:00 tsm_server.log
-rw-r–r– 1 root system 0 Feb 16 00:00 wmqsa.log
MFNBU01:/var/hacmp/log#more hacmp.out
Warning: There is no cluster found.
cllsclstr: No cluster defined.
cllsclstr: Error reading configuration.

Reference string: Wed.Mar.25.09:50:19.BEIST.2015.release_service_addr.All_service_addrs.MFNBU_RG.ref
+MFNBU_RG:release_service_addr[+190] clgetif -a MFNBU_SVR
+MFNBU_RG:release_service_addr[+190] LC_ALL=C
MFNBU_SVR: hostname not found.
+MFNBU_RG:release_service_addr[+191] return_code=1
+MFNBU_RG:release_service_addr[+192] [ 1 -ne 0 ]
+MFNBU_RG:release_service_addr[+196] [ 1 -eq 1 ]
+MFNBU_RG:release_service_addr[+196] [[ UNDEFINED != UNDEFINED ]]
+MFNBU_RG:release_service_addr[+201] export NSORDER=
+MFNBU_RG:release_service_addr[+201] [[ true = true ]]
+MFNBU_RG:release_service_addr[+205] cl_RMupdate resource_error MFNBU_SVR release_service_addr
2015-03-25T09:50:19.952857

MFNBU01:/var/hacmp/log#smit hacmp
lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk
lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk
lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk
lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk
PowerHA SystemMirror

Move cursor to desired item and press Enter.

Cluster Nodes and Networks
Cluster Applications and Resources

System Management (C-SPOC)
Problem Determination Tools
Custom Cluster Configuration

Can’t find what you are looking for ?

F1=Help F2=Refresh F3=Cancel Esc+8=Image
PowerHA SystemMirror

Move cursor to desired item and press Enter.

Cluster Nodes and Networks
Cluster Applications and Resources
Cluster Applications and Resources
F1=Help F2=Refresh Esc+8=Image
lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk
Cluster Applications and Resources

Move cursor to desired item and press Enter.

Make Applications Highly Available (Use Smart Assists)
Resources
Resource Groups

Verify and Synchronize Cluster Configuration

Cluster Applications and Resources

Move cursor to desired item and press Enter.

Make Applications Highly Available (Use Smart Assists)
Resources
F1=Help F2=Refresh Esc+8=Image
lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk
Resource Groups

Move cursor to desired item and press Enter.

Add a Resource Group
Change/Show Nodes and Policies for a Resource Group
Change/Show Resources and Attributes for a Resource Group
Remove a Resource Group
Configure Resource Group Run-Time Policies
Show All Resources by Node or Resource Group

Verify and Synchronize Cluster Configuration

Resource Groups

Move cursor to desired item and press Enter.

Add a Resource Group
Change/Show Nodes and Policies for a Resource Group
F1=Help F2=Refresh Esc+8=Image
lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk
lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk
lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk

F1=Help F2=R Esc+8=Image
lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk
x x
x x
x x
x x
x x
x x
x x
x x
F1=Help F2=Rx x Esc+8=Image
lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk
x Change/Show Resources and Attributes for a Resource Group x
x x
x F1=Help F2=Refresh F3=Cancel x
x Esc+8=Image Esc+0=Exit Enter=Do x
lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk
x Change/Show Resources and Attributes for a Resource Group x
x x
x Move cursor to desired item and press Enter. x
x x

lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk
F1=Help F2=Refresh x Processing data … x Esc+8=Image
lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk
lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk
Change/Show All Resources and Attributes for a Resource Group

Type or select values in entry fields.
Press Enter AFTER making all desired changes.

[Entry Fields]
Resource Group Name MFNBU_RG
Participating Nodes (Default Node Priority) MFNBU01 MFNBU02

Startup Policy Online On Home Node Only
Fallover Policy Fallover To Next Priority Node In The List
Fallback Policy Never Fallback

Service IP Labels/Addresses [MFNBU_SVR] +
Application Controllers [MFNBU_AS] +

Volume Groups [vgnbu ] +
Use forced varyon of volume groups, if necessary false +
Automatically Import Volume Groups false +

Filesystems (empty is ALL for VGs specified) [/opt/VRTSnbu ] +
Filesystems Consistency Check fsck +
Filesystems Recovery Method sequential +
Filesystems mounted before IP configured false +

Filesystems/Directories to Export (NFSv2/3) [] +
Filesystems/Directories to Export (NFSv4) [] +
Stable Storage Path (NFSv4) [] +
Filesystems/Directories to NFS Mount []
Network For NFS Mount [] +

Tape Resources [] +
Raw Disk PVIDs [] +

Primary Workload Manager Class [] +
Secondary Workload Manager Class [] +

Miscellaneous Data []
WPAR Name [] +
F1=Help F2=Refresh F3=Cancel F4=List
F1=Help F2=Refresh F4=List
Esc+5=Reset Esc+6=Command Esc+8=Image
F1=Help F2=Refresh lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk F4=List
COMMAND STATUS

Command: running stdout: no stderr: no

Before command completion, additional instructions may appear below.

COMMAND STATUS

Command: running stdout: yes stderr: no

Before command completion, additional instructions may appear below.

Service label ‘MFNBU’ did not pass validation.
Please verify that this service label is properly
defined within the cluster configuration.
lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk
COMMAND STATUS

Command: failed stdout: yes stderr: no

Before command completion, additional instructions may appear below.

Service label ‘MFNBU’ did not pass validation.
Please verify that this service label is properly
F1=Help F2=Refresh F3=Cancel Esc+6=Command
Esc+8=Image Esc+9=Shell Esc+0=Exit /=Find
COMMAND STATUS

Command: failed stdout: yes stderr: no

Before command completion, additional instructions may appear below.

Change/Show All Resources and Attributes for a Resource Group

Type or select values in entry fields.
Press Enter AFTER making all desired changes.

[Entry Fields]
Resource Group Name MFNBU_RG
Participating Nodes (Default Node Priority) MFNBU01 MFNBU02

Startup Policy Online On Home Node Only
Fallover Policy Fallover To Next Priority Node In The List
Fallback Policy Never Fallback

Service IP Labels/Addresses [MFNBU] +
Application Controllers [MFNBU_AS] +

Volume Groups [vgnbu ] +
Use forced varyon of volume groups, if necessary false +
Automatically Import Volume Groups false +

Filesystems (empty is ALL for VGs specified) [/opt/VRTSnbu ] +
Filesystems Consistency Check fsck +
Filesystems Recovery Method sequential +
Filesystems mounted before IP configured false +

Filesystems/Directories to Export (NFSv2/3) [] +
Filesystems/Directories to Export (NFSv4) [] +
Stable Storage Path (NFSv4) [] +
Filesystems/Directories to NFS Mount []
Network For NFS Mount [] +

Tape Resources [] +
Raw Disk PVIDs [] +

Primary Workload Manager Class [] +
Secondary Workload Manager Class [] +

Miscellaneous Data []
WPAR Name [] +
F1=Help F2=Refresh F3=Cancel F4=List
Esc+5=Reset Esc+6=Command Esc+7=Edit Esc+8=Image
Resource Groups

Move cursor to desired item and press Enter.

Add a Resource Group
Change/Show Nodes and Policies for a Resource Group
Change/Show Resources and Attributes for a Resource Group
Remove a Resource Group
Configure Resource Group Run-Time Policies
Show All Resources by Node or Resource Group

Verify and Synchronize Cluster Configuration

Learn more about Resource Groups

Cluster Applications and Resources

Move cursor to desired item and press Enter.

Make Applications Highly Available (Use Smart Assists)
Resources
Resource Groups

Verify and Synchronize Cluster Configuration

PowerHA SystemMirror

Move cursor to desired item and press Enter.

Cluster Nodes and Networks
Cluster Applications and Resources

System Management (C-SPOC)
Problem Determination Tools
Custom Cluster Configuration

Can’t find what you are looking for ?
PowerHA SystemMirror

Move cursor to desired item and press Enter.

Cluster Nodes and Networks
MFNBU01:/var/hacmp/log#lsha
Current state: ST_RP_FAILED
sccsid = “@(#)36 1.135.7.2 src/43haes/usr/sbin/cluster/hacmprd/main.C, hacmp.pe, 61haes_r711, 1225A_hacmp711 5/22/12 11:46:10”
build = “Dec 5 2012 11:50:45 1241C_hacmp711”
i_local_nodeid 0, i_local_siteid -1, my_handle 1
ml_idx[1]=0 ml_idx[2]=1
tp is 20433718
Events on event queue:
te_type 4, te_nodeid 1, te_network -1
te_type 4, te_nodeid 2, te_network -1
te_type 36, te_nodeid 1, te_network 1
te_type 11, te_nodeid 1, te_network -1
te_type 11, te_nodeid 2, te_network -1
There are 0 events on the Ibcast queue
There are 0 events on the RM Ibcast queue
CLversion: 13
local node vrmf is 7114
cluster fix level is “4”
The following timer(s) are currently active:
Event error node list: MFNBU01
Current DNP values
DNP Values for NodeId – 1 NodeName – MFNBU01
PgSpFree = 4190764 PvPctBusy = 0 PctTotalTimeIdle = 99.886272
DNP Values for NodeId – 2 NodeName – MFNBU02
PgSpFree = 4190430 PvPctBusy = 0 PctTotalTimeIdle = 99.854695
trcOn 0, kTraceOn 0, stopTraceOnExit 0, cdNodeOn 0
Last event run was FAIL_NODE on node 2
MFNBU01:/var/hacmp/log#lsha
Current state: ST_RP_FAILED
sccsid = “@(#)36 1.135.7.2 src/43haes/usr/sbin/cluster/hacmprd/main.C, hacmp.pe, 61haes_r711, 1225A_hacmp711 5/22/12 11:46:10”
build = “Dec 5 2012 11:50:45 1241C_hacmp711”
i_local_nodeid 0, i_local_siteid -1, my_handle 1
ml_idx[1]=0 ml_idx[2]=1
tp is 20433718
Events on event queue:
te_type 4, te_nodeid 1, te_network -1
te_type 4, te_nodeid 2, te_network -1
te_type 36, te_nodeid 1, te_network 1
te_type 11, te_nodeid 1, te_network -1
te_type 11, te_nodeid 2, te_network -1
There are 0 events on the Ibcast queue
There are 0 events on the RM Ibcast queue
CLversion: 13
local node vrmf is 7114
cluster fix level is “4”
The following timer(s) are currently active:
Event error node list: MFNBU01
Current DNP values
DNP Values for NodeId – 1 NodeName – MFNBU01
PgSpFree = 4190764 PvPctBusy = 0 PctTotalTimeIdle = 99.886272
DNP Values for NodeId – 2 NodeName – MFNBU02
PgSpFree = 4190430 PvPctBusy = 0 PctTotalTimeIdle = 99.854695
trcOn 0, kTraceOn 0, stopTraceOnExit 0, cdNodeOn 0
Last event run was FAIL_NODE on node 2
MFNBU01:/var/hacmp/log#lsha
Current state: ST_RP_FAILED
sccsid = “@(#)36 1.135.7.2 src/43haes/usr/sbin/cluster/hacmprd/main.C, hacmp.pe, 61haes_r711, 1225A_hacmp711 5/22/12 11:46:10”
build = “Dec 5 2012 11:50:45 1241C_hacmp711”
i_local_nodeid 0, i_local_siteid -1, my_handle 1
ml_idx[1]=0 ml_idx[2]=1
tp is 20433718
Events on event queue:
te_type 4, te_nodeid 1, te_network -1
te_type 4, te_nodeid 2, te_network -1
te_type 36, te_nodeid 1, te_network 1
te_type 11, te_nodeid 1, te_network -1
te_type 11, te_nodeid 2, te_network -1
There are 0 events on the Ibcast queue
There are 0 events on the RM Ibcast queue
CLversion: 13
local node vrmf is 7114
cluster fix level is “4”
The following timer(s) are currently active:
Event error node list: MFNBU01
Current DNP values
DNP Values for NodeId – 1 NodeName – MFNBU01
PgSpFree = 4190764 PvPctBusy = 0 PctTotalTimeIdle = 99.886272
DNP Values for NodeId – 2 NodeName – MFNBU02
PgSpFree = 4190430 PvPctBusy = 0 PctTotalTimeIdle = 99.854695
trcOn 0, kTraceOn 0, stopTraceOnExit 0, cdNodeOn 0
Last event run was FAIL_NODE on node 2
MFNBU01:/var/hacmp/log#clRGinfo
—————————————————————————–
Group Name State Node
—————————————————————————–
MFNBU_RG ERROR MFNBU01
OFFLINE MFNBU02

MFNBU01:/var/hacmp/log#lsha
Current state: ST_RP_FAILED
sccsid = “@(#)36 1.135.7.2 src/43haes/usr/sbin/cluster/hacmprd/main.C, hacmp.pe, 61haes_r711, 1225A_hacmp711 5/22/12 11:46:10”
build = “Dec 5 2012 11:50:45 1241C_hacmp711”
i_local_nodeid 0, i_local_siteid -1, my_handle 1
ml_idx[1]=0 ml_idx[2]=1
tp is 20433718
Events on event queue:
te_type 4, te_nodeid 1, te_network -1
te_type 4, te_nodeid 2, te_network -1
te_type 36, te_nodeid 1, te_network 1
te_type 11, te_nodeid 1, te_network -1
te_type 11, te_nodeid 2, te_network -1
There are 0 events on the Ibcast queue
There are 0 events on the RM Ibcast queue
CLversion: 13
local node vrmf is 7114
cluster fix level is “4”
The following timer(s) are currently active:
Event error node list: MFNBU01
Current DNP values
DNP Values for NodeId – 1 NodeName – MFNBU01
PgSpFree = 4190764 PvPctBusy = 0 PctTotalTimeIdle = 99.886272
DNP Values for NodeId – 2 NodeName – MFNBU02
PgSpFree = 4190430 PvPctBusy = 0 PctTotalTimeIdle = 99.854695
trcOn 0, kTraceOn 0, stopTraceOnExit 0, cdNodeOn 0
Last event run was FAIL_NODE on node 2
MFNBU01:/var/hacmp/log#lsha
Current state: ST_RP_FAILED
sccsid = “@(#)36 1.135.7.2 src/43haes/usr/sbin/cluster/hacmprd/main.C, hacmp.pe, 61haes_r711, 1225A_hacmp711 5/22/12 11:46:10”
build = “Dec 5 2012 11:50:45 1241C_hacmp711”
i_local_nodeid 0, i_local_siteid -1, my_handle 1
ml_idx[1]=0 ml_idx[2]=1
tp is 20433718
Events on event queue:
te_type 4, te_nodeid 1, te_network -1
te_type 4, te_nodeid 2, te_network -1
te_type 36, te_nodeid 1, te_network 1
te_type 11, te_nodeid 1, te_network -1
te_type 11, te_nodeid 2, te_network -1
There are 0 events on the Ibcast queue
There are 0 events on the RM Ibcast queue
CLversion: 13
local node vrmf is 7114
cluster fix level is “4”
The following timer(s) are currently active:
Event error node list: MFNBU01
Current DNP values
DNP Values for NodeId – 1 NodeName – MFNBU01
PgSpFree = 4190764 PvPctBusy = 0 PctTotalTimeIdle = 99.886272
DNP Values for NodeId – 2 NodeName – MFNBU02
PgSpFree = 4190430 PvPctBusy = 0 PctTotalTimeIdle = 99.854695
trcOn 0, kTraceOn 0, stopTraceOnExit 0, cdNodeOn 0
Last event run was FAIL_NODE on node 2
MFNBU01:/var/hacmp/log#lssrc -g cluster
Subsystem Group PID Status
clstrmgrES cluster 16187542 active
clinfoES cluster 15532074 active
MFNBU01:/var/hacmp/log#stopsrc -g cluster
0513-044 The clstrmgrES Subsystem was requested to stop.
0513-044 The clinfoES Subsystem was requested to stop.
MFNBU01:/var/hacmp/log#lssrc -g cluster
Subsystem Group PID Status
clstrmgrES cluster 16187542 stopping
MFNBU01:/var/hacmp/log#clRGinfo
—————————————————————————–
Group Name State Node
—————————————————————————–
MFNBU_RG ERROR MFNBU01
OFFLINE MFNBU02

MFNBU01:/var/hacmp/log#lsha
Current state: ST_RP_FAILED
sccsid = “@(#)36 1.135.7.2 src/43haes/usr/sbin/cluster/hacmprd/main.C, hacmp.pe, 61haes_r711, 1225A_hacmp711 5/22/12 11:46:10”
build = “Dec 5 2012 11:50:45 1241C_hacmp711”
i_local_nodeid 0, i_local_siteid -1, my_handle 1
ml_idx[1]=0 ml_idx[2]=1
tp is 20433718
Events on event queue:
te_type 4, te_nodeid 1, te_network -1
te_type 4, te_nodeid 2, te_network -1
te_type 36, te_nodeid 1, te_network 1
te_type 11, te_nodeid 1, te_network -1
te_type 11, te_nodeid 2, te_network -1
There are 0 events on the Ibcast queue
There are 0 events on the RM Ibcast queue
CLversion: 13
local node vrmf is 7114
cluster fix level is “4”
The following timer(s) are currently active:
Event error node list: MFNBU01
Current DNP values
DNP Values for NodeId – 1 NodeName – MFNBU01
PgSpFree = 4190764 PvPctBusy = 0 PctTotalTimeIdle = 99.886272
DNP Values for NodeId – 2 NodeName – MFNBU02
PgSpFree = 4190430 PvPctBusy = 0 PctTotalTimeIdle = 99.854695
trcOn 0, kTraceOn 0, stopTraceOnExit 0, cdNodeOn 0
Last event run was FAIL_NODE on node 2
MFNBU01:/var/hacmp/log#lssrc -g cluster
Subsystem Group PID Status
clstrmgrES cluster 16187542 stopping
MFNBU01:/var/hacmp/log#lssrc -g cluster
Subsystem Group PID Status
clstrmgrES cluster 16187542 stopping
MFNBU01:/var/hacmp/log#clRGinfo
—————————————————————————–
Group Name State Node
—————————————————————————–
MFNBU_RG ERROR MFNBU01
OFFLINE MFNBU02

MFNBU01:/var/hacmp/log#clRGinfo
—————————————————————————–
Group Name State Node
—————————————————————————–
MFNBU_RG ERROR MFNBU01
OFFLINE MFNBU02

MFNBU01:/var/hacmp/log#lssrc -g cluster
Subsystem Group PID Status
clstrmgrES cluster 16187542 stopping
MFNBU01:/var/hacmp/log#cldump

Obtaining information via SNMP from Node: MFNBU01…

_____________________________________________________________________________
Cluster Name: MFNBU_clu
Cluster State: UP
Cluster Substate: STABLE
_____________________________________________________________________________

Node Name: MFNBU01 State: UP

Network Name: net_ether_01 State: UP

Address: 2.43.192.136 Label: MFNBU01_bt1 State: UP
Address: 3.43.192.136 Label: MFNBU01_bt2 State: UP
Address: 40.43.192.138 Label: MFNBU_SVR State: UP

Node Name: MFNBU02 State: UP

Network Name: net_ether_01 State: UP

Address: 2.43.192.137 Label: MFNBU02_bt1 State: UP
Address: 3.43.192.137 Label: MFNBU02_bt2 State: UP

Cluster Name: MFNBU_clu

Resource Group Name: MFNBU_RG
Startup Policy: Online On Home Node Only
Fallover Policy: Fallover To Next Priority Node In The List
Fallback Policy: Never Fallback
Site Policy: ignore
Node Group State
—————————- —————
MFNBU01 ERROR
MFNBU02 OFFLINE
MFNBU01:/var/hacmp/log#cldisp
Cluster: MFNBU_clu
Cluster services: active
State of cluster: up
Substate: stable

#############
APPLICATIONS
#############
Cluster MFNBU_clu provides the following applications: MFNBU_AS
Application: MFNBU_AS
MFNBU_AS is started by /hacmp/MFNBU_start.sh
MFNBU_AS is stopped by /hacmp/MFNBU_stop.sh
No application monitors are configured for MFNBU_AS.
This application is part of resource group ‘MFNBU_RG’.
Resource group policies:
Startup: on home node only
Fallover: to next priority node in the list
Fallback: never
State of MFNBU_AS: error
Nodes configured to provide MFNBU_AS: MFNBU01 {up} MFNBU02 {up}
Resources associated with MFNBU_AS:
Service Labels
MFNBU_SVR(40.43.192.138)
Interfaces configured to provide MFNBU_SVR:
MFNBU01_bt1
with IP address: 2.43.192.136
on interface: en4
on node: MFNBU01
on network: net_ether_01
MFNBU01_bt2
with IP address: 3.43.192.136
on interface: en8
on node: MFNBU01
on network: net_ether_01
MFNBU02_bt2
with IP address: 3.43.192.137
on interface: en8
on node: MFNBU02
on network: net_ether_01
MFNBU02_bt1
with IP address: 2.43.192.137
on interface: en4
on node: MFNBU02
on network: net_ether_01
Shared Volume Groups:
vgnbu

#############
TOPOLOGY
#############
MFNBU_clu consists of the following nodes: MFNBU01 MFNBU02
MFNBU01
Network interfaces:
MFNBU01_bt1
with IP address: 2.43.192.136
on interface: en4
on network: net_ether_01
MFNBU01_bt2
with IP address: 3.43.192.136
on interface: en8
on network: net_ether_01
MFNBU02
Network interfaces:
MFNBU02_bt2
with IP address: 3.43.192.137
on interface: en8
on network: net_ether_01
MFNBU02_bt1
with IP address: 2.43.192.137
on interface: en4
on network: net_ether_01
MFNBU01:/var/hacmp/log#cltopinfo
Cluster Name: MFNBU_clu
Cluster Connection Authentication Mode: Standard
Cluster Message Authentication Mode: None
Cluster Message Encryption: None
Use Persistent Labels for Communication: No
Repository Disk: hdisk11
Cluster IP Address: 238.43.192.138
There are 2 node(s) and 1 network(s) defined

NODE MFNBU01:
Network net_ether_01
MFNBU_SVR 40.43.192.138
MFNBU01_bt1 2.43.192.136
MFNBU01_bt2 3.43.192.136

NODE MFNBU02:
Network net_ether_01
MFNBU_SVR 40.43.192.138
MFNBU02_bt2 3.43.192.137
MFNBU02_bt1 2.43.192.137

Resource Group MFNBU_RG
Startup Policy Online On Home Node Only
Fallover Policy Fallover To Next Priority Node In The List
Fallback Policy Never Fallback
Participating Nodes MFNBU01 MFNBU02
Service IP Label MFNBU_SVR
MFNBU01:/var/hacmp/log#clshowsrv -a
Subsystem Group PID Status
clstrmgrES cluster 16187542 stopping
clinfoES cluster inoperative
clcomd caa 14549194 active
MFNBU01:/var/hacmp/log#cllsserv
MFNBU_AS /hacmp/MFNBU_start.sh /hacmp/MFNBU_stop.sh background
MFNBU01:/var/hacmp/log#clconfig -v -O
ksh: clconfig: not found.
MFNBU01:/var/hacmp/log#/usr/es/sbin/cluster/diag/clconfig -v -O
HACMPnode ODM on node MFNBU02 verified.

HACMPnetwork ODM on node MFNBU02 verified.

HACMPcluster ODM on node MFNBU02 verified.

HACMPnim ODM on node MFNBU02 verified.

HACMPadapter ODM on node MFNBU02 verified.

HACMPtopsvcs ODM on node MFNBU02 verified.

HACMPsite ODM on node MFNBU02 verified.

HACMPsircol ODM on node MFNBU02 verified.

HACMPeventmgr ODM on node MFNBU02 verified.

HACMPnode ODM on node MFNBU02 verified.

HACMPgroup ODM on node MFNBU02 verified.

HACMPresource ODM on node MFNBU02 verified.

HACMPserver ODM on node MFNBU02 verified.

HACMPcommadapter ODM on node MFNBU02 verified.

HACMPcommlink ODM on node MFNBU02 verified.

HACMPx25 ODM on node MFNBU02 verified.

HACMPsna ODM on node MFNBU02 verified.

HACMPevent ODM on node MFNBU02 verified.

HACMPcustom ODM on node MFNBU02 verified.

HACMPlogs ODM on node MFNBU02 verified.

HACMPtape ODM on node MFNBU02 verified.

HACMPmonitor ODM on node MFNBU02 verified.

HACMPpager ODM on node MFNBU02 verified.

HACMPport ODM on node MFNBU02 verified.

HACMPnpp ODM on node MFNBU02 verified.

HACMPude ODM on node MFNBU02 verified.

HACMPdisksubsys ODM on node MFNBU02 verified.

HACMPpprc ODM on node MFNBU02 verified.

HACMPpairtasks ODM on node MFNBU02 verified.

HACMPpathtasks ODM on node MFNBU02 verified.

HACMPercmf ODM on node MFNBU02 verified.

HACMPercmfglobals ODM on node MFNBU02 verified.

HACMPtimer ODM on node MFNBU02 verified.

HACMPsiteinfo ODM on node MFNBU02 verified.

HACMPtimersvc ODM on node MFNBU02 verified.

HACMPfilecollection ODM on node MFNBU02 verified.

HACMPfcfile ODM on node MFNBU02 verified.

HACMPrgdependency ODM on node MFNBU02 verified.

HACMPrg_loc_dependency ODM on node MFNBU02 verified.

HACMPsvc ODM on node MFNBU02 verified.

HACMPsvcpprc ODM on node MFNBU02 verified.

HACMPsvcrelationship ODM on node MFNBU02 verified.

HACMPsa_metadata ODM on node MFNBU02 verified.

HACMPcsserver ODM on node MFNBU02 verified.

HACMPoemfsmethods ODM on node MFNBU02 verified.

HACMPoemvgmethods ODM on node MFNBU02 verified.

HACMPoemvolumegroup ODM on node MFNBU02 verified.

HACMPoemfilesystem ODM on node MFNBU02 verified.

HACMPdisktype ODM on node MFNBU02 verified.

HACMPpprcconsistgrp ODM on node MFNBU02 verified.

HACMPsr ODM on node MFNBU02 verified.

HACMPtc ODM on node MFNBU02 verified.

HACMPras ODM on node MFNBU02 verified.

HACMPresourcetype ODM on node MFNBU02 verified.

HACMPudresource ODM on node MFNBU02 verified.

HACMPudres_def ODM on node MFNBU02 verified.

HACMPLDAP ODM on node MFNBU02 verified.

Verification to be performed on the following:
Cluster Topology
Cluster Resources

Retrieving data from available cluster nodes. This could take a few minutes.

Start data collection on node MFNBU01
Start data collection on node MFNBU02
Collector on node MFNBU02 completed
Collector on node MFNBU01 completed
Data collection complete
WARNING: Cluster verification detected that some cluster components are
inactive. Please use the matrix below to verify the status of
inactive components:
Node: MFNBU01
Resource Group: MFNBU_RG State: ERROR

Verifying Cluster Topology…

Completed 10 percent of the verification checks
Completed 20 percent of the verification checks
A corrective action is available for the condition reported below:

ERROR: /etc/hosts on node MFNBU01 contains IP address ‘40.43.192.138’, but it does
not map to IP label ‘MFNBU_SVR’.

To correct the above condition, run verification & synchronization with
“Automatically correct errors found during verification?” set to either ‘Yes’
or ‘Interactive’. The cluster must be down for the corrective action to run.

Corrective actions can be enabled for Verification and Synchronization in the
PowerHA SystemMirror extended Verification and Synchronization SMIT fastpath “cl_sync”.
Alternatively use the Initialization and Standard Configuration -> Verification
and Synchronization path where corrective actions are always executed in
interactive mode.
A corrective action is available for the condition reported below:

ERROR: /etc/hosts on node MFNBU02 contains IP address ‘40.43.192.138’, but it does
not map to IP label ‘MFNBU_SVR’.

To correct the above condition, run verification & synchronization with
“Automatically correct errors found during verification?” set to either ‘Yes’
or ‘Interactive’. The cluster must be down for the corrective action to run.

Corrective actions can be enabled for Verification and Synchronization in the
PowerHA SystemMirror extended Verification and Synchronization SMIT fastpath “cl_sync”.
Alternatively use the Initialization and Standard Configuration -> Verification
and Synchronization path where corrective actions are always executed in
interactive mode.
Completed 30 percent of the verification checks
Saving existing /var/hacmp/clverify/ver_mping/ver_mping.log to /var/hacmp/clverify/ver_mping/ver_mping.log.bak
Verifying clcomd communication, please be patient.

Verifying multicast communication with mping.

Verifying Cluster Resources…

Completed 40 percent of the verification checks

WARNING: Application monitors are required for detecting application failures
in order for PowerHA SystemMirror to recover from them. Application monitors are started
by PowerHA SystemMirror when the resource group in which they participate is activated.
The following application(s), shown with their associated resource group,
do not have an application monitor configured:

Application Server Resource Group
——————————– ———————————
MFNBU_AS MFNBU_RG
Completed 50 percent of the verification checks
Completed 60 percent of the verification checks
Completed 70 percent of the verification checks
Completed 80 percent of the verification checks
WARNING: MFNBU01 has an active aliased service IP label MFNBU_SVR
attached to physical interface: en8. This service IP label
is part of resource group MFNBU_RG, which is in the OFFLINE state
on node MFNBU01.
WARNING:
To ensure PowerHA SystemMirror brings the resource group online on this node:
1. It is recommended that you start cluster services on
this node using the SMIT option to “Manage Resource Groups: Manually”.
2. Once cluster services are running on this node, then use
SMIT to bring the resource group ONLINE on this node.

To bring the resource group online on another node, you must:
1. Manually bring the resources offline.
2. Using SMIT, move the resource group to the node of your choosing.
Completed 90 percent of the verification checks
Completed 100 percent of the verification checks
WARNING: Node MFNBU01 has cluster.es.nfs.rte installed however grace periods are not fully enabled on this node. Grace periods must be enabled before NFSv4 stable storage can be used.

PowerHA SystemMirror will attempt to fix this opportunistically when acquiring NFS resources on this node however the change won’t take effect until the next time that nfsd is started.

If this warning persists, the administrator should perform the following steps to enable grace periods on MFNBU01 at the next planned downtime:
1. stopsrc -s nfsd
2. smitty nfsgrcperiod
3. startsrc -s nfsd

WARNING: Node MFNBU02 has cluster.es.nfs.rte installed however grace periods are not fully enabled on this node. Grace periods must be enabled before NFSv4 stable storage can be used.

PowerHA SystemMirror will attempt to fix this opportunistically when acquiring NFS resources on this node however the change won’t take effect until the next time that nfsd is started.

If this warning persists, the administrator should perform the following steps to enable grace periods on MFNBU02 at the next planned downtime:
1. stopsrc -s nfsd
2. smitty nfsgrcperiod
3. startsrc -s nfsd

Verification exiting with error count: 2

clconfig: Verification determined 2 error(s) occurred. Please correct
any errors and retry.
MFNBU01:/var/hacmp/log#clcycle
0513-095 The request for subsystem refresh was completed successfully.
MFNBU01:/var/hacmp/log#lssrc -g cluster
Subsystem Group PID Status
clstrmgrES cluster 16187542 stopping
MFNBU01:/var/hacmp/log#clrsh
Usage: clrsh [-p] [-n] host command
MFNBU01:/var/hacmp/log#odmget HACMPlogs

HACMPlogs:
name = “clstrmgr.debug”
description = “Generated by the clstrmgr daemon”
defaultdir = “/var/hacmp/log”
value = “/var/hacmp/log”
rfs = “”

HACMPlogs:
name = “cluster.log”
description = “Generated by cluster scripts and daemons”
defaultdir = “/var/hacmp/adm”
value = “/var/hacmp/adm”
rfs = “”

HACMPlogs:
name = “cluster.mmddyyyy”
description = “Cluster history files generated daily”
defaultdir = “/var/hacmp/adm/history/”
value = “/var/hacmp/adm/history”
rfs = “”

HACMPlogs:
name = “cspoc.log”
description = “Generated by CSPOC commands”
defaultdir = “/var/hacmp/log”
value = “/var/hacmp/log”
rfs = “”

HACMPlogs:
name = “emuhacmp.out”
description = “Generated by the event emulator scripts”
defaultdir = “/var/hacmp/log”
value = “/var/hacmp/log”
rfs = “”

HACMPlogs:
name = “hacmp.out”
description = “Generated by event scripts and utilities”
defaultdir = “/var/hacmp/log”
value = “/var/hacmp/log”
rfs = “”

HACMPlogs:
name = “clavan.log”
description = “Generated by Application Availability Analysis tool”
defaultdir = “/var/hacmp/log”
value = “/var/hacmp/log”
rfs = “”

HACMPlogs:
name = “clverify.log”
description = “Generated by Cluster Verification utility”
defaultdir = “/var/hacmp/clverify”
value = “/var/hacmp/clverify”
rfs = “”

HACMPlogs:
name = “clcomd.log”
description = “Generated by clcomd daemon”
defaultdir = “/var/log/clcomd”
value = “/var/log/clcomd”
rfs = “”

HACMPlogs:
name = “clcomddiag.log”
description = “Generated by clcomd daemon, debug information”
defaultdir = “/var/log/clcomd”
value = “/var/log/clcomd”
rfs = “”

HACMPlogs:
name = “clconfigassist.log”
description = “Generated by Two-Node Cluster Configuration Assistant”
defaultdir = “/var/hacmp/log”
value = “/var/hacmp/log”
rfs = “”

HACMPlogs:
name = “cl2siteconfig_assist.log”
description = “Generated by Two-Site Cluster Configuration Assistant”
defaultdir = “/var/hacmp/log”
value = “/var/hacmp/log”
rfs = “”

HACMPlogs:
name = “clutils.log”
description = “Generated by cluster utilities and file propagation”
defaultdir = “/var/hacmp/log”
value = “/var/hacmp/log”
rfs = “”

HACMPlogs:
name = “cl_testtool.log”
description = “Generated by the Cluster Test Tool”
defaultdir = “/var/hacmp/log”
value = “/var/hacmp/log”
rfs = “”

HACMPlogs:
name = “autoverify.log”
description = “Generated by Auto Verify and Synchronize”
defaultdir = “/var/hacmp/log”
value = “/var/hacmp/log”
rfs = “”

HACMPlogs:
name = “sa.log”
description = “Generated by Application Discovery”
defaultdir = “/var/hacmp/log”
value = “/var/hacmp/log”
rfs = “”

HACMPlogs:
name = “clstrmgr.debug.long”
description = “Detail information from the clstrmgr daemon”
defaultdir = “/var/hacmp/log”
value = “/var/hacmp/log”
rfs = “”

HACMPlogs:
name = “cspoc.log.long”
description = “Detail information from CSPOC commands”
defaultdir = “/var/hacmp/log”
value = “/var/hacmp/log”
rfs = “”

HACMPlogs:
name = “cspoc.log.remote”
description = “Generated by remote node running CSPOC commands”
defaultdir = “/var/hacmp/log”
value = “/var/hacmp/log”
rfs = “”

HACMPlogs:
name = “clinfo.log”
description = “Generated by client node running clinfo”
defaultdir = “/var/hacmp/log”
value = “/var/hacmp/log”
rfs = “”

HACMPlogs:
name = “migration.log”
description = “Generated by cluster migration”
defaultdir = “/var/hacmp/log”
value = “/var/hacmp/log”
rfs = “”

HACMPlogs:
name = “dnssa.log”
description = “DNS Smart Assist Log”
defaultdir = “/var/hacmp/log”
value = “/var/hacmp/log”
rfs = “”

HACMPlogs:
name = “dhcpsa.log”
description = “DHCP Smart Assist Log”
defaultdir = “/var/hacmp/log”
value = “/var/hacmp/log”
rfs = “”

HACMPlogs:
name = “domino_server.log”
description = “Domino server Log”
defaultdir = “/var/hacmp/log”
value = “/var/hacmp/log”
rfs = “”

HACMPlogs:
name = “sax.log”
description = “Generated by utilities which serves director requests”
defaultdir = “/var/hacmp/log”
value = “/var/hacmp/log”
rfs = “”

HACMPlogs:
name = “ihssa.log”
description = “Smart Assist for IBM HTTP Server Log”
defaultdir = “/var/hacmp/log”
value = “/var/hacmp/log”
rfs = “”

HACMPlogs:
name = “maxdbsa.log”
description = “Smart Assist for MaxDB Log”
defaultdir = “/var/hacmp/log”
value = “/var/hacmp/log”
rfs = “”

HACMPlogs:
name = “clevents”
description = “Events log for Director Interface”
defaultdir = “/var/hacmp/log”
value = “/var/hacmp/log”
rfs = “”

HACMPlogs:
name = “hswizard.log”
description = “SAP liveCache Hot Standby Configiuration Wizard Log”
defaultdir = “/var/hacmp/log”
value = “/var/hacmp/log”
rfs = “”

HACMPlogs:
name = “wmqsa.log”
description = “MQ Series Smart Assist log”
defaultdir = “/var/hacmp/log”
value = “/var/hacmp/log”
rfs = “”
MFNBU01:/var/hacmp/log#clstop -f -N MFNBU01

Broadcast message from root@MFNBU01 (tty) at 10:10:09 …

PowerHA SystemMirror on MFNBU01 shutting down.

Please exit any cluster applications…
MFNBU01:/var/hacmp/log#clRGinfo
—————————————————————————–
Group Name State Node
—————————————————————————–
MFNBU_RG ERROR MFNBU01
OFFLINE MFNBU02

MFNBU01:/var/hacmp/log#lssrc -g cluster
Subsystem Group PID Status
clstrmgrES cluster 16187542 stopping
MFNBU01:/var/hacmp/log#lsha
Current state: ST_RP_FAILED
sccsid = “@(#)36 1.135.7.2 src/43haes/usr/sbin/cluster/hacmprd/main.C, hacmp.pe, 61haes_r711, 1225A_hacmp711 5/22/12 11:46:10”
build = “Dec 5 2012 11:50:45 1241C_hacmp711”
i_local_nodeid 0, i_local_siteid -1, my_handle 1
ml_idx[1]=0 ml_idx[2]=1
tp is 20433718
Events on event queue:
te_type 4, te_nodeid 1, te_network -1
te_type 4, te_nodeid 2, te_network -1
te_type 36, te_nodeid 1, te_network 1
te_type 11, te_nodeid 1, te_network -1
te_type 11, te_nodeid 2, te_network -1
There are 0 events on the Ibcast queue
There are 0 events on the RM Ibcast queue
CLversion: 13
local node vrmf is 7114
cluster fix level is “4”
The following timer(s) are currently active:
Event error node list: MFNBU01
Current DNP values
DNP Values for NodeId – 1 NodeName – MFNBU01
PgSpFree = 4190764 PvPctBusy = 0 PctTotalTimeIdle = 99.886272
DNP Values for NodeId – 2 NodeName – MFNBU02
PgSpFree = 4190430 PvPctBusy = 0 PctTotalTimeIdle = 99.854695
trcOn 0, kTraceOn 0, stopTraceOnExit 0, cdNodeOn 0
Last event run was FAIL_NODE on node 2
MFNBU01:/var/hacmp/log#lssrc -g cluster
Subsystem Group PID Status
clstrmgrES cluster 16187542 stopping
MFNBU01:/var/hacmp/log#cllslv
MFNBU_RG lvdiskstu
MFNBU_RG lvnbu
MFNBU_RG caalv_private1
MFNBU_RG caalv_private2
MFNBU_RG caalv_private3
MFNBU_RG powerha_crlv
MFNBU01:/var/hacmp/log#stopsrc -s clstrmgrES
0513-006 The Subsystem, clstrmgrES, is currently stopping its execution.
MFNBU01:/var/hacmp/log#lssrc -s clstrmgrES
Subsystem Group PID Status
clstrmgrES cluster 16187542 stopping
MFNBU01:/var/hacmp/log#lsha
Current state: ST_RP_FAILED
sccsid = “@(#)36 1.135.7.2 src/43haes/usr/sbin/cluster/hacmprd/main.C, hacmp.pe, 61haes_r711, 1225A_hacmp711 5/22/12 11:46:10”
build = “Dec 5 2012 11:50:45 1241C_hacmp711”
i_local_nodeid 0, i_local_siteid -1, my_handle 1
ml_idx[1]=0 ml_idx[2]=1
tp is 20433718
Events on event queue:
te_type 4, te_nodeid 1, te_network -1
te_type 4, te_nodeid 2, te_network -1
te_type 36, te_nodeid 1, te_network 1
te_type 11, te_nodeid 1, te_network -1
te_type 11, te_nodeid 2, te_network -1
There are 0 events on the Ibcast queue
There are 0 events on the RM Ibcast queue
CLversion: 13
local node vrmf is 7114
cluster fix level is “4”
The following timer(s) are currently active:
Event error node list: MFNBU01
Current DNP values
DNP Values for NodeId – 1 NodeName – MFNBU01
PgSpFree = 4190764 PvPctBusy = 0 PctTotalTimeIdle = 99.886272
DNP Values for NodeId – 2 NodeName – MFNBU02
PgSpFree = 4190430 PvPctBusy = 0 PctTotalTimeIdle = 99.854695
trcOn 0, kTraceOn 0, stopTraceOnExit 0, cdNodeOn 0
Last event run was FAIL_NODE on node 2

MFNBU01:/var/hacmp/log#stopsrc -f -s clstrmgrES
0513-006 The Subsystem, clstrmgrES, is currently stopping its execution.
MFNBU01:/var/hacmp/log#lssrc -s clstrmgrES
Subsystem Group PID Status
clstrmgrES cluster 16187542 stopping
MFNBU01:/var/hacmp/log#ps -ef|grep clstrmgrES
MFNBU01:/var/hacmp/log#ps -ef|grep cls
root 8847482 16187542 0 09:56:18 – 0:00 /usr/es/sbin/cluster/clstrmgr
root 16187542 4063468 0 15:49:15 – 0:02 /usr/es/sbin/cluster/clstrmgr
MFNBU01:/var/hacmp/log#kill -9 8847482
MFNBU01:/var/hacmp/log#ps -ef|grep cls
root 14680082 12582988 0 10:17:56 pts/1 0:00 grep cls
root 16187542 4063468 0 15:49:15 – 0:02 /usr/es/sbin/cluster/clstrmgr
MFNBU01:/var/hacmp/log#kill -9 16187542
MFNBU01:/var/hacmp/log#lssrc -s clstrmgrES
Subsystem Group PID Status
clstrmgrES cluster inoperative
MFNBU01:/var/hacmp/log#lsha
0513-036 The request could not be passed to the clstrmgrES subsystem.
Start the subsystem and try your command again.
MFNBU01:/var/hacmp/log#startsrc -s clstrmgrES
0513-059 The clstrmgrES Subsystem has been started. Subsystem PID is 16187546.
MFNBU01:/var/hacmp/log#lsha
Current state: ST_INIT
sccsid = “@(#)36 1.135.7.2 src/43haes/usr/sbin/cluster/hacmprd/main.C, hacmp.pe, 61haes_r711, 1225A_hacmp711 5/22/12 11:46:10”
build = “Dec 5 2012 11:50:45 1241C_hacmp711”
MFNBU01:/var/hacmp/log#smit hacmp

Starting Cluster Services on node: MFNBU01
This may take a few minutes. Please wait…
MFNBU01: start_cluster: Starting PowerHA SystemMirror
MFNBU01: 6750218 – 0:00 syslogd
MFNBU01: Setting routerevalidate to 1
MFNBU01: 0513-059 The clevmgrdES Subsystem has been started. Subsystem PID is 10551344.
MFNBU01: Mar 25 2015 10:23:31 Starting execution of /usr/es/sbin/cluster/etc/rc.cluster
MFNBU01: with parameters: -boot -N -A -C interactive -P cl_rc_cluster
[MORE…14]

WARNING: MFNBU01 has an active aliased service IP label MFNBU
attached to physical interface: en8. This service IP label
is part of resource group MFNBU_RG, which is in the OFFLINE state
on node MFNBU01.

WARNING: Application monitors are required for detecting application failures
in order for PowerHA SystemMirror to recover from them. Application monitors are started
by PowerHA SystemMirror when the resource group in which they participate is activated.
The following application(s), shown with their associated resource group,
do not have an application monitor configured:

Application Server Resource Group
——————————– ———————————
MFNBU_AS MFNBU_RG
WARNING: MFNBU01 has an active aliased service IP label MFNBU
attached to physical interface: en8. This service IP label
is part of resource group MFNBU_RG, which is in the OFFLINE state
on node MFNBU01.

Would you like to bring the resources of this resource group:
MFNBU_RG offline, so that you can then move the resource group
to a node other than node:MFNBU01 [Yes / No]:
Starting Corrective Action: cl_resource_resources_offline.
<01> Issued resource offline event to clean resources that belong to
resource group MFNBU_RG on node MFNBU01.
The output of the resource_offline event is logged in /tmp/hamcp.out.
The output of the resource_offline event is logged in /tmp/hamcp.out.
WARNING: Node MFNBU01 has cluster.es.nfs.rte installed however grace periods are not fully enabled on this node. Grace periods must be enabled before NFSv4 stable storag
e can be used.

PowerHA SystemMirror will attempt to fix this opportunistically when acquiring NFS resources on this node however the change won’t take effect until the next time that n
fsd is started.

If this warning persists, the administrator should perform the following steps to enable grace periods on MFNBU01 at the next planned downtime:
1. stopsrc -s nfsd
2. smitty nfsgrcperiod
3. startsrc -s nfsd

WARNING: Node MFNBU02 has cluster.es.nfs.rte installed however grace periods are not fully enabled on this node. Grace periods must be enabled before NFSv4 stable storag
e can be used.

#ps -ef |grep clsmgrES 杀掉进程后解决

ST_RP_FAILED: event script failed正常来讲是脚本异常引起的,正常操作为 smit hacmp -> Problem Determination Tools -> Recover From HACMP Script Failure

If you enjoyed this post, make sure you subscribe to my RSS feed!
此条目发表在AIX, 操作系统分类目录,贴了标签。将固定链接加入收藏夹。

发表评论

电子邮件地址不会被公开。 必填项已用*标注

*
To prove you're a person (not a spam script), type the security word shown in the picture. Click on the picture to hear an audio file of the word.
Anti-spam image