18c to 19c Clusterware upgrade failed on node 1 while running rootupgrade.sh,Node 2 is running with 18c
crs status from node 1
cd /u01/app/19.3.0.0/grid/bin
[grid@racnode01 ~]$ crsctl stat res -t -init
--------------------------------------------------------------------------------
Name Target State Server State details
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.asm
1 ONLINE OFFLINE STABLE
ora.cluster_interconnect.haip
1 ONLINE OFFLINE STABLE
ora.crf
1 ONLINE ONLINE racnode01 STABLE
ora.crsd
1 ONLINE OFFLINE STABLE
ora.cssd
1 ONLINE OFFLINE STABLE
ora.cssdmonitor
1 ONLINE ONLINE racnode01 STABLE
ora.ctssd
1 ONLINE OFFLINE STABLE
ora.diskmon
1 OFFLINE OFFLINE STABLE
ora.driver.afd
1 ONLINE ONLINE racnode01 STABLE
ora.drivers.acfs
1 ONLINE ONLINE racnode01 STABLE
ora.evmd
1 ONLINE INTERMEDIATE racnode01 STABLE
ora.gipcd
1 ONLINE ONLINE racnode01 STABLE
ora.gpnpd
1 ONLINE ONLINE racnode01 STABLE
ora.mdnsd
1 ONLINE ONLINE racnode01 STABLE
ora.storage
1 ONLINE OFFLINE STABLE
--------------------------------------------------------------------------------
[grid@racnode01 ~]$
Step 1:Shut down crs on the first node
root@racnode01:# cd /u01/app/19.3.0.0/grid/bin
root@racnode01:# ./crsctl stop crs -f
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'racnode01'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'racnode01'
CRS-2673: Attempting to stop 'ora.crf' on 'racnode01'
CRS-2673: Attempting to stop 'ora.cssdmonitor' on 'racnode01'
CRS-2673: Attempting to stop 'ora.driver.afd' on 'racnode01'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'racnode01'
CRS-2673: Attempting to stop 'ora.evmd' on 'racnode01'
CRS-2677: Stop of 'ora.cssdmonitor' on 'racnode01' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'racnode01'
CRS-2677: Stop of 'ora.driver.afd' on 'racnode01' succeeded
CRS-2677: Stop of 'ora.crf' on 'racnode01' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'racnode01'
CRS-2677: Stop of 'ora.drivers.acfs' on 'racnode01' succeeded
CRS-2677: Stop of 'ora.mdnsd' on 'racnode01' succeeded
CRS-2677: Stop of 'ora.evmd' on 'racnode01' succeeded
CRS-2677: Stop of 'ora.gpnpd' on 'racnode01' succeeded
CRS-2677: Stop of 'ora.gipcd' on 'racnode01' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'racnode01' has completed
CRS-4133: Oracle High Availability Services has been stopped.
Step 2. Unset the Oracle ASM rolling migration mode from any other node running GI other than first node
SQL> SELECT SYS_CONTEXT('sys_cluster_properties', 'cluster_state') FROM DUAL;
SYS_CONTEXT('SYS_CLUSTER_PROPERTIES','CLUSTER_STATE')
----------------------------------------------------------
In Rolling Upgrade
SQL> ALTER SYSTEM STOP ROLLING MIGRATION;
System altered.
SQL> SELECT SYS_CONTEXT('sys_cluster_properties', 'cluster_state') FROM DUAL;
SYS_CONTEXT('SYS_CLUSTER_PROPERTIES','CLUSTER_STATE')
------------------------------------------------------------
Normal
Step 3. Run command from any other node running GI other than first node
root@racnode02:# ./clscfg -nodedowngrade -h racnode01
clscfg: -nodedowngrade mode specified
clscfg: EXISTING configuration version 5 detected.
clscfg: version 5 is 12c Release 2.
Successfully restored the node specific configuration from backup for host:[racnode01]
root@racnode02:#
Step 4. From Node1 Grid home, run the command to downgrade Clusterware
root@racnode01:# pwd
/u01/app/19.3.0.0/grid/crs/install
root@racnode01:# ./rootcrs.sh -downgrade -online
Using configuration parameter file: /u01/app/19.3.0.0/grid/crs/install/crsconfig_params
The log of current session can be found at:
/app/grid/crsdata/racnode01/crsconfig/crsdowngrade_racnode01_2020-08-18_04-09-29AM.log
2020/08/18 04:13:32 CLSRSC-591: successfully downgraded Oracle Clusterware stack on this node
2020/08/18 04:13:33 CLSRSC-592: Run 'crsctl start crs' from home /u01/app/18.3.0.0/grid on each node to complete downgrade.
root@racnode01:#
root@racnode01:#
Step 5: Start crs from 18c home
root@racnode01:# cd /u01/app/18.3.0.0/grid/bin
root@racnode01:# ./crsctl start crs
CRS-4123: Oracle High Availability Services has been started.
[grid@racnode02 bin]$ ./crsctl stat res -t
--------------------------------------------------------------------------------
Name Target State Server State details
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.ASMNET1LSNR_ASM.lsnr
ONLINE ONLINE racnode01 STABLE
ONLINE ONLINE racnode02 STABLE
ora.LISTENER.lsnr
ONLINE ONLINE racnode01 STABLE
ONLINE ONLINE racnode02 STABLE
ora.POC_DATA.dg
ONLINE ONLINE racnode01 STABLE
ONLINE ONLINE racnode02 STABLE
ora.POC_FRA.dg
ONLINE ONLINE racnode01 STABLE
ONLINE ONLINE racnode02 STABLE
ora.VOTEDISK.GHCHKPT.advm
OFFLINE OFFLINE racnode01 STABLE
OFFLINE OFFLINE racnode02 STABLE
ora.VOTEDISK.dg
ONLINE ONLINE racnode01 STABLE
ONLINE ONLINE racnode02 STABLE
ora.chad
OFFLINE OFFLINE racnode01 STABLE
OFFLINE OFFLINE racnode02 STABLE
ora.helper
OFFLINE OFFLINE racnode01 IDLE,STABLE
OFFLINE OFFLINE racnode02 STABLE
ora.net1.network
ONLINE ONLINE racnode01 STABLE
ONLINE ONLINE racnode02 STABLE
ora.ons
ONLINE ONLINE racnode01 STABLE
ONLINE ONLINE racnode02 STABLE
ora.proxy_advm
ONLINE ONLINE racnode01 STABLE
ONLINE ONLINE racnode02 STABLE
ora.votedisk.ghchkpt.acfs
OFFLINE OFFLINE racnode01 STABLE
OFFLINE OFFLINE racnode02 STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE racnode01 STABLE
ora.LISTENER_SCAN2.lsnr
1 ONLINE ONLINE racnode02 STABLE
ora.LISTENER_SCAN3.lsnr
1 ONLINE ONLINE racnode02 STABLE
ora.MGMTLSNR
1 ONLINE ONLINE racnode02 XXX.XX.XX.XX,STABLE
ora.asm
1 ONLINE ONLINE racnode01 Started,STABLE
2 ONLINE ONLINE racnode02 Started,STABLE
3 OFFLINE OFFLINE STABLE
ora.cvu
1 ONLINE OFFLINE STABLE
ora.racnode01.vip
1 ONLINE ONLINE racnode01 STABLE
ora.racnode02.vip
1 ONLINE ONLINE racnode02 STABLE
ora.mgmtdb
1 ONLINE ONLINE racnode02 Open,STABLE
ora.poc01.db
1 ONLINE ONLINE racnode01 Open,HOME=/u01/app/o
racle/product/11.2.0
_64,STABLE
2 ONLINE ONLINE racnode02 Open,HOME=/u01/app/o
racle/product/11.2.0
_64,STABLE
ora.poc01.poc01_db1.svc
1 ONLINE ONLINE racnode01 STABLE
ora.qosmserver
1 OFFLINE OFFLINE STABLE
ora.rhpserver
1 OFFLINE OFFLINE STABLE
ora.scan1.vip
1 ONLINE ONLINE racnode01 STABLE
ora.scan2.vip
1 ONLINE ONLINE racnode02 STABLE
ora.scan3.vip
1 ONLINE ONLINE racnode02 STABLE
----------------------------------------------------------------------
No comments:
Post a Comment