Overview
We know that Exadata consists of
Storage grid, compute grid and Network grid. Exadata Storage runs the Exadata
storage server software which comes preinstalled and is responsible for
satisfying database IO requests and implementing unique Exadata features like
Smart scan, Smart Flash cache, and storage index and so on.
The Exadata storage cell consists
of:
- Hardware (Hard disk, Flash cache, Memory, Processor & IB ports)
- Exadata (Linux Operating System, Firmware, Exadata software)
The Exadata storage software
should be update periodically. Oracle
releases patches for Exadata to keep these components updated. These patches
can be applied online (Rolling) or offline (Non-Rolling).
About patchmgr
utility
The patchmgr
utility can be used for upgrading, rollback and backup Exadata Storage cells. patchmgr
utility can be used for upgrading Storage cells in a rolling or non-rolling
fashion. Non-Rolling is default. Storage server patches apply operating system, firmware, and
driver updates.
Launch patchmgr from the compute node that is
node 1 that has user equivalence setup to all the storage cells.
In this
article I will demonstrate how to perform Exadata Storage Software upgrade using
patchmgr utility.
MOS Notes
Read the following MOS notes
carefully.
- Exadata Database Machine and Exadata Storage Server Supported Versions (Doc ID 888828.1)
- Exadata 12.1.2.2.0 release and patch (20131726) (Doc ID 2038073.1)
- 1070954.1: Oracle Exadata Database Machine exachk or HealthCheck
Note: Always run Exachk before
and after patching.
Software Download
Download the following patches
required for Upgrading Storage cells.
- Patch 20131726 - Storage server and InfiniBand switch software
Current Environment
- Exadata V2 Full Rack (8 Compute nodes, 14 Storage Cells and 3 IB Switches) running ESS version 12.1.2.1.1
Current Image version
Execute
the “imageinfo” command on one of the Storage cells to identify the current
Exadata Image version
[root@oracloudceladm01
~]# imageinfo
Kernel version:
2.6.39-400.248.3.el6uek.x86_64 #1 SMP Wed Mar 11 18:04:34 PDT 2015 x86_64
Cell version:
OSS_12.1.2.1.1_LINUX.X64_150316.2
Cell rpm version:
cell-12.1.2.1.1_LINUX.X64_150316.2-1.x86_64
Active image version: 12.1.2.1.1.150316.2
Active image
activated: 2015-04-27 16:55:04 -0500
Active image status:
success
Active system
partition on device: /dev/md5
Active software
partition on device: /dev/md7
Cell boot usb
partition: /dev/sdac1
Cell boot usb
version: 12.1.2.1.1.150316.2
Inactive image
version: 12.1.2.1.0.141206.1
Inactive image
activated: 2015-02-25 07:39:28 -0600
Inactive image
status: success
Inactive system
partition on device: /dev/md6
Inactive software
partition on device: /dev/md8
Inactive marker for
the rollback: /boot/I_am_hd_boot.inactive
Inactive grub config
for the rollback: /boot/grub/grub.conf.inactive
Inactive kernel
version for the rollback: 2.6.39-400.243.1.el6uek.x86_64
Rollback to the
inactive partitions: Possible
Pre-requisites
- root user access for compute node and Storage cells.
- root user equivalence must be setup between compute node and storage cells.
#dcli -g dbs_group -l root –k
- Shutdown database(s) and Clusterware stack.
$ srvctl stop database -d dbm01
# $GRID_HOME/crsctl stop
cluster –all
# dcli -g dbs_group -l root
‘/u01/app/12.1.0.2/grid/bin/crsctl stop crs’
cell_group
file containing storage cell names or IP address per line
- Storage Cell software downloaded and stage into a directory
# pwd /u01/patches/ESS_121220
Steps to perform Storage Cell Patching
- Identify the Storage Software Patch to be applied. Here we have copied the Patch 20131726 to the following location.
[root@dm01db01
~]# cd /u01/stage/ESS_Patch/
[root@dm01db01
ESS_Patch]# ls -l
total
2537072
-rw-r--r--
1 root root 1550028016 Sep 27 07:39 p20131726_121220_Linux-x86-64.zip
- Ensure that root user ssh equivalence is setup by running the following command.
[root@dm01db01
ESS_Patch]# dcli -g ~/cell_group -l root 'hostname -i'
dm01cel01:
10.10.41.208
dm01cel02:
10.10.41.209
dm01cel03:
10.10.41.210
dm01cel04:
10.10.41.211
dm01cel05:
10.10.41.212
dm01cel06:
10.10.41.213
dm01cel07:
10.10.41.214
dm01cel08:
10.10.41.215
dm01cel09:
10.10.41.216
dm01cel10:
10.10.41.217
dm01cel11:
10.10.41.218
dm01cel12:
10.10.41.219
dm01cel13:
10.10.41.220
dm01cel14:
10.10.41.221
- Switch to grid software owner (Oracle) and update the disk_repair_time parameter to higher value to avoid dropping the grid disks from the ASM disk group.
[root@dm01db01
ESS_Patch]# su - oracle
dm01db01-dbm1
{/home/oracle}:ps -ef|grep pmon
oracle 9171
1 0 Sep02 ? 00:02:10 mdb_pmon_-MGMTDB
oracle 22767
1 0 Sep02 ? 00:06:34 ora_pmon_dbm1
oracle 25181
1 0 Sep02 ? 00:02:58 asm_pmon_+ASM1
oracle 25619 25442
0 01:50 pts/0 00:00:00 grep
pmon
dm01db01-dbm1
{/home/oracle}:. oraenv
ORACLE_SID
= [dbm1] ? +ASM1
The
Oracle base remains unchanged with value /u01/app/oracle
dm01db01-+ASM1
{/home/oracle}:sqlplus / as sysasm
SQL*Plus:
Release 12.1.0.2.0 Production on Sat Oct 3 01:50:37 2015
Copyright
(c) 1982, 2014, Oracle. All rights
reserved.
Connected
to:
Oracle
Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With
the Real Application Clusters and Automatic Storage Management options
SQL>
col value for a40
SQL>
select dg.name,a.value from v$asm_diskgroup dg, v$asm_attribute a where
dg.group_number=a.group_number and
a.name='disk_repair_time';
NAME VALUE
------------------------------
----------------------------------------
DATA_DM01 8.5H
RECO_DM01 3.6h
SQL>
alter diskgroup RECO_DM01 set attribute 'disk_repair_time'='8.5h';
Diskgroup
altered.
SQL>
select dg.name,a.value from v$asm_diskgroup dg, v$asm_attribute a where
dg.group_number=a.group_number and
a.name='disk_repair_time';
NAME VALUE
------------------------------
----------------------------------------
DATA_DM01 8.5H
RECO_DM01 8.5h
SQL>
exit
Disconnected
from Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit
Production
With
the Real Application Clusters and Automatic Storage Management options
dm01db01-+ASM1
{/home/oracle}:exit
logout
- Shutdown the Clusterware on all the Exadata compute nodes as follows:
[root@dm01db01
ESS_Patch]# cd /u01/app/12.1.0.2/grid/bin/
[root@dm01db01
bin]# ./crsctl stop cluster -all
CRS-2673:
Attempting to stop 'ora.crsd' on 'dm01db02'
CRS-2673:
Attempting to stop 'ora.crsd' on 'dm01db04'
CRS-2673:
Attempting to stop 'ora.crsd' on 'dm01db03'
CRS-2790:
Starting shutdown of Cluster Ready Services-managed resources on 'dm01db02'
CRS-2673:
Attempting to stop 'ora.crsd' on 'dm01db01'
CRS-2673:
Attempting to stop 'ora.dbm.dbmlab.svc' on 'dm01db02'
CRS-2673:
Attempting to stop 'ora.LISTENER_IB.lsnr' on 'dm01db02'
CRS-2790:
Starting shutdown of Cluster Ready Services-managed resources on 'dm01db04'
CRS-2673:
Attempting to stop 'ora.LISTENER_SCAN1.lsnr' on 'dm01db02'
CRS-2673:
Attempting to stop 'ora.dbm.dbmlab.svc' on 'dm01db04'
CRS-2673:
Attempting to stop 'ora.LISTENER.lsnr' on 'dm01db02'
CRS-2673:
Attempting to stop 'ora.dbm.oragraph.svc' on 'dm01db04'
CRS-2790:
Starting shutdown of Cluster Ready Services-managed resources on 'dm01db03'
CRS-2673:
Attempting to stop 'ora.LISTENER_SCAN3.lsnr' on 'dm01db04'
CRS-2673:
Attempting to stop 'ora.dbm.oragraph.svc' on 'dm01db03'
CRS-2673:
Attempting to stop 'ora.LISTENER_IB.lsnr' on 'dm01db04'
CRS-2673:
Attempting to stop 'ora.LISTENER.lsnr' on 'dm01db03'
CRS-2673:
Attempting to stop 'ora.LISTENER_IB.lsnr' on 'dm01db03'
CRS-2790:
Starting shutdown of Cluster Ready Services-managed resources on 'dm01db01'
CRS-2673:
Attempting to stop 'ora.dbm.dbmlab.svc' on 'dm01db01'
CRS-2677:
Stop of 'ora.dbm.dbmlab.svc' on 'dm01db02' succeeded
CRS-2673:
Attempting to stop 'ora.dbm.oragraph.svc' on 'dm01db01'
CRS-2673:
Attempting to stop 'ora.dbm.oragraph.svc' on 'dm01db02'
CRS-2673:
Attempting to stop 'ora.LISTENER_SCAN2.lsnr' on 'dm01db01'
CRS-2677:
Stop of 'ora.dbm.oragraph.svc' on 'dm01db02' succeeded
CRS-2673:
Attempting to stop 'ora.LISTENER.lsnr' on 'dm01db01'
CRS-2673:
Attempting to stop 'ora.LISTENER_IB.lsnr' on 'dm01db01'
CRS-2673:
Attempting to stop 'ora.dbm.db' on 'dm01db02'
CRS-2677:
Stop of 'ora.LISTENER_SCAN3.lsnr' on 'dm01db04' succeeded
CRS-2673:
Attempting to stop 'ora.scan3.vip' on 'dm01db04'
CRS-2677:
Stop of 'ora.LISTENER.lsnr' on 'dm01db03' succeeded
CRS-2677:
Stop of 'ora.LISTENER_IB.lsnr' on 'dm01db03' succeeded
CRS-2677:
Stop of 'ora.LISTENER_SCAN2.lsnr' on 'dm01db01' succeeded
CRS-2673:
Attempting to stop 'ora.scan2.vip' on 'dm01db01'
CRS-2677:
Stop of 'ora.LISTENER.lsnr' on 'dm01db01' succeeded
CRS-2673:
Attempting to stop 'ora.crsd' on 'dm01db05'
CRS-2673:
Attempting to stop 'ora.crsd' on 'dm01db06'
CRS-2673:
Attempting to stop 'ora.crsd' on 'dm01db07'
CRS-2673:
Attempting to stop 'ora.crsd' on 'dm01db08'
CRS-2790:
Starting shutdown of Cluster Ready Services-managed resources on 'dm01db05'
CRS-2673:
Attempting to stop 'ora.dbm.oragraph.svc' on 'dm01db05'
CRS-2673:
Attempting to stop 'ora.LISTENER_IB.lsnr' on 'dm01db05'
CRS-2673:
Attempting to stop 'ora.oc4j' on 'dm01db05'
CRS-2790:
Starting shutdown of Cluster Ready Services-managed resources on 'dm01db07'
CRS-2673:
Attempting to stop 'ora.LISTENER.lsnr' on 'dm01db07'
CRS-2673:
Attempting to stop 'ora.LISTENER_IB.lsnr' on 'dm01db07'
CRS-2790:
Starting shutdown of Cluster Ready Services-managed resources on 'dm01db08'
CRS-2673:
Attempting to stop 'ora.LISTENER_IB.lsnr' on 'dm01db08'
CRS-2673:
Attempting to stop 'ora.dbm.dbmlab.svc' on 'dm01db08'
CRS-2673:
Attempting to stop 'ora.dbm.oragraph.svc' on 'dm01db08'
CRS-2673:
Attempting to stop 'ora.LISTENER.lsnr' on 'dm01db08'
CRS-2677:
Stop of 'ora.LISTENER_IB.lsnr' on 'dm01db05' succeeded
CRS-2673:
Attempting to stop 'ora.dm01db05_2.vip' on 'dm01db05'
CRS-2677:
Stop of 'ora.LISTENER.lsnr' on 'dm01db07' succeeded
CRS-2677:
Stop of 'ora.LISTENER_IB.lsnr' on 'dm01db04' succeeded
CRS-2677:
Stop of 'ora.LISTENER_IB.lsnr' on 'dm01db08' succeeded
CRS-2677:
Stop of 'ora.LISTENER_IB.lsnr' on 'dm01db01' succeeded
CRS-2673:
Attempting to stop 'ora.dm01db01_2.vip' on 'dm01db01'
CRS-2677:
Stop of 'ora.dm01db01_2.vip' on 'dm01db01' succeeded
CRS-2677:
Stop of 'ora.dm01db05_2.vip' on 'dm01db05' succeeded
CRS-2790:
Starting shutdown of Cluster Ready Services-managed resources on 'dm01db06'
CRS-2673:
Attempting to stop 'ora.dbm.oragraph.svc' on 'dm01db06'
CRS-2673:
Attempting to stop 'ora.LISTENER_IB.lsnr' on 'dm01db06'
CRS-2677:
Stop of 'ora.LISTENER_IB.lsnr' on 'dm01db06' succeeded
CRS-2673:
Attempting to stop 'ora.dm01db06_2.vip' on 'dm01db06'
CRS-2673:
Attempting to stop 'ora.dm01db08_2.vip' on 'dm01db08'
CRS-2677:
Stop of 'ora.LISTENER.lsnr' on 'dm01db08' succeeded
CRS-2677:
Stop of 'ora.scan2.vip' on 'dm01db01' succeeded
CRS-2677:
Stop of 'ora.LISTENER_IB.lsnr' on 'dm01db07' succeeded
CRS-2673:
Attempting to stop 'ora.dm01db04_2.vip' on 'dm01db04'
CRS-2677:
Stop of 'ora.LISTENER_IB.lsnr' on 'dm01db02' succeeded
CRS-2673:
Attempting to stop 'ora.dm01db02_2.vip' on 'dm01db02'
CRS-2673:
Attempting to stop 'ora.dm01db03_2.vip' on 'dm01db03'
CRS-2677:
Stop of 'ora.LISTENER_SCAN1.lsnr' on 'dm01db02' succeeded
CRS-2677:
Stop of 'ora.LISTENER.lsnr' on 'dm01db02' succeeded
CRS-2673:
Attempting to stop 'ora.dm01db02.vip' on 'dm01db02'
CRS-2673:
Attempting to stop 'ora.scan1.vip' on 'dm01db02'
CRS-2677:
Stop of 'ora.dm01db03_2.vip' on 'dm01db03' succeeded
CRS-2677:
Stop of 'ora.dbm.db' on 'dm01db02' succeeded
CRS-2673:
Attempting to stop 'ora.DATA_DM01.dg' on 'dm01db02'
CRS-2677:
Stop of 'ora.dm01db08_2.vip' on 'dm01db08' succeeded
CRS-2673:
Attempting to stop 'ora.RECO_DM01.dg' on 'dm01db02'
CRS-2677:
Stop of 'ora.dm01db06_2.vip' on 'dm01db06' succeeded
CRS-2677:
Stop of 'ora.dm01db02.vip' on 'dm01db02' succeeded
CRS-2673:
Attempting to stop 'ora.dm01db07_2.vip' on 'dm01db07'
CRS-2677:
Stop of 'ora.dm01db07_2.vip' on 'dm01db07' succeeded
CRS-2677:
Stop of 'ora.dm01db04_2.vip' on 'dm01db04' succeeded
CRS-2677:
Stop of 'ora.scan3.vip' on 'dm01db04' succeeded
CRS-2677:
Stop of 'ora.dm01db02_2.vip' on 'dm01db02' succeeded
CRS-2677:
Stop of 'ora.scan1.vip' on 'dm01db02' succeeded
CRS-2677:
Stop of 'ora.DATA_DM01.dg' on 'dm01db02' succeeded
CRS-2677:
Stop of 'ora.RECO_DM01.dg' on 'dm01db02' succeeded
CRS-2673:
Attempting to stop 'ora.asm' on 'dm01db02'
CRS-2677:
Stop of 'ora.asm' on 'dm01db02' succeeded
CRS-2677:
Stop of 'ora.oc4j' on 'dm01db05' succeeded
CRS-2673:
Attempting to stop 'ora.ons' on 'dm01db02'
CRS-2673:
Attempting to stop 'ora.net2.network' on 'dm01db02'
CRS-2677:
Stop of 'ora.net2.network' on 'dm01db02' succeeded
CRS-2677:
Stop of 'ora.ons' on 'dm01db02' succeeded
CRS-2673:
Attempting to stop 'ora.net1.network' on 'dm01db02'
CRS-2677:
Stop of 'ora.net1.network' on 'dm01db02' succeeded
CRS-2792:
Shutdown of Cluster Ready Services-managed resources on 'dm01db02' has
completed
CRS-2677:
Stop of 'ora.dbm.oragraph.svc' on 'dm01db03' succeeded
CRS-2673:
Attempting to stop 'ora.dbm.dbmlab.svc' on 'dm01db03'
CRS-2677:
Stop of 'ora.dbm.dbmlab.svc' on 'dm01db03' succeeded
CRS-2673:
Attempting to stop 'ora.dbm.db' on 'dm01db03'
CRS-2673:
Attempting to stop 'ora.dm01db03.vip' on 'dm01db03'
CRS-2677:
Stop of 'ora.crsd' on 'dm01db02' succeeded
CRS-2673:
Attempting to stop 'ora.ctssd' on 'dm01db02'
CRS-2673:
Attempting to stop 'ora.evmd' on 'dm01db02'
CRS-2673:
Attempting to stop 'ora.storage' on 'dm01db02'
CRS-2677:
Stop of 'ora.storage' on 'dm01db02' succeeded
CRS-2673:
Attempting to stop 'ora.asm' on 'dm01db02'
CRS-2677:
Stop of 'ora.dm01db03.vip' on 'dm01db03' succeeded
CRS-2677:
Stop of 'ora.dbm.db' on 'dm01db03' succeeded
CRS-2673:
Attempting to stop 'ora.RECO_DM01.dg' on 'dm01db03'
CRS-2673:
Attempting to stop 'ora.DATA_DM01.dg' on 'dm01db03'
CRS-2677:
Stop of 'ora.ctssd' on 'dm01db02' succeeded
CRS-2677:
Stop of 'ora.DATA_DM01.dg' on 'dm01db03' succeeded
CRS-2677:
Stop of 'ora.RECO_DM01.dg' on 'dm01db03' succeeded
CRS-2673:
Attempting to stop 'ora.asm' on 'dm01db03'
CRS-2677:
Stop of 'ora.asm' on 'dm01db03' succeeded
CRS-2673:
Attempting to stop 'ora.ons' on 'dm01db03'
CRS-2673:
Attempting to stop 'ora.net2.network' on 'dm01db03'
CRS-2677:
Stop of 'ora.net2.network' on 'dm01db03' succeeded
CRS-2677:
Stop of 'ora.ons' on 'dm01db03' succeeded
CRS-2673:
Attempting to stop 'ora.net1.network' on 'dm01db03'
CRS-2677:
Stop of 'ora.net1.network' on 'dm01db03' succeeded
CRS-2792:
Shutdown of Cluster Ready Services-managed resources on 'dm01db03' has
completed
CRS-2677:
Stop of 'ora.crsd' on 'dm01db03' succeeded
CRS-2673:
Attempting to stop 'ora.ctssd' on 'dm01db03'
CRS-2673:
Attempting to stop 'ora.evmd' on 'dm01db03'
CRS-2673:
Attempting to stop 'ora.storage' on 'dm01db03'
CRS-2677:
Stop of 'ora.storage' on 'dm01db03' succeeded
CRS-2673:
Attempting to stop 'ora.asm' on 'dm01db03'
CRS-2677:
Stop of 'ora.dbm.oragraph.svc' on 'dm01db01' succeeded
CRS-2677:
Stop of 'ora.dbm.dbmlab.svc' on 'dm01db01' succeeded
CRS-2673:
Attempting to stop 'ora.dbm.db' on 'dm01db01'
CRS-2677:
Stop of 'ora.dbm.oragraph.svc' on 'dm01db05' succeeded
CRS-2673:
Attempting to stop 'ora.dbm.dbmlab.svc' on 'dm01db05'
CRS-2673:
Attempting to stop 'ora.dm01db01.vip' on 'dm01db01'
CRS-2677:
Stop of 'ora.dbm.dbmlab.svc' on 'dm01db08' succeeded
CRS-2677:
Stop of 'ora.dbm.oragraph.svc' on 'dm01db08' succeeded
CRS-2677:
Stop of 'ora.dbm.dbmlab.svc' on 'dm01db05' succeeded
CRS-2673:
Attempting to stop 'ora.dbm.db' on 'dm01db05'
CRS-2673:
Attempting to stop 'ora.dbm.db' on 'dm01db08'
CRS-2673:
Attempting to stop 'ora.LISTENER.lsnr' on 'dm01db05'
CRS-2673:
Attempting to stop 'ora.dm01db08.vip' on 'dm01db08'
CRS-2677:
Stop of 'ora.LISTENER.lsnr' on 'dm01db05' succeeded
CRS-2673:
Attempting to stop 'ora.dm01db05.vip' on 'dm01db05'
CRS-2677:
Stop of 'ora.ctssd' on 'dm01db03' succeeded
CRS-2677:
Stop of 'ora.evmd' on 'dm01db03' succeeded
CRS-2677:
Stop of 'ora.dm01db01.vip' on 'dm01db01' succeeded
CRS-2677:
Stop of 'ora.dm01db08.vip' on 'dm01db08' succeeded
CRS-2677:
Stop of 'ora.dbm.db' on 'dm01db08' succeeded
CRS-2673:
Attempting to stop 'ora.DATA_DM01.dg' on 'dm01db08'
CRS-2677:
Stop of 'ora.dbm.db' on 'dm01db05' succeeded
CRS-2673:
Attempting to stop 'ora.RECO_DM01.dg' on 'dm01db08'
CRS-2673:
Attempting to stop 'ora.DATA_DM01.dg' on 'dm01db05'
CRS-2673:
Attempting to stop 'ora.RECO_DM01.dg' on 'dm01db05'
CRS-2677:
Stop of 'ora.DATA_DM01.dg' on 'dm01db05' succeeded
CRS-2677:
Stop of 'ora.dm01db05.vip' on 'dm01db05' succeeded
CRS-2677:
Stop of 'ora.RECO_DM01.dg' on 'dm01db05' succeeded
CRS-2673:
Attempting to stop 'ora.asm' on 'dm01db05'
CRS-2677:
Stop of 'ora.asm' on 'dm01db05' succeeded
CRS-2677:
Stop of 'ora.dbm.db' on 'dm01db01' succeeded
CRS-2673:
Attempting to stop 'ora.RECO_DM01.dg' on 'dm01db01'
CRS-2673:
Attempting to stop 'ora.mgmtdb' on 'dm01db01'
CRS-2673:
Attempting to stop 'ora.dbm.dbmlab.svc' on 'dm01db07'
CRS-2673:
Attempting to stop 'ora.dbm.oragraph.svc' on 'dm01db07'
CRS-2677:
Stop of 'ora.RECO_DM01.dg' on 'dm01db01' succeeded
CRS-2677:
Stop of 'ora.DATA_DM01.dg' on 'dm01db08' succeeded
CRS-2677:
Stop of 'ora.RECO_DM01.dg' on 'dm01db08' succeeded
CRS-2673:
Attempting to stop 'ora.asm' on 'dm01db08'
CRS-2677:
Stop of 'ora.asm' on 'dm01db08' succeeded
CRS-2673:
Attempting to stop 'ora.net2.network' on 'dm01db08'
CRS-2677:
Stop of 'ora.net2.network' on 'dm01db08' succeeded
CRS-2677:
Stop of 'ora.mgmtdb' on 'dm01db01' succeeded
CRS-2673:
Attempting to stop 'ora.DATA_DM01.dg' on 'dm01db01'
CRS-2673:
Attempting to stop 'ora.MGMTLSNR' on 'dm01db01'
CRS-2677:
Stop of 'ora.DATA_DM01.dg' on 'dm01db01' succeeded
CRS-2677:
Stop of 'ora.MGMTLSNR' on 'dm01db01' succeeded
CRS-2673:
Attempting to stop 'ora.asm' on 'dm01db01'
CRS-2673:
Attempting to stop 'ora.ons' on 'dm01db08'
CRS-2677:
Stop of 'ora.asm' on 'dm01db01' succeeded
CRS-2673:
Attempting to stop 'ora.net2.network' on 'dm01db01'
CRS-2673:
Attempting to stop 'ora.ons' on 'dm01db01'
CRS-2677:
Stop of 'ora.net2.network' on 'dm01db01' succeeded
CRS-2677:
Stop of 'ora.ons' on 'dm01db01' succeeded
CRS-2673:
Attempting to stop 'ora.net1.network' on 'dm01db01'
CRS-2677:
Stop of 'ora.net1.network' on 'dm01db01' succeeded
CRS-2792:
Shutdown of Cluster Ready Services-managed resources on 'dm01db01' has
completed
CRS-2677:
Stop of 'ora.ons' on 'dm01db08' succeeded
CRS-2673:
Attempting to stop 'ora.net1.network' on 'dm01db08'
CRS-2677:
Stop of 'ora.net1.network' on 'dm01db08' succeeded
CRS-2792:
Shutdown of Cluster Ready Services-managed resources on 'dm01db08' has
completed
CRS-2677:
Stop of 'ora.dbm.oragraph.svc' on 'dm01db04' succeeded
CRS-2677:
Stop of 'ora.dbm.dbmlab.svc' on 'dm01db04' succeeded
CRS-2673:
Attempting to stop 'ora.dbm.db' on 'dm01db04'
CRS-2673:
Attempting to stop 'ora.LISTENER.lsnr' on 'dm01db04'
CRS-2677:
Stop of 'ora.LISTENER.lsnr' on 'dm01db04' succeeded
CRS-2673:
Attempting to stop 'ora.dm01db04.vip' on 'dm01db04'
CRS-2677:
Stop of 'ora.crsd' on 'dm01db01' succeeded
CRS-2673:
Attempting to stop 'ora.ctssd' on 'dm01db01'
CRS-2673:
Attempting to stop 'ora.evmd' on 'dm01db01'
CRS-2673:
Attempting to stop 'ora.storage' on 'dm01db01'
CRS-2677:
Stop of 'ora.storage' on 'dm01db01' succeeded
CRS-2673:
Attempting to stop 'ora.asm' on 'dm01db01'
CRS-2677:
Stop of 'ora.crsd' on 'dm01db08' succeeded
CRS-2673:
Attempting to stop 'ora.ctssd' on 'dm01db08'
CRS-2673:
Attempting to stop 'ora.evmd' on 'dm01db08'
CRS-2673:
Attempting to stop 'ora.storage' on 'dm01db08'
CRS-2677:
Stop of 'ora.storage' on 'dm01db08' succeeded
CRS-2673:
Attempting to stop 'ora.asm' on 'dm01db08'
CRS-2677:
Stop of 'ora.dbm.db' on 'dm01db04' succeeded
CRS-2673:
Attempting to stop 'ora.DATA_DM01.dg' on 'dm01db04'
CRS-2673:
Attempting to stop 'ora.RECO_DM01.dg' on 'dm01db04'
CRS-2677:
Stop of 'ora.DATA_DM01.dg' on 'dm01db04' succeeded
CRS-2677:
Stop of 'ora.asm' on 'dm01db02' succeeded
CRS-2673:
Attempting to stop 'ora.cluster_interconnect.haip' on 'dm01db02'
CRS-2677:
Stop of 'ora.RECO_DM01.dg' on 'dm01db04' succeeded
CRS-2673:
Attempting to stop 'ora.asm' on 'dm01db04'
CRS-2677:
Stop of 'ora.asm' on 'dm01db04' succeeded
CRS-2677:
Stop of 'ora.dm01db04.vip' on 'dm01db04' succeeded
CRS-2673:
Attempting to stop 'ora.net2.network' on 'dm01db04'
CRS-2673:
Attempting to stop 'ora.ons' on 'dm01db04'
CRS-2677:
Stop of 'ora.net2.network' on 'dm01db04' succeeded
CRS-2677:
Stop of 'ora.ons' on 'dm01db04' succeeded
CRS-2673:
Attempting to stop 'ora.net1.network' on 'dm01db04'
CRS-2677:
Stop of 'ora.net1.network' on 'dm01db04' succeeded
CRS-2792:
Shutdown of Cluster Ready Services-managed resources on 'dm01db04' has
completed
CRS-2677:
Stop of 'ora.ctssd' on 'dm01db01' succeeded
CRS-2677:
Stop of 'ora.ctssd' on 'dm01db08' succeeded
CRS-2677:
Stop of 'ora.evmd' on 'dm01db08' succeeded
CRS-2677:
Stop of 'ora.cluster_interconnect.haip' on 'dm01db02' succeeded
CRS-2677:
Stop of 'ora.crsd' on 'dm01db04' succeeded
CRS-2673:
Attempting to stop 'ora.ctssd' on 'dm01db04'
CRS-2673:
Attempting to stop 'ora.evmd' on 'dm01db04'
CRS-2673:
Attempting to stop 'ora.storage' on 'dm01db04'
CRS-2677:
Stop of 'ora.storage' on 'dm01db04' succeeded
CRS-2673:
Attempting to stop 'ora.asm' on 'dm01db04'
CRS-2677:
Stop of 'ora.asm' on 'dm01db03' succeeded
CRS-2673:
Attempting to stop 'ora.cluster_interconnect.haip' on 'dm01db03'
CRS-2677:
Stop of 'ora.dbm.dbmlab.svc' on 'dm01db07' succeeded
CRS-2677:
Stop of 'ora.dbm.oragraph.svc' on 'dm01db07' succeeded
CRS-2673:
Attempting to stop 'ora.dbm.db' on 'dm01db07'
CRS-2673:
Attempting to stop 'ora.dm01db07.vip' on 'dm01db07'
CRS-2677:
Stop of 'ora.cluster_interconnect.haip' on 'dm01db03' succeeded
CRS-2673:
Attempting to stop 'ora.cssd' on 'dm01db03'
CRS-2677:
Stop of 'ora.ctssd' on 'dm01db04' succeeded
CRS-2677:
Stop of 'ora.evmd' on 'dm01db04' succeeded
CRS-2677:
Stop of 'ora.dm01db07.vip' on 'dm01db07' succeeded
CRS-2677:
Stop of 'ora.cssd' on 'dm01db03' succeeded
CRS-2673:
Attempting to stop 'ora.diskmon' on 'dm01db03'
CRS-2677:
Stop of 'ora.dbm.db' on 'dm01db07' succeeded
CRS-2673:
Attempting to stop 'ora.DATA_DM01.dg' on 'dm01db07'
CRS-2673:
Attempting to stop 'ora.RECO_DM01.dg' on 'dm01db07'
CRS-2677:
Stop of 'ora.DATA_DM01.dg' on 'dm01db07' succeeded
CRS-2677:
Stop of 'ora.RECO_DM01.dg' on 'dm01db07' succeeded
CRS-2673:
Attempting to stop 'ora.asm' on 'dm01db07'
CRS-2677:
Stop of 'ora.asm' on 'dm01db07' succeeded
CRS-2673:
Attempting to stop 'ora.ons' on 'dm01db07'
CRS-2673:
Attempting to stop 'ora.net2.network' on 'dm01db07'
CRS-2677:
Stop of 'ora.net2.network' on 'dm01db07' succeeded
CRS-2677:
Stop of 'ora.ons' on 'dm01db07' succeeded
CRS-2673:
Attempting to stop 'ora.net1.network' on 'dm01db07'
CRS-2677:
Stop of 'ora.net1.network' on 'dm01db07' succeeded
CRS-2792:
Shutdown of Cluster Ready Services-managed resources on 'dm01db07' has completed
CRS-2677:
Stop of 'ora.diskmon' on 'dm01db03' succeeded
CRS-2677:
Stop of 'ora.crsd' on 'dm01db07' succeeded
CRS-2673:
Attempting to stop 'ora.ctssd' on 'dm01db07'
CRS-2673:
Attempting to stop 'ora.evmd' on 'dm01db07'
CRS-2673:
Attempting to stop 'ora.storage' on 'dm01db07'
CRS-2677:
Stop of 'ora.storage' on 'dm01db07' succeeded
CRS-2673:
Attempting to stop 'ora.asm' on 'dm01db07'
CRS-2677:
Stop of 'ora.dbm.oragraph.svc' on 'dm01db06' succeeded
CRS-2673:
Attempting to stop 'ora.LISTENER.lsnr' on 'dm01db06'
CRS-2673:
Attempting to stop 'ora.dbm.dbmlab.svc' on 'dm01db06'
CRS-2677:
Stop of 'ora.dbm.dbmlab.svc' on 'dm01db06' succeeded
CRS-2673:
Attempting to stop 'ora.dbm.db' on 'dm01db06'
CRS-2677:
Stop of 'ora.ctssd' on 'dm01db07' succeeded
CRS-2677:
Stop of 'ora.evmd' on 'dm01db07' succeeded
CRS-2677:
Stop of 'ora.LISTENER.lsnr' on 'dm01db06' succeeded
CRS-2673:
Attempting to stop 'ora.dm01db06.vip' on 'dm01db06'
CRS-2677:
Stop of 'ora.dbm.db' on 'dm01db06' succeeded
CRS-2673:
Attempting to stop 'ora.RECO_DM01.dg' on 'dm01db06'
CRS-2677:
Stop of 'ora.RECO_DM01.dg' on 'dm01db06' succeeded
CRS-2673:
Attempting to stop 'ora.DATA_DM01.dg' on 'dm01db06'
CRS-2677:
Stop of 'ora.DATA_DM01.dg' on 'dm01db06' succeeded
CRS-2673:
Attempting to stop 'ora.asm' on 'dm01db06'
CRS-2677:
Stop of 'ora.asm' on 'dm01db06' succeeded
CRS-2677:
Stop of 'ora.dm01db06.vip' on 'dm01db06' succeeded
CRS-2673:
Attempting to stop 'ora.ons' on 'dm01db06'
CRS-2673:
Attempting to stop 'ora.net2.network' on 'dm01db06'
CRS-2677:
Stop of 'ora.net2.network' on 'dm01db06' succeeded
CRS-2677:
Stop of 'ora.ons' on 'dm01db06' succeeded
CRS-2673:
Attempting to stop 'ora.net1.network' on 'dm01db06'
CRS-2673:
Attempting to stop 'ora.net2.network' on 'dm01db05'
CRS-2673:
Attempting to stop 'ora.ons' on 'dm01db05'
CRS-2677:
Stop of 'ora.net2.network' on 'dm01db05' succeeded
CRS-2677:
Stop of 'ora.net1.network' on 'dm01db06' succeeded
CRS-2792:
Shutdown of Cluster Ready Services-managed resources on 'dm01db06' has
completed
CRS-2677:
Stop of 'ora.ons' on 'dm01db05' succeeded
CRS-2673:
Attempting to stop 'ora.net1.network' on 'dm01db05'
CRS-2677:
Stop of 'ora.net1.network' on 'dm01db05' succeeded
CRS-2792:
Shutdown of Cluster Ready Services-managed resources on 'dm01db05' has
completed
CRS-2677:
Stop of 'ora.asm' on 'dm01db01' succeeded
CRS-2673:
Attempting to stop 'ora.cluster_interconnect.haip' on 'dm01db01'
CRS-2677:
Stop of 'ora.crsd' on 'dm01db06' succeeded
CRS-2673:
Attempting to stop 'ora.ctssd' on 'dm01db06'
CRS-2673:
Attempting to stop 'ora.evmd' on 'dm01db06'
CRS-2673:
Attempting to stop 'ora.storage' on 'dm01db06'
CRS-2677:
Stop of 'ora.storage' on 'dm01db06' succeeded
CRS-2673:
Attempting to stop 'ora.asm' on 'dm01db06'
CRS-2677:
Stop of 'ora.crsd' on 'dm01db05' succeeded
CRS-2673:
Attempting to stop 'ora.ctssd' on 'dm01db05'
CRS-2673:
Attempting to stop 'ora.evmd' on 'dm01db05'
CRS-2673:
Attempting to stop 'ora.storage' on 'dm01db05'
CRS-2677:
Stop of 'ora.storage' on 'dm01db05' succeeded
CRS-2673:
Attempting to stop 'ora.asm' on 'dm01db05'
CRS-2677:
Stop of 'ora.cluster_interconnect.haip' on 'dm01db01' succeeded
CRS-2677:
Stop of 'ora.asm' on 'dm01db08' succeeded
CRS-2673:
Attempting to stop 'ora.cluster_interconnect.haip' on 'dm01db08'
CRS-2677:
Stop of 'ora.ctssd' on 'dm01db06' succeeded
CRS-2677:
Stop of 'ora.evmd' on 'dm01db06' succeeded
CRS-2677:
Stop of 'ora.ctssd' on 'dm01db05' succeeded
CRS-2677:
Stop of 'ora.evmd' on 'dm01db05' succeeded
CRS-2677:
Stop of 'ora.cluster_interconnect.haip' on 'dm01db08' succeeded
CRS-2673:
Attempting to stop 'ora.cssd' on 'dm01db08'
CRS-2677:
Stop of 'ora.cssd' on 'dm01db08' succeeded
CRS-2673:
Attempting to stop 'ora.diskmon' on 'dm01db08'
CRS-2677:
Stop of 'ora.evmd' on 'dm01db01' succeeded
CRS-2673:
Attempting to stop 'ora.cssd' on 'dm01db01'
CRS-2677:
Stop of 'ora.diskmon' on 'dm01db08' succeeded
CRS-2677:
Stop of 'ora.cssd' on 'dm01db01' succeeded
CRS-2673:
Attempting to stop 'ora.diskmon' on 'dm01db01'
CRS-2677:
Stop of 'ora.asm' on 'dm01db07' succeeded
CRS-2673:
Attempting to stop 'ora.cluster_interconnect.haip' on 'dm01db07'
CRS-2677:
Stop of 'ora.diskmon' on 'dm01db01' succeeded
CRS-2677:
Stop of 'ora.asm' on 'dm01db04' succeeded
CRS-2673:
Attempting to stop 'ora.cluster_interconnect.haip' on 'dm01db04'
CRS-2677:
Stop of 'ora.cluster_interconnect.haip' on 'dm01db04' succeeded
CRS-2673:
Attempting to stop 'ora.cssd' on 'dm01db04'
CRS-2677:
Stop of 'ora.cluster_interconnect.haip' on 'dm01db07' succeeded
CRS-2673:
Attempting to stop 'ora.cssd' on 'dm01db07'
CRS-2677:
Stop of 'ora.evmd' on 'dm01db02' succeeded
CRS-2673:
Attempting to stop 'ora.cssd' on 'dm01db02'
CRS-2677:
Stop of 'ora.cssd' on 'dm01db04' succeeded
CRS-2673:
Attempting to stop 'ora.diskmon' on 'dm01db04'
CRS-2677:
Stop of 'ora.cssd' on 'dm01db07' succeeded
CRS-2673:
Attempting to stop 'ora.diskmon' on 'dm01db07'
CRS-2677:
Stop of 'ora.cssd' on 'dm01db02' succeeded
CRS-2673:
Attempting to stop 'ora.diskmon' on 'dm01db02'
CRS-2677:
Stop of 'ora.diskmon' on 'dm01db02' succeeded
CRS-2677:
Stop of 'ora.diskmon' on 'dm01db07' succeeded
CRS-2677:
Stop of 'ora.diskmon' on 'dm01db04' succeeded
CRS-2677:
Stop of 'ora.asm' on 'dm01db05' succeeded
CRS-2673:
Attempting to stop 'ora.cluster_interconnect.haip' on 'dm01db05'
CRS-2677:
Stop of 'ora.cluster_interconnect.haip' on 'dm01db05' succeeded
CRS-2673:
Attempting to stop 'ora.cssd' on 'dm01db05'
CRS-2677:
Stop of 'ora.cssd' on 'dm01db05' succeeded
CRS-2673:
Attempting to stop 'ora.diskmon' on 'dm01db05'
CRS-2677:
Stop of 'ora.asm' on 'dm01db06' succeeded
CRS-2673:
Attempting to stop 'ora.cluster_interconnect.haip' on 'dm01db06'
CRS-2677:
Stop of 'ora.diskmon' on 'dm01db05' succeeded
CRS-2677:
Stop of 'ora.cluster_interconnect.haip' on 'dm01db06' succeeded
CRS-2673:
Attempting to stop 'ora.cssd' on 'dm01db06'
CRS-2677:
Stop of 'ora.cssd' on 'dm01db06' succeeded
CRS-2673:
Attempting to stop 'ora.diskmon' on 'dm01db06'
CRS-2677:
Stop of 'ora.diskmon' on 'dm01db06' succeeded
[root@dm01db01
bin]# ps -ef|grep grid
root 2411
1 0 Sep02 ? 03:07:44 /u01/app/12.1.0.2/grid/jdk/jre/bin/java
-Xms128m -Xmx512m –classpath /u01/app/12.1.0.2/grid/tfa/dm01db01/tfa_home/jlib/RATFA.jar:/u01/app/12.1.0.2/grid/tfa/dm01db01/tfa_home/jlib/je-5.0.84.jar:/u01/app/12.1.0.2/grid/tfa/dm01db01/tfa_home/jlib/ojdbc6.jar:/u01/app/12.1.0.2/grid/tfa/dm01db01/tfa_home/jlib/commons-io-2.2.jar
oracle.rat.tfa.TFAMain /u01/app/12.1.0.2/grid/tfa/dm01db01/tfa_home
root 13292
1 0 Sep02 ? 07:14:18
/u01/app/12.1.0.2/grid/bin/osysmond.bin
root 18910
1 0 Sep02 ? 03:54:06 /u01/app/12.1.0.2/grid/bin/ohasd.bin
reboot
root 18960
1 0 Sep02 ? 02:33:20
/u01/app/12.1.0.2/grid/bin/orarootagent.bin
oracle 19021
1 0 Sep02 ? 02:01:17
/u01/app/12.1.0.2/grid/bin/oraagent.bin
oracle 19034
1 0 Sep02 ? 00:54:37
/u01/app/12.1.0.2/grid/bin/mdnsd.bin
oracle 19070
1 0 Sep02 ? 00:59:15
/u01/app/12.1.0.2/grid/bin/gpnpd.bin
oracle 19205
1 2 Sep02 ? 15:31:16
/u01/app/12.1.0.2/grid/bin/gipcd.bin
root 20098
1583 0 01:54 ? 00:00:00 /bin/sh
/u01/app/12.1.0.2/grid/tfa/dm01db01/tfa_home/bin/tfactl -check
root 20106 20098 0 01:54 ? 00:00:00
/u01/app/12.1.0.2/grid/perl/bin/perl
/u01/app/12.1.0.2/grid/tfa/dm01db01/tfa_home/bin/tfactl.pl
-check
root 20120 20106 0 01:54 ? 00:00:00
/u01/app/12.1.0.2/grid/jdk/jre/bin/java -Xms128m -Xmx512m -Djavax.net.ssl.trustStore=/u01/app/12.1.0.2/grid/tfa/dm01db01/tfa_home/public.jks
–classpath /u01/app/12.1.0.2/grid/tfa/dm01db01/tfa_home/jlib/RATFA.jar:/u01/app/12.1.0.2/grid/tfa/dm01db01/tfa_home/jlib/je-
5.0.84.jar:/u01/app/12.1.0.2/grid/tfa/dm01db01/tfa_home/jlib/ojdbc6.jar:/u01/app/12.1.0.2/grid/tfa/dm01db01/tfa_home/jlib/commons-io-2.2.jar
oracle.rat.tfa.CommandLine dm01db01:checkTFAMain -port 5000
root 20156 30058 0 01:54 pts/0 00:00:00 grep grid
[root@dm01db01
bin]# dcli -g ~/dbs_group -l root '/u01/app/12.1.0.2/grid/bin/crsctl stop crs'
dm01db01:
CRS-2791: Starting shutdown of Oracle High Availability Services-managed
resources on 'dm01db01'
dm01db01:
CRS-2673: Attempting to stop 'ora.crf' on 'dm01db01'
dm01db01:
CRS-2673: Attempting to stop 'ora.mdnsd' on 'dm01db01'
dm01db01:
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'dm01db01'
dm01db01:
CRS-2673: Attempting to stop 'ora.gpnpd' on 'dm01db01'
dm01db01:
CRS-2677: Stop of 'ora.drivers.acfs' on 'dm01db01' succeeded
dm01db01:
CRS-2677: Stop of 'ora.mdnsd' on 'dm01db01' succeeded
dm01db01:
CRS-2677: Stop of 'ora.crf' on 'dm01db01' succeeded
dm01db01:
CRS-2673: Attempting to stop 'ora.gipcd' on 'dm01db01'
dm01db01:
CRS-2677: Stop of 'ora.gpnpd' on 'dm01db01' succeeded
dm01db01:
CRS-2677: Stop of 'ora.gipcd' on 'dm01db01' succeeded
dm01db01:
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on
'dm01db01' has completed
dm01db01:
CRS-4133: Oracle High Availability Services has been stopped.
dm01db02:
CRS-2791: Starting shutdown of Oracle High Availability Services-managed
resources on 'dm01db02'
dm01db02:
CRS-2673: Attempting to stop 'ora.crf' on 'dm01db02'
dm01db02:
CRS-2673: Attempting to stop 'ora.mdnsd' on 'dm01db02'
dm01db02:
CRS-2673: Attempting to stop 'ora.gpnpd' on 'dm01db02'
dm01db02:
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'dm01db02'
dm01db02:
CRS-2677: Stop of 'ora.drivers.acfs' on 'dm01db02' succeeded
dm01db02:
CRS-2677: Stop of 'ora.crf' on 'dm01db02' succeeded
dm01db02:
CRS-2677: Stop of 'ora.mdnsd' on 'dm01db02' succeeded
dm01db02:
CRS-2673: Attempting to stop 'ora.gipcd' on 'dm01db02'
dm01db02:
CRS-2677: Stop of 'ora.gpnpd' on 'dm01db02' succeeded
dm01db02:
CRS-2677: Stop of 'ora.gipcd' on 'dm01db02' succeeded
dm01db02:
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on
'dm01db02' has completed
dm01db02:
CRS-4133: Oracle High Availability Services has been stopped.
dm01db03:
CRS-2791: Starting shutdown of Oracle High Availability Services-managed
resources on 'dm01db03'
dm01db03:
CRS-2673: Attempting to stop 'ora.gpnpd' on 'dm01db03'
dm01db03:
CRS-2673: Attempting to stop 'ora.crf' on 'dm01db03'
dm01db03:
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'dm01db03'
dm01db03:
CRS-2673: Attempting to stop 'ora.mdnsd' on 'dm01db03'
dm01db03:
CRS-2677: Stop of 'ora.drivers.acfs' on 'dm01db03' succeeded
dm01db03:
CRS-2677: Stop of 'ora.crf' on 'dm01db03' succeeded
dm01db03:
CRS-2673: Attempting to stop 'ora.gipcd' on 'dm01db03'
dm01db03:
CRS-2677: Stop of 'ora.gpnpd' on 'dm01db03' succeeded
dm01db03:
CRS-2677: Stop of 'ora.mdnsd' on 'dm01db03' succeeded
dm01db03:
CRS-2677: Stop of 'ora.gipcd' on 'dm01db03' succeeded
dm01db03:
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on
'dm01db03' has completed
dm01db03:
CRS-4133: Oracle High Availability Services has been stopped.
dm01db04:
CRS-2791: Starting shutdown of Oracle High Availability Services-managed
resources on 'dm01db04'
dm01db04:
CRS-2673: Attempting to stop 'ora.mdnsd' on 'dm01db04'
dm01db04:
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'dm01db04'
dm01db04:
CRS-2673: Attempting to stop 'ora.crf' on 'dm01db04'
dm01db04:
CRS-2673: Attempting to stop 'ora.gpnpd' on 'dm01db04'
dm01db04:
CRS-2677: Stop of 'ora.drivers.acfs' on 'dm01db04' succeeded
dm01db04:
CRS-2677: Stop of 'ora.mdnsd' on 'dm01db04' succeeded
dm01db04:
CRS-2677: Stop of 'ora.crf' on 'dm01db04' succeeded
dm01db04:
CRS-2673: Attempting to stop 'ora.gipcd' on 'dm01db04'
dm01db04:
CRS-2677: Stop of 'ora.gpnpd' on 'dm01db04' succeeded
dm01db04:
CRS-2677: Stop of 'ora.gipcd' on 'dm01db04' succeeded
dm01db04:
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on
'dm01db04' has completed
dm01db04:
CRS-4133: Oracle High Availability Services has been stopped.
dm01db05:
CRS-2791: Starting shutdown of Oracle High Availability Services-managed
resources on 'dm01db05'
dm01db05:
CRS-2673: Attempting to stop 'ora.crf' on 'dm01db05'
dm01db05:
CRS-2673: Attempting to stop 'ora.gpnpd' on 'dm01db05'
dm01db05:
CRS-2673: Attempting to stop 'ora.mdnsd' on 'dm01db05'
dm01db05:
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'dm01db05'
dm01db05:
CRS-2677: Stop of 'ora.drivers.acfs' on 'dm01db05' succeeded
dm01db05:
CRS-2677: Stop of 'ora.crf' on 'dm01db05' succeeded
dm01db05:
CRS-2677: Stop of 'ora.gpnpd' on 'dm01db05' succeeded
dm01db05:
CRS-2673: Attempting to stop 'ora.gipcd' on 'dm01db05'
dm01db05:
CRS-2677: Stop of 'ora.mdnsd' on 'dm01db05' succeeded
dm01db05:
CRS-2677: Stop of 'ora.gipcd' on 'dm01db05' succeeded
dm01db05:
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'dm01db05'
has completed
dm01db05:
CRS-4133: Oracle High Availability Services has been stopped.
dm01db06:
CRS-2791: Starting shutdown of Oracle High Availability Services-managed
resources on 'dm01db06'
dm01db06:
CRS-2673: Attempting to stop 'ora.crf' on 'dm01db06'
dm01db06:
CRS-2673: Attempting to stop 'ora.mdnsd' on 'dm01db06'
dm01db06:
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'dm01db06'
dm01db06:
CRS-2673: Attempting to stop 'ora.gpnpd' on 'dm01db06'
dm01db06:
CRS-2677: Stop of 'ora.drivers.acfs' on 'dm01db06' succeeded
dm01db06:
CRS-2677: Stop of 'ora.crf' on 'dm01db06' succeeded
dm01db06:
CRS-2673: Attempting to stop 'ora.gipcd' on 'dm01db06'
dm01db06:
CRS-2677: Stop of 'ora.mdnsd' on 'dm01db06' succeeded
dm01db06:
CRS-2677: Stop of 'ora.gpnpd' on 'dm01db06' succeeded
dm01db06:
CRS-2677: Stop of 'ora.gipcd' on 'dm01db06' succeeded
dm01db06:
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on
'dm01db06' has completed
dm01db06:
CRS-4133: Oracle High Availability Services has been stopped.
dm01db07:
CRS-2791: Starting shutdown of Oracle High Availability Services-managed
resources on 'dm01db07'
dm01db07:
CRS-2673: Attempting to stop 'ora.mdnsd' on 'dm01db07'
dm01db07:
CRS-2673: Attempting to stop 'ora.crf' on 'dm01db07'
dm01db07:
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'dm01db07'
dm01db07:
CRS-2673: Attempting to stop 'ora.gpnpd' on 'dm01db07'
dm01db07:
CRS-2677: Stop of 'ora.drivers.acfs' on 'dm01db07' succeeded
dm01db07:
CRS-2677: Stop of 'ora.mdnsd' on 'dm01db07' succeeded
dm01db07:
CRS-2677: Stop of 'ora.crf' on 'dm01db07' succeeded
dm01db07:
CRS-2673: Attempting to stop 'ora.gipcd' on 'dm01db07'
dm01db07:
CRS-2677: Stop of 'ora.gpnpd' on 'dm01db07' succeeded
dm01db07:
CRS-2677: Stop of 'ora.gipcd' on 'dm01db07' succeeded
dm01db07:
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on
'dm01db07' has completed
dm01db07:
CRS-4133: Oracle High Availability Services has been stopped.
dm01db08:
CRS-2791: Starting shutdown of Oracle High Availability Services-managed
resources on 'dm01db08'
dm01db08:
CRS-2673: Attempting to stop 'ora.mdnsd' on 'dm01db08'
dm01db08:
CRS-2673: Attempting to stop 'ora.crf' on 'dm01db08'
dm01db08:
CRS-2673: Attempting to stop 'ora.gpnpd' on 'dm01db08'
dm01db08:
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'dm01db08'
dm01db08:
CRS-2677: Stop of 'ora.drivers.acfs' on 'dm01db08' succeeded
dm01db08:
CRS-2677: Stop of 'ora.crf' on 'dm01db08' succeeded
dm01db08:
CRS-2673: Attempting to stop 'ora.gipcd' on 'dm01db08'
dm01db08:
CRS-2677: Stop of 'ora.mdnsd' on 'dm01db08' succeeded
dm01db08:
CRS-2677: Stop of 'ora.gpnpd' on 'dm01db08' succeeded
dm01db08:
CRS-2677: Stop of 'ora.gipcd' on 'dm01db08' succeeded
dm01db08:
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on
'dm01db08' has completed
dm01db08:
CRS-4133: Oracle High Availability Services has been stopped.
[root@dm01db01
bin]# ps -ef|grep grid
root 2411
1 0 Sep02 ? 03:07:44
/u01/app/12.1.0.2/grid/jdk/jre/bin/java -Xms128m -Xmx512m –classpath /u01/app/12.1.0.2/grid/tfa/dm01db01/tfa_home/jlib/RATFA.jar:/u01/app/12.1.0.2/grid/tfa/dm01db01/tfa_home/jlib/je-
5.0.84.jar:/u01/app/12.1.0.2/grid/tfa/dm01db01/tfa_home/jlib/ojdbc6.jar:/u01/app/12.1.0.2/grid/tfa/dm01db01/tfa_home/jlib/commons-io-2.2.jar
oracle.rat.tfa.TFAMain /u01/app/12.1.0.2/grid/tfa/dm01db01/tfa_home
root 26496 30058 0 01:54 pts/0 00:00:00 grep grid
[root@dm01db01
bin]# dcli -g ~/dbs_group -l root '/u01/app/12.1.0.2/grid/bin/crsctl check crs'
dm01db01:
CRS-4639: Could not contact Oracle High Availability Services
dm01db02:
CRS-4639: Could not contact Oracle High Availability Services
dm01db03:
CRS-4639: Could not contact Oracle High Availability Services
dm01db04:
CRS-4639: Could not contact Oracle High Availability Services
dm01db05:
CRS-4639: Could not contact Oracle High Availability Services
dm01db06:
CRS-4639: Could not contact Oracle High Availability Services
dm01db07:
CRS-4639: Could not contact Oracle High Availability Services
dm01db08:
CRS-4639: Could not contact Oracle High Availability Services
- Get the current Exadata Storage Software version:
Compute Node:
[root@dm01db01
bin]# imageinfo
Kernel
version: 2.6.39-400.248.3.el6uek.x86_64 #1 SMP Wed Mar 11 18:04:34 PDT 2015
x86_64
Image version: 12.1.2.1.1.150316.2
Image
activated: 2015-04-27 20:52:07 -0500
Image
status: success
System
partition on device: /dev/mapper/VGExaDb-LVDbSys1
Storage Cell
[root@dm01db01
bin]# ssh dm01cel01 imageinfo
Kernel
version: 2.6.39-400.248.3.el6uek.x86_64 #1 SMP Wed Mar 11 18:04:34 PDT 2015
x86_64
Cell
version: OSS_12.1.2.1.1_LINUX.X64_150316.2
Cell
rpm version: cell-12.1.2.1.1_LINUX.X64_150316.2-1.x86_64
Active image version:
12.1.2.1.1.150316.2
Active
image activated: 2015-04-27 16:55:04 -0500
Active
image status: success
Active
system partition on device: /dev/md5
Active
stage partition on device: /dev/md7
Cell
boot usb partition: /dev/sdac1
Cell
boot usb version: 12.1.2.1.1.150316.2
Inactive
image version: 12.1.2.1.0.141206.1
Inactive
image activated: 2015-02-25 07:39:28 -0600
Inactive
image status: success
Inactive
system partition on device: /dev/md6
Inactive
stage partition on device: /dev/md8
Inactive
marker for the rollback: /boot/I_am_hd_boot.inactive
Inactive
grub config for the rollback: /boot/grub/grub.conf.inactive
Inactive
kernel version for the rollback: 2.6.39-400.243.1.el6uek.x86_64
Rollback
to the inactive partitions: Possible
- Shutdown the Cell services on all the Storage cells as follows:
[root@dm01db01
bin]# dcli -g ~/cell_group -l root "cellcli -e alter cell shutdown
services all"
dm01cel01:
dm01cel01:
Stopping the RS, CELLSRV, and MS services...
dm01cel01:
The SHUTDOWN of services was successful.
dm01cel02:
dm01cel02:
Stopping the RS, CELLSRV, and MS services...
dm01cel02:
The SHUTDOWN of services was successful.
dm01cel03:
dm01cel03:
Stopping the RS, CELLSRV, and MS services...
dm01cel03:
The SHUTDOWN of services was successful.
dm01cel04:
dm01cel04:
Stopping the RS, CELLSRV, and MS services...
dm01cel04:
The SHUTDOWN of services was successful.
dm01cel05:
dm01cel05:
Stopping the RS, CELLSRV, and MS services...
dm01cel05:
The SHUTDOWN of services was successful.
dm01cel06:
dm01cel06:
Stopping the RS, CELLSRV, and MS services...
dm01cel06:
The SHUTDOWN of services was successful.
dm01cel07:
dm01cel07:
Stopping the RS, CELLSRV, and MS services...
dm01cel07:
The SHUTDOWN of services was successful.
dm01cel08:
dm01cel08:
Stopping the RS, CELLSRV, and MS services...
dm01cel08:
The SHUTDOWN of services was successful.
dm01cel09:
dm01cel09:
Stopping the RS, CELLSRV, and MS services...
dm01cel09:
The SHUTDOWN of services was successful.
dm01cel10:
dm01cel10:
Stopping the RS, CELLSRV, and MS services...
dm01cel10:
The SHUTDOWN of services was successful.
dm01cel11:
dm01cel11:
Stopping the RS, CELLSRV, and MS services...
dm01cel11:
The SHUTDOWN of services was successful.
dm01cel12:
dm01cel12:
Stopping the RS, CELLSRV, and MS services...
dm01cel12:
The SHUTDOWN of services was successful.
dm01cel13:
dm01cel13:
Stopping the RS, CELLSRV, and MS services...
dm01cel13:
The SHUTDOWN of services was successful.
dm01cel14:
dm01cel14:
Stopping the RS, CELLSRV, and MS services...
dm01cel14:
The SHUTDOWN of services was successful.
- Navigate to the patch staging directory and unzip the storage cell software
[root@dm01db01
bin]# cd /u01/stage/ESS_Patch/
[root@dm01db01
ESS_Patch]# ls -l
total
2537072
-rw-r--r--
1 root root 1550028016 Sep 27 07:39 p20131726_121220_Linux-x86-64.zip
[root@dm01db01
ESS_Patch]# unzip p20131726_121220_Linux-x86-64.zip
Archive: p20131726_121220_Linux-x86-64.zip
creating: patch_12.1.2.2.0.150917/
creating: patch_12.1.2.2.0.150917/linux.db.rpms/
inflating:
patch_12.1.2.2.0.150917/linux.db.rpms/perl-XML-Parser-2.34-6.1.2.2.1.x86_64.rpm
inflating: patch_12.1.2.2.0.150917/dostep.sh
inflating:
patch_12.1.2.2.0.150917/sundcs_36p_repository_2.1.6_2.pkg
inflating: patch_12.1.2.2.0.150917/11_2_2_1_0.pm
inflating:
patch_12.1.2.2.0.150917/11_2_1_1_0.pm
inflating:
patch_12.1.2.2.0.150917/md5sum_files.lst
inflating:
patch_12.1.2.2.0.150917/ExaXMLNode.pm
inflating:
patch_12.1.2.2.0.150917/exadataLogger.pm
creating: patch_12.1.2.2.0.150917/ibdiagtools/
inflating:
patch_12.1.2.2.0.150917/ibdiagtools/cable_check.pl
inflating:
patch_12.1.2.2.0.150917/ibdiagtools/SampleOutputs.txt
creating:
patch_12.1.2.2.0.150917/ibdiagtools/netcheck/
inflating: patch_12.1.2.2.0.150917/ibdiagtools/netcheck/spawnProc.pm
inflating:
patch_12.1.2.2.0.150917/ibdiagtools/netcheck/remoteScriptGenerator.pm
inflating:
patch_12.1.2.2.0.150917/ibdiagtools/netcheck/OSAdapter.pm
inflating:
patch_12.1.2.2.0.150917/ibdiagtools/netcheck/runDiagnostics.pm
inflating:
patch_12.1.2.2.0.150917/ibdiagtools/netcheck/SolarisAdapter.pm
inflating:
patch_12.1.2.2.0.150917/ibdiagtools/netcheck/LinuxAdapter.pm
inflating:
patch_12.1.2.2.0.150917/ibdiagtools/netcheck/remoteConfig.pm
inflating: patch_12.1.2.2.0.150917/ibdiagtools/netcheck/CommonUtils.pm
inflating:
patch_12.1.2.2.0.150917/ibdiagtools/netcheck/remoteLauncher.pm
inflating:
patch_12.1.2.2.0.150917/ibdiagtools/monitord
inflating:
patch_12.1.2.2.0.150917/ibdiagtools/README
creating: patch_12.1.2.2.0.150917/ibdiagtools/topologies/
inflating:
patch_12.1.2.2.0.150917/ibdiagtools/topologies/Node.pm
inflating:
patch_12.1.2.2.0.150917/ibdiagtools/topologies/Group.pm
inflating:
patch_12.1.2.2.0.150917/ibdiagtools/topologies/VerifyTopologyUtility.pm
inflating:
patch_12.1.2.2.0.150917/ibdiagtools/topologies/Switch.pm
inflating:
patch_12.1.2.2.0.150917/ibdiagtools/topologies/verifylib.pm
inflating:
patch_12.1.2.2.0.150917/ibdiagtools/topologies/Rack.pm
inflating: patch_12.1.2.2.0.150917/ibdiagtools/setup-ssh
extracting:
patch_12.1.2.2.0.150917/ibdiagtools/xmonib.sh
inflating:
patch_12.1.2.2.0.150917/ibdiagtools/dcli
inflating:
patch_12.1.2.2.0.150917/ibdiagtools/ibping_test
inflating:
patch_12.1.2.2.0.150917/ibdiagtools/checkbadlinks.pl
extracting:
patch_12.1.2.2.0.150917/ibdiagtools/VERSION_FILE
inflating:
patch_12.1.2.2.0.150917/ibdiagtools/infinicheck
inflating:
patch_12.1.2.2.0.150917/ibdiagtools/verify-topology
inflating:
patch_12.1.2.2.0.150917/ibdiagtools/tar_ibdiagtools
inflating:
patch_12.1.2.2.0.150917/ibdiagtools/topology-zfs
creating: patch_12.1.2.2.0.150917/plugins/
inflating:
patch_12.1.2.2.0.150917/plugins/000-check_dummy_bash
inflating:
patch_12.1.2.2.0.150917/plugins/000-check_dummy_perl
inflating: patch_12.1.2.2.0.150917/plugins/010-check_17854520.sh
inflating:
patch_12.1.2.2.0.150917/11_2_1_2_1.pm
inflating:
patch_12.1.2.2.0.150917/12.1.2.2.0.150917.patch.tar
inflating:
patch_12.1.2.2.0.150917/sundcs_36p_repository_2.1.5_1.pkg
inflating: patch_12.1.2.2.0.150917/ExadataImageNotification.pl
inflating:
patch_12.1.2.2.0.150917/exadata.img.hw
inflating: patch_12.1.2.2.0.150917/README.txt
creating: patch_12.1.2.2.0.150917/etc/
creating:
patch_12.1.2.2.0.150917/etc/config/
inflating: patch_12.1.2.2.0.150917/etc/config/inventory.xml
inflating: patch_12.1.2.2.0.150917/sgutil.pm
inflating: patch_12.1.2.2.0.150917/dcli
inflating: patch_12.1.2.2.0.150917/patchmgr
inflating:
patch_12.1.2.2.0.150917/11_2_1_2_2.pm
inflating: patch_12.1.2.2.0.150917/imageLogger
inflating:
patch_12.1.2.2.0.150917/preconfig.pm
inflating:
patch_12.1.2.2.0.150917/12.1.2.2.0.150917.iso
inflating: patch_12.1.2.2.0.150917/ipconf.pl
inflating:
patch_12.1.2.2.0.150917/ExadataSendNotification.pm
inflating:
patch_12.1.2.2.0.150917/README.html
inflating:
patch_12.1.2.2.0.150917/upgradeIBSwitch.sh
inflating:
patch_12.1.2.2.0.150917/11_1_3_2_0.pm
inflating:
patch_12.1.2.2.0.150917/dostep.sh.tmpl
inflating:
patch_12.1.2.2.0.150917/exadata.img.env
- Ensure root user ssh equivalence is setup between compute node and all storage cells
[root@dm01db01
ESS_Patch]# dcli -g ~/cell_group -l root 'hostname'
dm01cel01:
dm01cel01.domain.com
dm01cel02:
dm01cel02.domain.com
dm01cel03:
dm01cel03.domain.com
dm01cel04:
dm01cel04.domain.com
dm01cel05:
dm01cel05.domain.com
dm01cel06:
dm01cel06.domain.com
dm01cel07:
dm01cel07.domain.com
dm01cel08:
dm01cel08.domain.com
dm01cel09:
dm01cel09.domain.com
dm01cel10:
dm01cel10.domain.com
dm01cel11:
dm01cel11.domain.com
dm01cel12:
dm01cel12.domain.com
dm01cel13:
dm01cel13.domain.com
dm01cel14:
dm01cel14.domain.com
- Change directory to the actual patch directory after unzip
[root@dm01db01
ESS_Patch]# cd patch_12.1.2.2.0.150917/
- Copy the cell_group file to the storage software directory
[root@dm01db01
patch_12.1.2.2.0.150917]# cp ~/cell_group .
[root@dm01db01
patch_12.1.2.2.0.150917]# ls -l
total
1543432
-r-xr-xr-x
1 root root 10769 Oct 9 2013
11_1_3_2_0.pm
-r-xr-xr-x
1 root root 29096 Oct 9 2013
11_2_1_1_0.pm
-r-xr-xr-x
1 root root 32529 Oct 9 2013
11_2_1_2_1.pm
-r-xr-xr-x
1 root root 33620 Oct 9 2013
11_2_1_2_2.pm
-r-xr-xr-x
1 root root 43240 Apr 19 2014 11_2_2_1_0.pm
-rw-rw-r--
1 root root 1343023104 Sep 17 19:54 12.1.2.2.0.150917.iso
-rw-rw-r--
1 root root 2140160 Sep 17 19:54
12.1.2.2.0.150917.patch.tar
-rwxr-xr--
1 root root 140 Oct 3 02:02 cell_group
-r-xr-xr-x
1 root root 49094 Sep 17 19:46 dcli
-r-xr-xr-x
1 root root 111833 Sep 17 19:54
dostep.sh
-r-xr-xr-x
1 root root 111833 Sep 17 19:54
dostep.sh.tmpl
drwxrwxr-x
3 root root 4096 Sep 17 19:54 etc
-r-xr-xr-x
1 root root 56712 Sep 17 19:46
ExadataImageNotification.pl
-r-xr-xr-x
1 root root 2343 Dec 2 2014
exadata.img.env
-r-xr-xr-x
1 root root 16131 May 29 02:42
exadata.img.hw
-r-xr-xr-x
1 root root 53928 Oct 9 2013
exadataLogger.pm
-r-xr-xr-x
1 root root 1570 Sep 17 19:46
ExadataSendNotification.pm
-r-xr-xr-x
1 root root 6133 Oct 9 2013
ExaXMLNode.pm
drwxr-xr-x
4 root root 4096 Sep 17 19:54 ibdiagtools
-r-xr-xr-x
1 root root 42496 Jul 15 02:54
imageLogger
-r-xr-xr-x
1 root root 511906 Sep 4 02:48 ipconf.pl
drwxrwxr-x
2 root root 4096 Sep 17 19:54
linux.db.rpms
-rw-rw-r--
1 root root 3155 Sep 17 19:54 md5sum_files.lst
-r-xr-xr-x
1 root root 187138 Sep 17 19:54
patchmgr
drwxr-xr-x
2 root root 4096 Sep 17 19:46
plugins
-r-xr-xr-x
1 root root 41891 Jul 29 02:39
preconfig.pm
-rw-rw-r--
1 root root 73916 Sep 22 15:41
README.html
-r-xr-xr-x
1 root root 148 Sep 17 19:46
README.txt
-r-xr-xr-x
1 root root 47458 Sep 3 05:04 sgutil.pm
-r--r--r--
1 root root 116010429 Sep 17 19:46
sundcs_36p_repository_2.1.5_1.pkg
-r--r--r--
1 root root 116051693 Sep 17 19:46
sundcs_36p_repository_2.1.6_2.pkg
-r-xr-xr-x
1 root root 114855 Sep 17 19:46
upgradeIBSwitch.sh
- It is recommended to reset the storage cells to a known state using the following command
[root@dm01db01
patch_12.1.2.2.0.150917]# ./patchmgr -cells cell_group -reset_force
2015-10-03
02:03:00 -0500 :DONE: reset_force
- Clean up any previous patchmgr utility runs using the following command
[root@dm01db01
patch_12.1.2.2.0.150917]# ./patchmgr -cells cell_group -cleanup
2015-10-03
02:04:14 -0500 :Working: DO:
Cleanup ...
2015-10-03
02:04:16 -0500 :SUCCESS: DONE:
Cleanup
- Verify that the storage cells meet prerequisite checks using the following command.
Use
the -rolling option if you plan to use rolling updates.
Use
the -smtp_from and -smtp_to options to send patching progress e-mail
notifications.
[root@dm01db01
patch_12.1.2.2.0.150917]# ./patchmgr -cells cell_group -patch_check_prereq
-smtp_from "dm01db01" -smtp_to "abdul.mohammed@netsoftmate.com"
2015-10-11
04:17:59 -0500 :Working: DO: Check
cells have ssh equivalence for root user. Up to 10 seconds per cell ...
2015-10-11
04:18:02 -0500 :SUCCESS: DONE:
Check cells have ssh equivalence for root user.
2015-10-11
04:18:13 -0500 :Working: DO:
Initialize files, check space and state of cell services. Up to 1 minute ...
2015-10-11
04:18:52 -0500 :SUCCESS: DONE:
Initialize files, check space and state of cell services.
2015-10-11
04:18:52 -0500 :Working: DO: Copy,
extract prerequisite check archive to cells. If required start md11 mismatched
partner size correction. Up to 40 minutes ...
2015-10-11
04:19:08 -0500 Wait correction of degraded md11 due to md partner size
mismatch. Up to 30 minutes.
2015-10-11
04:19:09 -0500 :SUCCESS: DONE:
Copy, extract prerequisite check archive to cells. If required start md11
mismatched partner size correction.
2015-10-11
04:19:09 -0500 :Working: DO: Check
prerequisites on all cells. Up to 2 minutes ...
2015-10-11
04:20:33 -0500 :SUCCESS: DONE:
Check prerequisites on all cells.
2015-10-11
04:20:33 -0500 :Working: DO:
Execute plugin check for Patch Check Prereq ...
2015-10-11
04:20:33 -0500 :INFO: Patchmgr plugin start: Prereq check for exposure to bug
17854520 v1.3. Details
in
logfile /u01/stage/ESS_Patch/patch_12.1.2.2.0.150917/patchmgr.stdout.
2015-10-11
04:20:33 -0500 :SUCCESS: No exposure to bug 17854520 with non-rolling patching
2015-10-11
04:20:33 -0500 :SUCCESS: DONE:
Execute plugin check for Patch Check Prereq.
[root@dm01db01
patch_12.1.2.2.0.150917]#
- The prerequisite checks completed successfully.
We
can now start the patch application.
Use
the -rolling option if you plan to use rolling updates.
Use
the -smtp_from and -smtp_to options to send patching progress e-mail
notifications.
[root@dm01db01
patch_12.1.2.2.0.150917]# ./patchmgr -cells ~/cell_group -patch -smtp_from
"dm01db01" -smtp_to "abdul.mohammed@netsoftamte.com"
********************************************************************************
NOTE
Cells will reboot during the patch or rollback process.
NOTE
For non-rolling patch or rollback, ensure all ASM instances using
NOTE
the cells are shut down for the duration of the patch or rollback.
NOTE
For rolling patch or rollback, ensure all ASM instances using
NOTE
the cells are up for the duration of the patch or rollback.
WARNING
Do not start more than one instance of patchmgr.
WARNING
Do not interrupt the patchmgr session.
WARNING
Do not alter state of ASM instances during patch or rollback.
WARNING
Do not resize the screen. It may disturb the screen layout.
WARNING
Do not reboot cells or alter cell services during patch or rollback.
WARNING
Do not open log files in editor in write mode or try to alter them.
NOTE
All time estimates are approximate.
NOTE
You may interrupt this patchmgr run in next 60 seconds with CONTROL-c.
********************************************************************************
2015-10-18
05:11:01 -0500 :Working: DO: Check
cells have ssh equivalence for root user. Up to 10 seconds per cell ...
2015-10-18
05:11:03 -0500 :SUCCESS: DONE:
Check cells have ssh equivalence for root user.
2015-10-18
05:11:14 -0500 :Working: DO:
Initialize files, check space and state of cell services. Up to 1 minute ...
2015-10-18
05:11:43 -0500 :SUCCESS: DONE:
Initialize files, check space and state of cell services.
2015-10-18
05:11:43 -0500 :Working: DO: Copy,
extract prerequisite check archive to cells. If required start md11 mismatched
partner size correction. Up to 40 minutes ...
2015-10-18
05:11:59 -0500 Wait correction of degraded md11 due to md partner size
mismatch. Up to 30 minutes.
2015-10-18
05:12:00 -0500 :SUCCESS: DONE:
Copy, extract prerequisite check archive to cells. If required start md11
mismatched partner size correction.
2015-10-18
05:12:00 -0500 :Working: DO: Check
prerequisites on all cells. Up to 2 minutes ...
2015-10-18
05:13:18 -0500 :SUCCESS: DONE:
Check prerequisites on all cells.
2015-10-18
05:13:18 -0500 :Working: DO: Copy
the patch to all cells. Up to 3 minutes ...
2015-10-18
05:15:59 -0500 :SUCCESS: DONE:
Copy the patch to all cells.
2015-10-18
05:16:01 -0500 :Working: DO:
Execute plugin check for Patch Check Prereq ...
2015-10-18
05:16:01 -0500 :INFO: Patchmgr plugin start: Prereq check for exposure to bug
17854520 v1.3. Details
in
logfile /u01/stage/ESS_Patch/patch_12.1.2.2.0.150917/patchmgr.stdout.
2015-10-18
05:16:01 -0500 :SUCCESS: No exposure to bug 17854520 with non-rolling patching
2015-10-18
05:16:01 -0500 :SUCCESS: DONE:
Execute plugin check for Patch Check Prereq.
2015-10-18
05:16:25 -0500 1 of 5 :Working: DO: Initiate patch on cells. Cells will remain
up. Up to 5 minutes
...
2015-10-18
05:16:40 -0500 1 of 5 :SUCCESS: DONE: Initiate patch on cells.
2015-10-18
05:16:40 -0500 2 of 5 :Working: DO: Waiting to finish pre-reboot patch actions.
Cells will remain up. Up to 45 minutes ...
2015-10-18
05:17:40 -0500 Wait for patch pre-reboot procedures
2015-10-18
05:29:05 -0500 2 of 5 :SUCCESS: DONE: Waiting to finish pre-reboot patch
actions.
2015-10-18
05:29:06 -0500 :Working: DO:
Execute plugin check for Patching ...
2015-10-18
05:29:07 -0500 :SUCCESS: DONE:
Execute plugin check for Patching.
2015-10-18
05:29:07 -0500 3 of 5 :Working: DO: Finalize patch on cells. Cells will reboot.
Up to 5 minutes ...
2015-10-18
05:29:28 -0500 3 of 5 :SUCCESS: DONE: Finalize patch on cells.
2015-10-18
05:29:28 -0500 4 of 5 :Working: DO: Wait for cells to reboot and come online.
Up to 120 minutes ...
2015-10-18
05:30:28 -0500 Wait for patch finalization and reboot
TIMEOUT
for following cells
dm01cel01 2015-10-18 08:51:52 -0500
dm01cel02 2015-10-18 08:51:52 -0500
dm01cel03 2015-10-18 08:51:52 -0500
dm01cel04 2015-10-18 08:51:52 -0500
dm01cel05 2015-10-18 08:51:52 -0500
dm01cel06 2015-10-18 08:51:52 -0500
dm01cel07 2015-10-18 08:51:52 -0500
dm01cel08 2015-10-18 08:51:52 -0500
dm01cel09 2015-10-18 08:51:52 -0500
dm01cel10 2015-10-18 08:51:52 -0500
dm01cel11 2015-10-18 08:51:52 -0500
dm01cel12 2015-10-18 08:51:52 -0500
dm01cel13 2015-10-18 08:51:52 -0500
dm01cel14 2015-10-18 08:51:52 -0500
/u01/stage/ESS_Patch/patch_12.1.2.2.0.150917/patchmgr.stdout,
/u01/stage/ESS_Patch/patch_12.1.2.2.0.150917/patchmgr.stderr
2015-10-18
08:51:55 -0500 4 of 5 : DONE: Wait for cells to reboot and come online.
2015-10-18
08:51:55 -0500 5 of 5 :Working: DO: Check the state of patch on cells. Up to 5
minutes ...
2015-10-18
08:54:09 -0500 5 of 5 :SUCCESS: DONE: Check the state of patch on cells.
2015-10-18
08:54:09 -0500 :Working: DO:
Execute plugin check for Post Patch ...
2015-10-18
08:54:13 -0500 :SUCCESS: DONE:
Execute plugin check for Post Patch.
- Check the Storage cell software is showing the new version as follows:
[root@dm01db01
~]# dcli -g cell_group -l root 'imageinfo | grep "Active image
version"'
dm01cel01:
Active image version: 12.1.2.2.0.150917
dm01cel02:
Active image version: 12.1.2.2.0.150917
dm01cel03:
Active image version: 12.1.2.2.0.150917
dm01cel04:
Active image version: 12.1.2.2.0.150917
dm01cel05:
Active image version: 12.1.2.2.0.150917
dm01cel06:
Active image version: 12.1.2.2.0.150917
dm01cel07:
Active image version: 12.1.2.2.0.150917
dm01cel08:
Active image version: 12.1.2.2.0.150917
dm01cel09:
Active image version: 12.1.2.2.0.150917
dm01cel10:
Active image version: 12.1.2.2.0.150917
dm01cel11:
Active image version: 12.1.2.2.0.150917
dm01cel12:
Active image version: 12.1.2.2.0.150917
dm01cel13:
Active image version: 12.1.2.2.0.150917
dm01cel14:
Active image version: 12.1.2.2.0.150917
- Check the previous Storage cell software is marked Inactive as follows:
[root@dm01db01
~]# dcli -g cell_group -l root 'imageinfo | grep "Inactive image
version"'
dm01cel01:
Inactive image version: 12.1.2.1.1.150316.2
dm01cel02:
Inactive image version: 12.1.2.1.1.150316.2
dm01cel03:
Inactive image version: 12.1.2.1.1.150316.2
dm01cel04:
Inactive image version: 12.1.2.1.1.150316.2
dm01cel05:
Inactive image version: 12.1.2.1.1.150316.2
dm01cel06:
Inactive image version: 12.1.2.1.1.150316.2
dm01cel07:
Inactive image version: 12.1.2.1.1.150316.2
dm01cel08:
Inactive image version: 12.1.2.1.1.150316.2
dm01cel09:
Inactive image version: 12.1.2.1.1.150316.2
dm01cel10:
Inactive image version: 12.1.2.1.1.150316.2
dm01cel11:
Inactive image version: 12.1.2.1.1.150316.2
dm01cel12:
Inactive image version: 12.1.2.1.1.150316.2
dm01cel13:
Inactive image version: 12.1.2.1.1.150316.2
dm01cel14:
Inactive image version: 12.1.2.1.1.150316.2
[root@dm01db01
~]#
- Clean up the cells using the -cleanup option to clean up all the temporary patch or rollback files on the cells
[root@dm01db01
patch_12.1.2.2.0.150917]# ./patchmgr -cells cell_group -cleanup
2015-11-04
13:02:12 -0600 :Working: DO:
Cleanup ...
2015-11-04
13:02:48 -0600 :SUCCESS: DONE:
Cleanup
This concludes our Exadata
Storage Cells patching.
Next we will see how to Patch
Exadata Infiniband Switches.
No comments:
Post a Comment