The patchmgr utility can be used for upgrading, rollback and backup Exadata Storage cells. patchmgr utility can be used for upgrading Storage cells in a rolling or non-rolling fashion. Non-Rolling is default. Storage server patches apply operating system, firmware, and driver updates.
Launch patchmgr from the compute node that is node 1 that has user equivalence setup to all the storage cells.
In this article I will demonstrate how to perform upgrade Exadata Storage cells using patchmgr utility.
MOS Notes
Read the following MOS notes carefully.
Software Download
Current Environment
Current Image version
Kernel version: 4.1.12-94.7.8.el6uek.x86_64 #2 SMP Thu Jan 11 20:41:01 PST 2018 x86_64
Cell version: OSS_12.2.1.1.6_LINUX.X64_180125.1
Cell rpm version: cell-12.2.1.1.6_LINUX.X64_180125.1-1.x86_64
Active image version: 12.2.1.1.6.180125.1
Active image kernel version: 4.1.12-94.7.8.el6uek
Active image activated: 2018-05-08 00:42:57 -0500
Active image status: success
Active system partition on device: /dev/md6
Active software partition on device: /dev/md8
Cell boot usb partition: /dev/sdac1
Cell boot usb version: 12.2.1.1.6.180125.1
Inactive image version: 12.1.2.3.6.170713
Inactive image activated: 2017-10-03 00:57:25 -0500
Inactive image status: success
Inactive system partition on device: /dev/md5
Inactive software partition on device: /dev/md7
Inactive marker for the rollback: /boot/I_am_hd_boot.inactive
Inactive grub config for the rollback: /boot/grub/grub.conf.inactive
Inactive kernel version for the rollback: 2.6.39-400.297.1.el6uek.x86_64
Rollback to the inactive partitions: Possible
Prerequisites
dm01cel01: Filesystem Size Used Avail Use% Mounted on
dm01cel01: /dev/md6 9.8G 4.4G 4.9G 48% /
dm01cel02: Filesystem Size Used Avail Use% Mounted on
dm01cel02: /dev/md6 9.8G 4.5G 4.8G 49% /
dm01cel03: Filesystem Size Used Avail Use% Mounted on
dm01cel03: /dev/md6 9.8G 4.5G 4.8G 49% /
dm01cel04: Filesystem Size Used Avail Use% Mounted on
dm01cel04: /dev/md6 9.8G 4.5G 4.8G 49% /
dm01cel05: Filesystem Size Used Avail Use% Mounted on
dm01cel05: /dev/md6 9.8G 4.5G 4.8G 49% /
dm01cel06: Filesystem Size Used Avail Use% Mounted on
dm01cel06: /dev/md6 9.8G 4.6G 4.7G 50% /
dm01cel07: Filesystem Size Used Avail Use% Mounted on
dm01cel07: /dev/md6 9.8G 4.5G 4.8G 48% /
[root@dm01db01 ~]# dcli -l root -g ~/cell_group "cellcli -e list physicaldisk where diskType=FlashDisk and status not = normal"
[root@dm01db01 ~]# dcli -g ~/dbs_group -l root 'dbmcli -e list physicaldisk where status!=normal'
[root@dm01db01 ~]# dcli -g ~/dbs_group -l root 'ipmitool sunoem cli "show -d properties -level all /SYS fault_state==Faulted"'
[root@dm01db01 ~]# dcli -g ~/cell_group -l root 'ipmitool sunoem cli "show -d properties -level all /SYS fault_state==Faulted"'
[root@dm01db01 ~]# dcli -l root -g ~/dbs_group "dbmcli -e drop alerthistory all"
[root@dm01db01 ~]# unzip p29194095_181000_Linux-x86-64.zip
Steps to perform Storage Cell Patching
uid=0(root) gid=0(root) groups=0(root),1(bin),2(daemon),3(sys),4(adm),6(disk),10(wheel)
dm01cel01: 01:46:18 up 194 days, 40 min, 0 users, load average: 0.17, 0.50, 0.61
dm01cel02: 01:46:18 up 194 days, 40 min, 0 users, load average: 0.05, 0.29, 0.45
dm01cel03: 01:46:18 up 194 days, 40 min, 0 users, load average: 0.25, 0.64, 0.63
dm01cel04: 01:46:18 up 194 days, 40 min, 0 users, load average: 0.12, 0.44, 0.53
dm01cel05: 01:46:18 up 194 days, 40 min, 0 users, load average: 0.15, 0.55, 0.65
dm01cel06: 01:46:18 up 194 days, 40 min, 0 users, load average: 0.33, 0.48, 0.55
dm01cel07: 01:46:18 up 194 days, 40 min, 0 users, load average: 0.09, 0.37, 0.52
SQL> select dg.name,a.value from v$asm_diskgroup dg, v$asm_attribute a where dg.group_number=a.group_number and a.name='disk_repair_time';
NAME VALUE
------------------------------ --------------------------
DATA_DM01 3.6H
RECO_DM01 3.6h
[root@dm01db01 ~]# dcli -g dbs_group -l root '/u01/app/11.2.0.4/grid/bin/crsctl stop crs'
Kernel version: 4.1.12-94.7.8.el6uek.x86_64 #2 SMP Thu Jan 11 20:41:01 PST 2018 x86_64
Cell version: OSS_12.2.1.1.6_LINUX.X64_180125.1
Cell rpm version: cell-12.2.1.1.6_LINUX.X64_180125.1-1.x86_64
Active image version: 12.2.1.1.6.180125.1
Active image kernel version: 4.1.12-94.7.8.el6uek
Active image activated: 2018-05-08 00:42:57 -0500
Active image status: success
Active system partition on device: /dev/md6
Active software partition on device: /dev/md8
Cell boot usb partition: /dev/sdac1
Cell boot usb version: 12.2.1.1.6.180125.1
Inactive image version: 12.1.2.3.6.170713
Inactive image activated: 2017-10-03 00:57:25 -0500
Inactive image status: success
Inactive system partition on device: /dev/md5
Inactive software partition on device: /dev/md7
Inactive marker for the rollback: /boot/I_am_hd_boot.inactive
Inactive grub config for the rollback: /boot/grub/grub.conf.inactive
Inactive kernel version for the rollback: 2.6.39-400.297.1.el6uek.x86_64
Rollback to the inactive partitions: Possible
dm01cel01:
dm01cel01: Stopping the RS, CELLSRV, and MS services...
dm01cel01: The SHUTDOWN of services was successful.
dm01cel02:
dm01cel02: Stopping the RS, CELLSRV, and MS services...
dm01cel02: The SHUTDOWN of services was successful.
dm01cel03:
dm01cel03: Stopping the RS, CELLSRV, and MS services...
dm01cel03: The SHUTDOWN of services was successful.
dm01cel04:
dm01cel04: Stopping the RS, CELLSRV, and MS services...
dm01cel04: The SHUTDOWN of services was successful.
dm01cel05:
dm01cel05: Stopping the RS, CELLSRV, and MS services...
dm01cel05: The SHUTDOWN of services was successful.
dm01cel06:
dm01cel06: Stopping the RS, CELLSRV, and MS services...
dm01cel06: The SHUTDOWN of services was successful.
dm01cel07:
dm01cel07: Stopping the RS, CELLSRV, and MS services...
dm01cel07: The SHUTDOWN of services was successful.
2019-02-10 01:56:19 -0600 :Working: DO: Force Cleanup
2019-02-10 01:56:21 -0600 :SUCCESS: DONE: Force Cleanup
2019-02-10 01:57:39 -0600 :Working: DO: Cleanup
2019-02-10 01:57:40 -0600 :SUCCESS: DONE: Cleanup
2019-02-10 02:01:53 -0600 :Working: DO: Check cells have ssh equivalence for root user. Up to 10 seconds per cell ...
2019-02-10 02:01:55 -0600 :SUCCESS: DONE: Check cells have ssh equivalence for root user.
2019-02-10 02:02:00 -0600 :Working: DO: Initialize files. Up to 1 minute ...
2019-02-10 02:02:01 -0600 :Working: DO: Setup work directory
2019-02-10 02:02:02 -0600 :SUCCESS: DONE: Setup work directory
2019-02-10 02:02:04 -0600 :SUCCESS: DONE: Initialize files.
2019-02-10 02:02:04 -0600 :Working: DO: Copy, extract prerequisite check archive to cells. If required start md11 mismatched partner size correction. Up to 40 minutes ...
2019-02-10 02:02:17 -0600 :INFO : Wait correction of degraded md11 due to md partner size mismatch. Up to 30 minutes.
2019-02-10 02:02:18 -0600 :SUCCESS: DONE: Copy, extract prerequisite check archive to cells. If required start md11 mismatched partner size correction.
2019-02-10 02:02:18 -0600 :Working: DO: Check space and state of cell services. Up to 20 minutes ...
2019-02-10 02:03:40 -0600 :SUCCESS: DONE: Check space and state of cell services.
2019-02-10 02:03:40 -0600 :Working: DO: Check prerequisites on all cells. Up to 2 minutes ...
2019-02-10 02:03:49 -0600 :SUCCESS: DONE: Check prerequisites on all cells.
2019-02-10 02:03:49 -0600 :Working: DO: Execute plugin check for Patch Check Prereq ...
2019-02-10 02:03:49 -0600 :INFO : Patchmgr plugin start: Prereq check for exposure to bug 22909764 v1.0.
2019-02-10 02:03:49 -0600 :INFO : Details in logfile /u01/app/oracle/software/exa_patches/patch_18.1.12.0.0.190111/patchmgr.stdout.
2019-02-10 02:03:49 -0600 :INFO : Patchmgr plugin start: Prereq check for exposure to bug 17854520 v1.3.
2019-02-10 02:03:49 -0600 :INFO : Details in logfile /u01/app/oracle/software/exa_patches/patch_18.1.12.0.0.190111/patchmgr.stdout.
2019-02-10 02:03:49 -0600 :SUCCESS: No exposure to bug 17854520 with non-rolling patching
2019-02-10 02:03:49 -0600 :INFO : Patchmgr plugin start: Prereq check for exposure to bug 22468216 v1.0.
2019-02-10 02:03:49 -0600 :INFO : Details in logfile /u01/app/oracle/software/exa_patches/patch_18.1.12.0.0.190111/patchmgr.stdout.
2019-02-10 02:03:49 -0600 :SUCCESS: Patchmgr plugin complete: Prereq check passed for the bug 22468216
2019-02-10 02:03:49 -0600 :INFO : Patchmgr plugin start: Prereq check for exposure to bug 24625612 v1.0.
2019-02-10 02:03:49 -0600 :INFO : Details in logfile /u01/app/oracle/software/exa_patches/patch_18.1.12.0.0.190111/patchmgr.stdout.
2019-02-10 02:03:49 -0600 :SUCCESS: Patchmgr plugin complete: Prereq check passed for the bug 24625612
2019-02-10 02:03:49 -0600 :SUCCESS: No exposure to bug with non-rolling patching
2019-02-10 02:03:49 -0600 :INFO : Patchmgr plugin start: Prereq check for exposure to bug 22651315 v1.0.
2019-02-10 02:03:49 -0600 :INFO : Details in logfile /u01/app/oracle/software/exa_patches/patch_18.1.12.0.0.190111/patchmgr.stdout.
2019-02-10 02:03:51 -0600 :SUCCESS: Patchmgr plugin complete: Prereq check passed for the bug 22651315
2019-02-10 02:03:51 -0600 :SUCCESS: DONE: Execute plugin check for Patch Check Prereq.
2019-02-10 02:03:51 -0600 :Working: DO: Check ASM deactivation outcome. Up to 1 minute ...
2019-02-10 02:04:02 -0600 :SUCCESS: DONE: Check ASM deactivation outcome.
********************************************************************************
NOTE Cells will reboot during the patch or rollback process.
NOTE For non-rolling patch or rollback, ensure all ASM instances using
NOTE the cells are shut down for the duration of the patch or rollback.
NOTE For rolling patch or rollback, ensure all ASM instances using
NOTE the cells are up for the duration of the patch or rollback.
WARNING Do not interrupt the patchmgr session.
WARNING Do not alter state of ASM instances during patch or rollback.
WARNING Do not resize the screen. It may disturb the screen layout.
WARNING Do not reboot cells or alter cell services during patch or rollback.
WARNING Do not open log files in editor in write mode or try to alter them.
NOTE All time estimates are approximate.
********************************************************************************
2019-02-10 02:08:27 -0600 :Working: DO: Check cells have ssh equivalence for root user. Up to 10 seconds per cell ...
2019-02-10 02:08:28 -0600 :SUCCESS: DONE: Check cells have ssh equivalence for root user.
2019-02-10 02:08:33 -0600 :Working: DO: Initialize files. Up to 1 minute ...
2019-02-10 02:08:34 -0600 :Working: DO: Setup work directory
2019-02-10 02:09:13 -0600 :SUCCESS: DONE: Setup work directory
2019-02-10 02:09:15 -0600 :SUCCESS: DONE: Initialize files.
2019-02-10 02:09:15 -0600 :Working: DO: Copy, extract prerequisite check archive to cells. If required start md11 mismatched partner size correction. Up to 40 minutes ...
2019-02-10 02:09:28 -0600 :INFO : Wait correction of degraded md11 due to md partner size mismatch. Up to 30 minutes.
2019-02-10 02:09:30 -0600 :SUCCESS: DONE: Copy, extract prerequisite check archive to cells. If required start md11 mismatched partner size correction.
2019-02-10 02:09:30 -0600 :Working: DO: Check space and state of cell services. Up to 20 minutes ...
2019-02-10 02:10:05 -0600 :SUCCESS: DONE: Check space and state of cell services.
2019-02-10 02:10:05 -0600 :Working: DO: Check prerequisites on all cells. Up to 2 minutes ...
2019-02-10 02:10:13 -0600 :SUCCESS: DONE: Check prerequisites on all cells.
2019-02-10 02:10:13 -0600 :Working: DO: Copy the patch to all cells. Up to 3 minutes ...
2019-02-10 02:12:01 -0600 :SUCCESS: DONE: Copy the patch to all cells.
2019-02-10 02:12:03 -0600 :Working: DO: Execute plugin check for Patch Check Prereq ...
2019-02-10 02:12:03 -0600 :INFO : Patchmgr plugin start: Prereq check for exposure to bug 22909764 v1.0.
2019-02-10 02:12:03 -0600 :INFO : Details in logfile /u01/app/oracle/software/exa_patches/patch_18.1.12.0.0.190111/patchmgr.stdout.
2019-02-10 02:12:03 -0600 :INFO : Patchmgr plugin start: Prereq check for exposure to bug 17854520 v1.3.
2019-02-10 02:12:03 -0600 :INFO : Details in logfile /u01/app/oracle/software/exa_patches/patch_18.1.12.0.0.190111/patchmgr.stdout.
2019-02-10 02:12:03 -0600 :SUCCESS: No exposure to bug 17854520 with non-rolling patching
2019-02-10 02:12:03 -0600 :INFO : Patchmgr plugin start: Prereq check for exposure to bug 22468216 v1.0.
2019-02-10 02:12:03 -0600 :INFO : Details in logfile /u01/app/oracle/software/exa_patches/patch_18.1.12.0.0.190111/patchmgr.stdout.
2019-02-10 02:12:03 -0600 :SUCCESS: Patchmgr plugin complete: Prereq check passed for the bug 22468216
2019-02-10 02:12:03 -0600 :INFO : Patchmgr plugin start: Prereq check for exposure to bug 24625612 v1.0.
2019-02-10 02:12:03 -0600 :INFO : Details in logfile /u01/app/oracle/software/exa_patches/patch_18.1.12.0.0.190111/patchmgr.stdout.
2019-02-10 02:12:03 -0600 :SUCCESS: Patchmgr plugin complete: Prereq check passed for the bug 24625612
2019-02-10 02:12:03 -0600 :SUCCESS: No exposure to bug with non-rolling patching
2019-02-10 02:12:03 -0600 :INFO : Patchmgr plugin start: Prereq check for exposure to bug 22651315 v1.0.
2019-02-10 02:12:03 -0600 :INFO : Details in logfile /u01/app/oracle/software/exa_patches/patch_18.1.12.0.0.190111/patchmgr.stdout.
2019-02-10 02:12:05 -0600 :SUCCESS: Patchmgr plugin complete: Prereq check passed for the bug 22651315
2019-02-10 02:12:06 -0600 :SUCCESS: DONE: Execute plugin check for Patch Check Prereq.
2019-02-10 02:12:12 -0600 1 of 5 :Working: DO: Initiate patch on cells. Cells will remain up. Up to 5 minutes ...
2019-02-10 02:12:16 -0600 1 of 5 :SUCCESS: DONE: Initiate patch on cells.
2019-02-10 02:12:16 -0600 2 of 5 :Working: DO: Waiting to finish pre-reboot patch actions. Cells will remain up. Up to 45 minutes ...
2019-02-10 02:13:16 -0600 :INFO : Wait for patch pre-reboot procedures
2019-02-10 02:14:34 -0600 2 of 5 :SUCCESS: DONE: Waiting to finish pre-reboot patch actions.
2019-02-10 02:14:34 -0600 :Working: DO: Execute plugin check for Patching ...
2019-02-10 02:14:34 -0600 :SUCCESS: DONE: Execute plugin check for Patching.
2019-02-10 02:14:35 -0600 3 of 5 :Working: DO: Finalize patch on cells. Cells will reboot. Up to 5 minutes ...
2019-02-10 02:14:39 -0600 3 of 5 :SUCCESS: DONE: Finalize patch on cells.
2019-02-10 02:15:41 -0600 4 of 5 :Working: DO: Wait for cells to reboot and come online. Up to 120 minutes ...
2019-02-10 02:16:41 -0600 :INFO : Wait for patch finalization and reboot
2019-02-10 02:44:33 -0600 4 of 5 :SUCCESS: DONE: Wait for cells to reboot and come online.
2019-02-10 02:44:33 -0600 5 of 5 :Working: DO: Check the state of patch on cells. Up to 5 minutes ...
2019-02-10 02:44:52 -0600 5 of 5 :SUCCESS: DONE: Check the state of patch on cells.
2019-02-10 02:44:52 -0600 :Working: DO: Execute plugin check for Pre Disk Activation ...
2019-02-10 02:44:53 -0600 :SUCCESS: DONE: Execute plugin check for Pre Disk Activation.
2019-02-10 02:44:53 -0600 :Working: DO: Activate grid disks...
2019-02-10 02:44:54 -0600 :INFO : Wait for checking and activating grid disks
2019-02-10 02:45:00 -0600 :SUCCESS: DONE: Activate grid disks.
2019-02-10 02:45:03 -0600 :Working: DO: Execute plugin check for Post Patch ...
2019-02-10 02:45:03 -0600 :SUCCESS: DONE: Execute plugin check for Post Patch.
2019-02-10 02:45:04 -0600 :Working: DO: Cleanup
2019-02-10 02:45:56 -0600 :SUCCESS: DONE: Cleanup
Kernel version: 4.1.12-94.8.10.el6uek.x86_64 #2 SMP Sat Dec 22 21:26:11 PST 2018 x86_64
Cell version: OSS_18.1.12.0.0_LINUX.X64_190111
Cell rpm version: cell-18.1.12.0.0_LINUX.X64_190111-1.x86_64
Active image version: 18.1.12.0.0.190111
Active image kernel version: 4.1.12-94.8.10.el6uek
Active image activated: 2019-02-10 02:43:36 -0600
Active image status: success
Active system partition on device: /dev/md5
Active software partition on device: /dev/md7
Cell boot usb partition: /dev/sdac1
Cell boot usb version: 18.1.12.0.0.190111
Inactive image version: 12.2.1.1.6.180125.1
Inactive image activated: 2018-05-16 00:58:24 -0500
Inactive image status: success
Inactive system partition on device: /dev/md6
Inactive software partition on device: /dev/md8
Inactive marker for the rollback: /boot/I_am_hd_boot.inactive
Inactive grub config for the rollback: /boot/grub/grub.conf.inactive
Inactive usb grub config for the rollback: /boot/grub/grub.conf.usb.inactive
Inactive kernel version for the rollback: 4.1.12-94.7.8.el6uek.x86_64
Rollback to the inactive partitions: Possible
Version : 12.1.1.1.1.140712
Image activation date : 2014-11-23 00:34:06 -0800
Imaging mode : fresh
Imaging status : success
Version : 12.1.1.1.2.150411
Image activation date : 2015-05-28 21:40:16 -0500
Imaging mode : out of partition upgrade
Imaging status : success
Version : 12.1.2.3.2.160721
Image activation date : 2016-10-14 02:45:04 -0500
Imaging mode : out of partition upgrade
Imaging status : success
Version : 12.1.2.3.4.170111
Image activation date : 2017-04-04 00:25:08 -0500
Imaging mode : out of partition upgrade
Imaging status : success
Version : 12.1.2.3.6.170713
Image activation date : 2017-10-19 03:40:28 -0500
Imaging mode : out of partition upgrade
Imaging status : success
Version : 12.2.1.1.6.180125.1
Image activation date : 2018-05-16 00:58:24 -0500
Imaging mode : out of partition upgrade
Imaging status : success
Version : 18.1.12.0.0.190111
Image activation date : 2019-02-10 02:43:36 -0600
Imaging mode : out of partition upgrade
Imaging status : success
dm01cel01: Active image version: 18.1.12.0.0.190111
dm01cel02: Active image version: 18.1.12.0.0.190111
dm01cel03: Active image version: 18.1.12.0.0.190111
dm01cel04: Active image version: 18.1.12.0.0.190111
dm01cel05: Active image version: 18.1.12.0.0.190111
dm01cel06: Active image version: 18.1.12.0.0.190111
dm01cel07: Active image version: 18.1.12.0.0.190111
2019-02-10 02:58:37 -0600 :Working: DO: Cleanup
2019-02-10 02:58:39 -0600 :SUCCESS: DONE: Cleanup
CRS-4639: Could not contact Oracle High Availability Services
[root@dm01db01 ~]# dcli -g dbs_group -l root '/u01/app/11.2.0.4/grid/bin/crsctl start crs'
dm01db01: CRS-4123: Oracle High Availability Services has been started.
dm01db02: CRS-4123: Oracle High Availability Services has been started.
dm01db03: CRS-4123: Oracle High Availability Services has been started.
dm01db04: CRS-4123: Oracle High Availability Services has been started.
[root@dm01db01 ~]# dcli -g dbs_group -l root '/u01/app/11.2.0.4/grid/bin/crsctl check crs'
dm01db01: CRS-4638: Oracle High Availability Services is online
dm01db01: CRS-4537: Cluster Ready Services is online
dm01db01: CRS-4529: Cluster Synchronization Services is online
dm01db01: CRS-4533: Event Manager is online
dm01db02: CRS-4638: Oracle High Availability Services is online
dm01db02: CRS-4537: Cluster Ready Services is online
dm01db02: CRS-4529: Cluster Synchronization Services is online
dm01db02: CRS-4533: Event Manager is online
dm01db03: CRS-4638: Oracle High Availability Services is online
dm01db03: CRS-4537: Cluster Ready Services is online
dm01db03: CRS-4529: Cluster Synchronization Services is online
dm01db03: CRS-4533: Event Manager is online
dm01db04: CRS-4638: Oracle High Availability Services is online
dm01db04: CRS-4537: Cluster Ready Services is online
dm01db04: CRS-4529: Cluster Synchronization Services is online
dm01db04: CRS-4533: Event Manager is online
[root@dm01db01 ~]# /u01/app/11.2.0.4/grid/bin/crsctl stat res -t | more
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA_dm01.dg
ONLINE ONLINE dm01db01
ONLINE ONLINE dm01db02
ONLINE ONLINE dm01db03
ONLINE ONLINE dm01db04
ora.DBFS_DG.dg
ONLINE ONLINE dm01db01
ONLINE ONLINE dm01db02
ONLINE ONLINE dm01db03
ONLINE ONLINE dm01db04
ora.LISTENER.lsnr
ONLINE ONLINE dm01db01
ONLINE ONLINE dm01db02
ONLINE ONLINE dm01db03
ONLINE ONLINE dm01db04
ora.RECO_dm01.dg
ONLINE ONLINE dm01db01
ONLINE ONLINE dm01db02
ONLINE ONLINE dm01db03
ONLINE ONLINE dm01db04
ora.asm
ONLINE ONLINE dm01db01 Started
ONLINE ONLINE dm01db02 Started
ONLINE ONLINE dm01db03 Started
ONLINE ONLINE dm01db04 Started
ora.gsd
OFFLINE OFFLINE dm01db01
OFFLINE OFFLINE dm01db02
OFFLINE OFFLINE dm01db03
OFFLINE OFFLINE dm01db04
ora.net1.network
ONLINE ONLINE dm01db01
ONLINE ONLINE dm01db02
ONLINE ONLINE dm01db03
ONLINE ONLINE dm01db04
ora.ons
ONLINE ONLINE dm01db01
ONLINE ONLINE dm01db02
ONLINE ONLINE dm01db03
ONLINE ONLINE dm01db04
ora.registry.acfs
ONLINE OFFLINE dm01db01
ONLINE OFFLINE dm01db02
ONLINE OFFLINE dm01db03
ONLINE OFFLINE dm01db04
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE dm01db04
ora.LISTENER_SCAN2.lsnr
1 ONLINE ONLINE dm01db03
ora.LISTENER_SCAN3.lsnr
1 ONLINE ONLINE dm01db01
ora.cvu
1 ONLINE ONLINE dm01db02
ora.dbm01.db
1 OFFLINE OFFLINE
2 OFFLINE OFFLINE
3 OFFLINE OFFLINE
4 OFFLINE OFFLINE
ora.dm01db01.vip
1 ONLINE ONLINE dm01db01
ora.dm01db02.vip
1 ONLINE ONLINE dm01db02
ora.dm01db03.vip
1 ONLINE ONLINE dm01db03
ora.dm01db04.vip
1 ONLINE ONLINE dm01db04
ora.oc4j
1 ONLINE ONLINE dm01db02
ora.orcldb.db
1 ONLINE ONLINE dm01db01 Open
2 ONLINE ONLINE dm01db02 Open
3 ONLINE ONLINE dm01db03 Open
4 ONLINE ONLINE dm01db04 Open
ora.nsmdb.db
1 ONLINE ONLINE dm01db01 Open
2 ONLINE ONLINE dm01db02 Open
3 ONLINE ONLINE dm01db03 Open
4 ONLINE ONLINE dm01db04 Open
ora.scan1.vip
1 ONLINE ONLINE dm01db04
ora.scan2.vip
1 ONLINE ONLINE dm01db03
ora.scan3.vip
1 ONLINE ONLINE dm01db01
$ srvctl status database -d nsmdb
Conclusion
In this article we have learned how to perform upgrade Exadata Storage cells using patchmgr utility. The patchmgr utility can be used for upgrading, rollback and backup Exadata Storage cells. patchmgr utility can be used for upgrading Storage cells in a rolling or non-rolling fashion. Non-Rolling is default. Storage server patches apply operating system, firmware, and driver updates. Launch patchmgr from the compute node that is node 1 that has user equivalence setup to all the storage cells.
Launch patchmgr from the compute node that is node 1 that has user equivalence setup to all the storage cells.
In this article I will demonstrate how to perform upgrade Exadata Storage cells using patchmgr utility.
MOS Notes
Read the following MOS notes carefully.
- Exadata Database Machine and Exadata Storage Server Supported Versions (Doc ID 888828.1)
- Exadata 18.1.12.0.0 release and patch (29194095) (Doc ID 2492012.1)
- Oracle Exadata Database Machine exachk or HealthCheck (Doc ID 1070954.1)
Software Download
- Download the following patches required for Upgrading Storage cells.
- Patch 29194095 - Storage server (18.1.12.0.0.190111) and InfiniBand switch software (2.2.11-2)
Current Environment
- Exadata X4-2 Half Rack (4 Compute nodes, 7 Storage Cells and 2 IB Switches) running ESS version 12.2.1.1.6
Current Image version
- Execute the “imageinfo” command on one of the Compute nodes to identify the current Exadata Image version
Kernel version: 4.1.12-94.7.8.el6uek.x86_64 #2 SMP Thu Jan 11 20:41:01 PST 2018 x86_64
Cell version: OSS_12.2.1.1.6_LINUX.X64_180125.1
Cell rpm version: cell-12.2.1.1.6_LINUX.X64_180125.1-1.x86_64
Active image version: 12.2.1.1.6.180125.1
Active image kernel version: 4.1.12-94.7.8.el6uek
Active image activated: 2018-05-08 00:42:57 -0500
Active image status: success
Active system partition on device: /dev/md6
Active software partition on device: /dev/md8
Cell boot usb partition: /dev/sdac1
Cell boot usb version: 12.2.1.1.6.180125.1
Inactive image version: 12.1.2.3.6.170713
Inactive image activated: 2017-10-03 00:57:25 -0500
Inactive image status: success
Inactive system partition on device: /dev/md5
Inactive software partition on device: /dev/md7
Inactive marker for the rollback: /boot/I_am_hd_boot.inactive
Inactive grub config for the rollback: /boot/grub/grub.conf.inactive
Inactive kernel version for the rollback: 2.6.39-400.297.1.el6uek.x86_64
Rollback to the inactive partitions: Possible
Prerequisites
- Install and configure VNC Server on Exadata compute node 1. It is recommended to use VNC or screen utility for patching to avoid disconnections due to network issues.
- Enable blackout (OEM, crontab and so on)
- Verify disk space on storage cells
dm01cel01: Filesystem Size Used Avail Use% Mounted on
dm01cel01: /dev/md6 9.8G 4.4G 4.9G 48% /
dm01cel02: Filesystem Size Used Avail Use% Mounted on
dm01cel02: /dev/md6 9.8G 4.5G 4.8G 49% /
dm01cel03: Filesystem Size Used Avail Use% Mounted on
dm01cel03: /dev/md6 9.8G 4.5G 4.8G 49% /
dm01cel04: Filesystem Size Used Avail Use% Mounted on
dm01cel04: /dev/md6 9.8G 4.5G 4.8G 49% /
dm01cel05: Filesystem Size Used Avail Use% Mounted on
dm01cel05: /dev/md6 9.8G 4.5G 4.8G 49% /
dm01cel06: Filesystem Size Used Avail Use% Mounted on
dm01cel06: /dev/md6 9.8G 4.6G 4.7G 50% /
dm01cel07: Filesystem Size Used Avail Use% Mounted on
dm01cel07: /dev/md6 9.8G 4.5G 4.8G 48% /
- Run Exachk before starting the actual patching. Correct any Critical issues and Failure that can conflict with patching.
- Verify hardware failure. Make sure there are no hardware failures before patching
[root@dm01db01 ~]# dcli -l root -g ~/cell_group "cellcli -e list physicaldisk where diskType=FlashDisk and status not = normal"
[root@dm01db01 ~]# dcli -g ~/dbs_group -l root 'dbmcli -e list physicaldisk where status!=normal'
[root@dm01db01 ~]# dcli -g ~/dbs_group -l root 'ipmitool sunoem cli "show -d properties -level all /SYS fault_state==Faulted"'
[root@dm01db01 ~]# dcli -g ~/cell_group -l root 'ipmitool sunoem cli "show -d properties -level all /SYS fault_state==Faulted"'
- Clear or acknowledge alerts on db and cell nodes
[root@dm01db01 ~]# dcli -l root -g ~/dbs_group "dbmcli -e drop alerthistory all"
- Download patches and copy them to the compute node 1 under staging directory
- Copy the patches to compute node 1 under staging aread and unzip the patches
[root@dm01db01 ~]# unzip p29194095_181000_Linux-x86-64.zip
- Read the readme file and document the steps for storage cell patching.
Steps to perform Storage Cell Patching
- Open VNC Session and login as root user
- Login as root user
uid=0(root) gid=0(root) groups=0(root),1(bin),2(daemon),3(sys),4(adm),6(disk),10(wheel)
- Check SSH user equivalence
dm01cel01: 01:46:18 up 194 days, 40 min, 0 users, load average: 0.17, 0.50, 0.61
dm01cel02: 01:46:18 up 194 days, 40 min, 0 users, load average: 0.05, 0.29, 0.45
dm01cel03: 01:46:18 up 194 days, 40 min, 0 users, load average: 0.25, 0.64, 0.63
dm01cel04: 01:46:18 up 194 days, 40 min, 0 users, load average: 0.12, 0.44, 0.53
dm01cel05: 01:46:18 up 194 days, 40 min, 0 users, load average: 0.15, 0.55, 0.65
dm01cel06: 01:46:18 up 194 days, 40 min, 0 users, load average: 0.33, 0.48, 0.55
dm01cel07: 01:46:18 up 194 days, 40 min, 0 users, load average: 0.09, 0.37, 0.52
- Adjust the disk_repair_time for Oracle ASM.
SQL> select dg.name,a.value from v$asm_diskgroup dg, v$asm_attribute a where dg.group_number=a.group_number and a.name='disk_repair_time';
NAME VALUE
------------------------------ --------------------------
DATA_DM01 3.6H
RECO_DM01 3.6h
- Shut down and stop the Oracle components on each database server using the following commands:
[root@dm01db01 ~]# dcli -g dbs_group -l root '/u01/app/11.2.0.4/grid/bin/crsctl stop crs'
- Get the current Cell Exadata Storage software version
Kernel version: 4.1.12-94.7.8.el6uek.x86_64 #2 SMP Thu Jan 11 20:41:01 PST 2018 x86_64
Cell version: OSS_12.2.1.1.6_LINUX.X64_180125.1
Cell rpm version: cell-12.2.1.1.6_LINUX.X64_180125.1-1.x86_64
Active image version: 12.2.1.1.6.180125.1
Active image kernel version: 4.1.12-94.7.8.el6uek
Active image activated: 2018-05-08 00:42:57 -0500
Active image status: success
Active system partition on device: /dev/md6
Active software partition on device: /dev/md8
Cell boot usb partition: /dev/sdac1
Cell boot usb version: 12.2.1.1.6.180125.1
Inactive image version: 12.1.2.3.6.170713
Inactive image activated: 2017-10-03 00:57:25 -0500
Inactive image status: success
Inactive system partition on device: /dev/md5
Inactive software partition on device: /dev/md7
Inactive marker for the rollback: /boot/I_am_hd_boot.inactive
Inactive grub config for the rollback: /boot/grub/grub.conf.inactive
Inactive kernel version for the rollback: 2.6.39-400.297.1.el6uek.x86_64
Rollback to the inactive partitions: Possible
- Shut down all cell services on all cells to be updated. Use dcli command to do all cells at the same time:
dm01cel01:
dm01cel01: Stopping the RS, CELLSRV, and MS services...
dm01cel01: The SHUTDOWN of services was successful.
dm01cel02:
dm01cel02: Stopping the RS, CELLSRV, and MS services...
dm01cel02: The SHUTDOWN of services was successful.
dm01cel03:
dm01cel03: Stopping the RS, CELLSRV, and MS services...
dm01cel03: The SHUTDOWN of services was successful.
dm01cel04:
dm01cel04: Stopping the RS, CELLSRV, and MS services...
dm01cel04: The SHUTDOWN of services was successful.
dm01cel05:
dm01cel05: Stopping the RS, CELLSRV, and MS services...
dm01cel05: The SHUTDOWN of services was successful.
dm01cel06:
dm01cel06: Stopping the RS, CELLSRV, and MS services...
dm01cel06: The SHUTDOWN of services was successful.
dm01cel07:
dm01cel07: Stopping the RS, CELLSRV, and MS services...
dm01cel07: The SHUTDOWN of services was successful.
- Reset the patchmgr state to a known state using the following command:
2019-02-10 01:56:19 -0600 :Working: DO: Force Cleanup
2019-02-10 01:56:21 -0600 :SUCCESS: DONE: Force Cleanup
- Clean up any previous patchmgr utility runs using the following command:
2019-02-10 01:57:39 -0600 :Working: DO: Cleanup
2019-02-10 01:57:40 -0600 :SUCCESS: DONE: Cleanup
- Verify that the cells meet prerequisite checks using the following command.
2019-02-10 02:01:53 -0600 :Working: DO: Check cells have ssh equivalence for root user. Up to 10 seconds per cell ...
2019-02-10 02:01:55 -0600 :SUCCESS: DONE: Check cells have ssh equivalence for root user.
2019-02-10 02:02:00 -0600 :Working: DO: Initialize files. Up to 1 minute ...
2019-02-10 02:02:01 -0600 :Working: DO: Setup work directory
2019-02-10 02:02:02 -0600 :SUCCESS: DONE: Setup work directory
2019-02-10 02:02:04 -0600 :SUCCESS: DONE: Initialize files.
2019-02-10 02:02:04 -0600 :Working: DO: Copy, extract prerequisite check archive to cells. If required start md11 mismatched partner size correction. Up to 40 minutes ...
2019-02-10 02:02:17 -0600 :INFO : Wait correction of degraded md11 due to md partner size mismatch. Up to 30 minutes.
2019-02-10 02:02:18 -0600 :SUCCESS: DONE: Copy, extract prerequisite check archive to cells. If required start md11 mismatched partner size correction.
2019-02-10 02:02:18 -0600 :Working: DO: Check space and state of cell services. Up to 20 minutes ...
2019-02-10 02:03:40 -0600 :SUCCESS: DONE: Check space and state of cell services.
2019-02-10 02:03:40 -0600 :Working: DO: Check prerequisites on all cells. Up to 2 minutes ...
2019-02-10 02:03:49 -0600 :SUCCESS: DONE: Check prerequisites on all cells.
2019-02-10 02:03:49 -0600 :Working: DO: Execute plugin check for Patch Check Prereq ...
2019-02-10 02:03:49 -0600 :INFO : Patchmgr plugin start: Prereq check for exposure to bug 22909764 v1.0.
2019-02-10 02:03:49 -0600 :INFO : Details in logfile /u01/app/oracle/software/exa_patches/patch_18.1.12.0.0.190111/patchmgr.stdout.
2019-02-10 02:03:49 -0600 :INFO : Patchmgr plugin start: Prereq check for exposure to bug 17854520 v1.3.
2019-02-10 02:03:49 -0600 :INFO : Details in logfile /u01/app/oracle/software/exa_patches/patch_18.1.12.0.0.190111/patchmgr.stdout.
2019-02-10 02:03:49 -0600 :SUCCESS: No exposure to bug 17854520 with non-rolling patching
2019-02-10 02:03:49 -0600 :INFO : Patchmgr plugin start: Prereq check for exposure to bug 22468216 v1.0.
2019-02-10 02:03:49 -0600 :INFO : Details in logfile /u01/app/oracle/software/exa_patches/patch_18.1.12.0.0.190111/patchmgr.stdout.
2019-02-10 02:03:49 -0600 :SUCCESS: Patchmgr plugin complete: Prereq check passed for the bug 22468216
2019-02-10 02:03:49 -0600 :INFO : Patchmgr plugin start: Prereq check for exposure to bug 24625612 v1.0.
2019-02-10 02:03:49 -0600 :INFO : Details in logfile /u01/app/oracle/software/exa_patches/patch_18.1.12.0.0.190111/patchmgr.stdout.
2019-02-10 02:03:49 -0600 :SUCCESS: Patchmgr plugin complete: Prereq check passed for the bug 24625612
2019-02-10 02:03:49 -0600 :SUCCESS: No exposure to bug with non-rolling patching
2019-02-10 02:03:49 -0600 :INFO : Patchmgr plugin start: Prereq check for exposure to bug 22651315 v1.0.
2019-02-10 02:03:49 -0600 :INFO : Details in logfile /u01/app/oracle/software/exa_patches/patch_18.1.12.0.0.190111/patchmgr.stdout.
2019-02-10 02:03:51 -0600 :SUCCESS: Patchmgr plugin complete: Prereq check passed for the bug 22651315
2019-02-10 02:03:51 -0600 :SUCCESS: DONE: Execute plugin check for Patch Check Prereq.
2019-02-10 02:03:51 -0600 :Working: DO: Check ASM deactivation outcome. Up to 1 minute ...
2019-02-10 02:04:02 -0600 :SUCCESS: DONE: Check ASM deactivation outcome.
- If the prerequisite checks pass, then start the update process.
********************************************************************************
NOTE Cells will reboot during the patch or rollback process.
NOTE For non-rolling patch or rollback, ensure all ASM instances using
NOTE the cells are shut down for the duration of the patch or rollback.
NOTE For rolling patch or rollback, ensure all ASM instances using
NOTE the cells are up for the duration of the patch or rollback.
WARNING Do not interrupt the patchmgr session.
WARNING Do not alter state of ASM instances during patch or rollback.
WARNING Do not resize the screen. It may disturb the screen layout.
WARNING Do not reboot cells or alter cell services during patch or rollback.
WARNING Do not open log files in editor in write mode or try to alter them.
NOTE All time estimates are approximate.
********************************************************************************
2019-02-10 02:08:27 -0600 :Working: DO: Check cells have ssh equivalence for root user. Up to 10 seconds per cell ...
2019-02-10 02:08:28 -0600 :SUCCESS: DONE: Check cells have ssh equivalence for root user.
2019-02-10 02:08:33 -0600 :Working: DO: Initialize files. Up to 1 minute ...
2019-02-10 02:08:34 -0600 :Working: DO: Setup work directory
2019-02-10 02:09:13 -0600 :SUCCESS: DONE: Setup work directory
2019-02-10 02:09:15 -0600 :SUCCESS: DONE: Initialize files.
2019-02-10 02:09:15 -0600 :Working: DO: Copy, extract prerequisite check archive to cells. If required start md11 mismatched partner size correction. Up to 40 minutes ...
2019-02-10 02:09:28 -0600 :INFO : Wait correction of degraded md11 due to md partner size mismatch. Up to 30 minutes.
2019-02-10 02:09:30 -0600 :SUCCESS: DONE: Copy, extract prerequisite check archive to cells. If required start md11 mismatched partner size correction.
2019-02-10 02:09:30 -0600 :Working: DO: Check space and state of cell services. Up to 20 minutes ...
2019-02-10 02:10:05 -0600 :SUCCESS: DONE: Check space and state of cell services.
2019-02-10 02:10:05 -0600 :Working: DO: Check prerequisites on all cells. Up to 2 minutes ...
2019-02-10 02:10:13 -0600 :SUCCESS: DONE: Check prerequisites on all cells.
2019-02-10 02:10:13 -0600 :Working: DO: Copy the patch to all cells. Up to 3 minutes ...
2019-02-10 02:12:01 -0600 :SUCCESS: DONE: Copy the patch to all cells.
2019-02-10 02:12:03 -0600 :Working: DO: Execute plugin check for Patch Check Prereq ...
2019-02-10 02:12:03 -0600 :INFO : Patchmgr plugin start: Prereq check for exposure to bug 22909764 v1.0.
2019-02-10 02:12:03 -0600 :INFO : Details in logfile /u01/app/oracle/software/exa_patches/patch_18.1.12.0.0.190111/patchmgr.stdout.
2019-02-10 02:12:03 -0600 :INFO : Patchmgr plugin start: Prereq check for exposure to bug 17854520 v1.3.
2019-02-10 02:12:03 -0600 :INFO : Details in logfile /u01/app/oracle/software/exa_patches/patch_18.1.12.0.0.190111/patchmgr.stdout.
2019-02-10 02:12:03 -0600 :SUCCESS: No exposure to bug 17854520 with non-rolling patching
2019-02-10 02:12:03 -0600 :INFO : Patchmgr plugin start: Prereq check for exposure to bug 22468216 v1.0.
2019-02-10 02:12:03 -0600 :INFO : Details in logfile /u01/app/oracle/software/exa_patches/patch_18.1.12.0.0.190111/patchmgr.stdout.
2019-02-10 02:12:03 -0600 :SUCCESS: Patchmgr plugin complete: Prereq check passed for the bug 22468216
2019-02-10 02:12:03 -0600 :INFO : Patchmgr plugin start: Prereq check for exposure to bug 24625612 v1.0.
2019-02-10 02:12:03 -0600 :INFO : Details in logfile /u01/app/oracle/software/exa_patches/patch_18.1.12.0.0.190111/patchmgr.stdout.
2019-02-10 02:12:03 -0600 :SUCCESS: Patchmgr plugin complete: Prereq check passed for the bug 24625612
2019-02-10 02:12:03 -0600 :SUCCESS: No exposure to bug with non-rolling patching
2019-02-10 02:12:03 -0600 :INFO : Patchmgr plugin start: Prereq check for exposure to bug 22651315 v1.0.
2019-02-10 02:12:03 -0600 :INFO : Details in logfile /u01/app/oracle/software/exa_patches/patch_18.1.12.0.0.190111/patchmgr.stdout.
2019-02-10 02:12:05 -0600 :SUCCESS: Patchmgr plugin complete: Prereq check passed for the bug 22651315
2019-02-10 02:12:06 -0600 :SUCCESS: DONE: Execute plugin check for Patch Check Prereq.
2019-02-10 02:12:12 -0600 1 of 5 :Working: DO: Initiate patch on cells. Cells will remain up. Up to 5 minutes ...
2019-02-10 02:12:16 -0600 1 of 5 :SUCCESS: DONE: Initiate patch on cells.
2019-02-10 02:12:16 -0600 2 of 5 :Working: DO: Waiting to finish pre-reboot patch actions. Cells will remain up. Up to 45 minutes ...
2019-02-10 02:13:16 -0600 :INFO : Wait for patch pre-reboot procedures
2019-02-10 02:14:34 -0600 2 of 5 :SUCCESS: DONE: Waiting to finish pre-reboot patch actions.
2019-02-10 02:14:34 -0600 :Working: DO: Execute plugin check for Patching ...
2019-02-10 02:14:34 -0600 :SUCCESS: DONE: Execute plugin check for Patching.
2019-02-10 02:14:35 -0600 3 of 5 :Working: DO: Finalize patch on cells. Cells will reboot. Up to 5 minutes ...
2019-02-10 02:14:39 -0600 3 of 5 :SUCCESS: DONE: Finalize patch on cells.
2019-02-10 02:15:41 -0600 4 of 5 :Working: DO: Wait for cells to reboot and come online. Up to 120 minutes ...
2019-02-10 02:16:41 -0600 :INFO : Wait for patch finalization and reboot
2019-02-10 02:44:33 -0600 4 of 5 :SUCCESS: DONE: Wait for cells to reboot and come online.
2019-02-10 02:44:33 -0600 5 of 5 :Working: DO: Check the state of patch on cells. Up to 5 minutes ...
2019-02-10 02:44:52 -0600 5 of 5 :SUCCESS: DONE: Check the state of patch on cells.
2019-02-10 02:44:52 -0600 :Working: DO: Execute plugin check for Pre Disk Activation ...
2019-02-10 02:44:53 -0600 :SUCCESS: DONE: Execute plugin check for Pre Disk Activation.
2019-02-10 02:44:53 -0600 :Working: DO: Activate grid disks...
2019-02-10 02:44:54 -0600 :INFO : Wait for checking and activating grid disks
2019-02-10 02:45:00 -0600 :SUCCESS: DONE: Activate grid disks.
2019-02-10 02:45:03 -0600 :Working: DO: Execute plugin check for Post Patch ...
2019-02-10 02:45:03 -0600 :SUCCESS: DONE: Execute plugin check for Post Patch.
2019-02-10 02:45:04 -0600 :Working: DO: Cleanup
2019-02-10 02:45:56 -0600 :SUCCESS: DONE: Cleanup
- Monitor the log files and cells being updated when e-mail alerts are not setup. open a new session and do a tail on the log file as shown below
- Verify the update status after the patchmgr utility completes as follows:
Kernel version: 4.1.12-94.8.10.el6uek.x86_64 #2 SMP Sat Dec 22 21:26:11 PST 2018 x86_64
Cell version: OSS_18.1.12.0.0_LINUX.X64_190111
Cell rpm version: cell-18.1.12.0.0_LINUX.X64_190111-1.x86_64
Active image version: 18.1.12.0.0.190111
Active image kernel version: 4.1.12-94.8.10.el6uek
Active image activated: 2019-02-10 02:43:36 -0600
Active image status: success
Active system partition on device: /dev/md5
Active software partition on device: /dev/md7
Cell boot usb partition: /dev/sdac1
Cell boot usb version: 18.1.12.0.0.190111
Inactive image version: 12.2.1.1.6.180125.1
Inactive image activated: 2018-05-16 00:58:24 -0500
Inactive image status: success
Inactive system partition on device: /dev/md6
Inactive software partition on device: /dev/md8
Inactive marker for the rollback: /boot/I_am_hd_boot.inactive
Inactive grub config for the rollback: /boot/grub/grub.conf.inactive
Inactive usb grub config for the rollback: /boot/grub/grub.conf.usb.inactive
Inactive kernel version for the rollback: 4.1.12-94.7.8.el6uek.x86_64
Rollback to the inactive partitions: Possible
- Check the imagehistory
Version : 12.1.1.1.1.140712
Image activation date : 2014-11-23 00:34:06 -0800
Imaging mode : fresh
Imaging status : success
Version : 12.1.1.1.2.150411
Image activation date : 2015-05-28 21:40:16 -0500
Imaging mode : out of partition upgrade
Imaging status : success
Version : 12.1.2.3.2.160721
Image activation date : 2016-10-14 02:45:04 -0500
Imaging mode : out of partition upgrade
Imaging status : success
Version : 12.1.2.3.4.170111
Image activation date : 2017-04-04 00:25:08 -0500
Imaging mode : out of partition upgrade
Imaging status : success
Version : 12.1.2.3.6.170713
Image activation date : 2017-10-19 03:40:28 -0500
Imaging mode : out of partition upgrade
Imaging status : success
Version : 12.2.1.1.6.180125.1
Image activation date : 2018-05-16 00:58:24 -0500
Imaging mode : out of partition upgrade
Imaging status : success
Version : 18.1.12.0.0.190111
Image activation date : 2019-02-10 02:43:36 -0600
Imaging mode : out of partition upgrade
Imaging status : success
- Verify the image on all cells
dm01cel01: Active image version: 18.1.12.0.0.190111
dm01cel02: Active image version: 18.1.12.0.0.190111
dm01cel03: Active image version: 18.1.12.0.0.190111
dm01cel04: Active image version: 18.1.12.0.0.190111
dm01cel05: Active image version: 18.1.12.0.0.190111
dm01cel06: Active image version: 18.1.12.0.0.190111
dm01cel07: Active image version: 18.1.12.0.0.190111
- Clean up the cells using the -cleanup option to clean up all the temporary update or rollback files on the cells.
2019-02-10 02:58:37 -0600 :Working: DO: Cleanup
2019-02-10 02:58:39 -0600 :SUCCESS: DONE: Cleanup
- Start Clusterware and databases
CRS-4639: Could not contact Oracle High Availability Services
[root@dm01db01 ~]# dcli -g dbs_group -l root '/u01/app/11.2.0.4/grid/bin/crsctl start crs'
dm01db01: CRS-4123: Oracle High Availability Services has been started.
dm01db02: CRS-4123: Oracle High Availability Services has been started.
dm01db03: CRS-4123: Oracle High Availability Services has been started.
dm01db04: CRS-4123: Oracle High Availability Services has been started.
[root@dm01db01 ~]# dcli -g dbs_group -l root '/u01/app/11.2.0.4/grid/bin/crsctl check crs'
dm01db01: CRS-4638: Oracle High Availability Services is online
dm01db01: CRS-4537: Cluster Ready Services is online
dm01db01: CRS-4529: Cluster Synchronization Services is online
dm01db01: CRS-4533: Event Manager is online
dm01db02: CRS-4638: Oracle High Availability Services is online
dm01db02: CRS-4537: Cluster Ready Services is online
dm01db02: CRS-4529: Cluster Synchronization Services is online
dm01db02: CRS-4533: Event Manager is online
dm01db03: CRS-4638: Oracle High Availability Services is online
dm01db03: CRS-4537: Cluster Ready Services is online
dm01db03: CRS-4529: Cluster Synchronization Services is online
dm01db03: CRS-4533: Event Manager is online
dm01db04: CRS-4638: Oracle High Availability Services is online
dm01db04: CRS-4537: Cluster Ready Services is online
dm01db04: CRS-4529: Cluster Synchronization Services is online
dm01db04: CRS-4533: Event Manager is online
[root@dm01db01 ~]# /u01/app/11.2.0.4/grid/bin/crsctl stat res -t | more
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA_dm01.dg
ONLINE ONLINE dm01db01
ONLINE ONLINE dm01db02
ONLINE ONLINE dm01db03
ONLINE ONLINE dm01db04
ora.DBFS_DG.dg
ONLINE ONLINE dm01db01
ONLINE ONLINE dm01db02
ONLINE ONLINE dm01db03
ONLINE ONLINE dm01db04
ora.LISTENER.lsnr
ONLINE ONLINE dm01db01
ONLINE ONLINE dm01db02
ONLINE ONLINE dm01db03
ONLINE ONLINE dm01db04
ora.RECO_dm01.dg
ONLINE ONLINE dm01db01
ONLINE ONLINE dm01db02
ONLINE ONLINE dm01db03
ONLINE ONLINE dm01db04
ora.asm
ONLINE ONLINE dm01db01 Started
ONLINE ONLINE dm01db02 Started
ONLINE ONLINE dm01db03 Started
ONLINE ONLINE dm01db04 Started
ora.gsd
OFFLINE OFFLINE dm01db01
OFFLINE OFFLINE dm01db02
OFFLINE OFFLINE dm01db03
OFFLINE OFFLINE dm01db04
ora.net1.network
ONLINE ONLINE dm01db01
ONLINE ONLINE dm01db02
ONLINE ONLINE dm01db03
ONLINE ONLINE dm01db04
ora.ons
ONLINE ONLINE dm01db01
ONLINE ONLINE dm01db02
ONLINE ONLINE dm01db03
ONLINE ONLINE dm01db04
ora.registry.acfs
ONLINE OFFLINE dm01db01
ONLINE OFFLINE dm01db02
ONLINE OFFLINE dm01db03
ONLINE OFFLINE dm01db04
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE dm01db04
ora.LISTENER_SCAN2.lsnr
1 ONLINE ONLINE dm01db03
ora.LISTENER_SCAN3.lsnr
1 ONLINE ONLINE dm01db01
ora.cvu
1 ONLINE ONLINE dm01db02
ora.dbm01.db
1 OFFLINE OFFLINE
2 OFFLINE OFFLINE
3 OFFLINE OFFLINE
4 OFFLINE OFFLINE
ora.dm01db01.vip
1 ONLINE ONLINE dm01db01
ora.dm01db02.vip
1 ONLINE ONLINE dm01db02
ora.dm01db03.vip
1 ONLINE ONLINE dm01db03
ora.dm01db04.vip
1 ONLINE ONLINE dm01db04
ora.oc4j
1 ONLINE ONLINE dm01db02
ora.orcldb.db
1 ONLINE ONLINE dm01db01 Open
2 ONLINE ONLINE dm01db02 Open
3 ONLINE ONLINE dm01db03 Open
4 ONLINE ONLINE dm01db04 Open
ora.nsmdb.db
1 ONLINE ONLINE dm01db01 Open
2 ONLINE ONLINE dm01db02 Open
3 ONLINE ONLINE dm01db03 Open
4 ONLINE ONLINE dm01db04 Open
ora.scan1.vip
1 ONLINE ONLINE dm01db04
ora.scan2.vip
1 ONLINE ONLINE dm01db03
ora.scan3.vip
1 ONLINE ONLINE dm01db01
- Verify the databases and start them if needed
$ srvctl status database -d nsmdb
Conclusion
In this article we have learned how to perform upgrade Exadata Storage cells using patchmgr utility. The patchmgr utility can be used for upgrading, rollback and backup Exadata Storage cells. patchmgr utility can be used for upgrading Storage cells in a rolling or non-rolling fashion. Non-Rolling is default. Storage server patches apply operating system, firmware, and driver updates. Launch patchmgr from the compute node that is node 1 that has user equivalence setup to all the storage cells.
No comments:
Post a Comment