Exadata Grid Infrastructure One-off Patch Application

There are times when you encounter issues related to Database/ASM/Clusterware and Oracle recommends you to apply one-patches to fix the issue. These one-off patches can be applicable to Grid and Oracle Home or in some cases just specific to a particular home. In my cases we were adding Exadata X7-2 Compute nodes and Storage Cells to an Existing Exadata X6-2 Rack, and the prerequisite to do that was to apply the one-off (27965497) to GI home before upgrading the Exadata Cluster.

In this article we will demonstrate how to apply a one-off patch to Grid Infrastructure home on Exadata Database Machine. Here my GI version is 11.2.0.4.180116

Steps to apply the one-off patch to GI home

Note: Read the readme.txt or readme.html file carefully as the steps may change for your environment.

• Get the Database Status and role

dm01db01-orcldb1 {/home/oracle}: sqlplus / as sysdba

SQL> select name, open_mode,database_role from gv$database;

NAME      OPEN_MODE            DATABASE_ROLE
--------- -------------------- ----------------
ORCLDB   MOUNTED              PHYSICAL STANDBY
ORCLDB   MOUNTED              PHYSICAL STANDBY
ORCLDB   MOUNTED              PHYSICAL STANDBY

• Set Grid Infrastructure environmental variable

dm01db01-orcldb1 {/home/oracle}: export ORACLE_SID=+ASM1

dm01db01-orcldb1 {/home/oracle}: export ORACLE_HOME=/u01/app/11.2.0.4/grid

dm01db01-+ASM1 {/home/oracle}:echo $ORACLE_HOME
/u01/app/11.2.0.4/grid

dm01db01-+ASM1 {/home/oracle}: export PATH=$PATH:/u01/app/11.2.0.4/grid/OPatch

• List the current GI patches

dm01db01-+ASM1 {/home/oracle}:opatch lspatches
26925255;DATABASE PATCH FOR EXADATA (Jan 2018 - 11.2.0.4.180116) : (26925255)
26609929;OCW Patch Set Update : 11.2.0.4.170814 (26609929)
23727132;
22502505;ACFS Patch Set Update : 11.2.0.4.160419 (22502505)

OPatch succeeded.

• Download, copy and unzip the one-off patch to a staging directory on Exadata Compute node 1.

[root@dm01db01 GI]# ls -ltr
total 442668
-rw-rw-r-- 1 oracle oinstall       267 May  8 02:21 bundle.xml
-rw-rw-r-- 1 oracle oinstall     91939 May 14 15:15 README.html
-rw-rw-r-- 1 oracle oinstall     50971 May 14 15:15 README.txt
-rw-r--r-- 1 oracle oinstall 453130872 May 15 12:02 p27965497_11204160419forACFS_Linux-x86-64.zip
drwxr-xr-x 5 oracle oinstall      4096 May 17 09:40 27965497

• Login as root user and Navigate to one-off patch directory

[root@dm01db01 ~]# cd /u01/GI/27965497

• Set the PATH to include opatch utility from GI home

 [root@dm01db01 27965497]# export PATH=$PATH:/u01/app/11.2.0.4/grid/OPatch

• Verify opatch utility location

[root@dm01db01 27965497]# which opatch
/u01/app/11.2.0.4/grid/OPatch/opatch

• Verify opatch version. Here the minimum verion required is 11.2.0.3.6

[root@dm01db01 27965497]# opatch version
OPatch Version: 11.2.0.3.18

OPatch succeeded.

• Make sure that the ASM and Database is up and running

[root@dm01db01 27965497]# ps -ef|grep smon
root     277568 254212  0 09:39 pts/1    00:00:00 grep smon
root     322798      1  3 May02 ?        13:08:10 /u01/app/11.2.0.4/grid/bin/osysmond.bin
oracle   326697      1  0 May02 ?        00:00:23 asm_smon_+ASM1
oracle   328622      1  0 May02 ?        00:00:15 ora_smon_orcldb1

• Create the OCM file as shown below

[root@dm01db01 27965497]# /u01/app/11.2.0.4/grid/OPatch/ocm/bin/emocmrsp
OCM Installation Response Generator 10.3.7.0.0 - Production
Copyright (c) 2005, 2012, Oracle and/or its affiliates.  All rights reserved.

Provide your email address to be informed of security issues, install and
initiate Oracle Configuration Manager. Easier for you if you use your My
Oracle Support Email address/User Name.
Visit http://www.oracle.com/support/policies.html for details.
Email address/User Name:

You have not provided an email address for notification of security issues.
Do you wish to remain uninformed of security issues ([Y]es, [N]o) [N]:  Y
The OCM configuration response file (ocm.rsp) was successfully created.
[root@dm01db01 27965497]#

[root@dm01db01 27965497]# ls -ltr
total 16
drwxr-xr-x 4 oracle oinstall 4096 May  8 02:18 etc
drwxr-xr-x 3 oracle oinstall 4096 May  8 02:18 custom
drwxr-xr-x 6 oracle oinstall 4096 May  8 02:18 files
-rw-r--r-- 1 root   root      621 May 17 09:40 ocm.rsp

• Apply the one-off patch to GI home as show below

[root@dm01db01 cfgtoollogs]# /u01/app/11.2.0.4/grid/OPatch/opatch auto /u01/patches/GI -oh /u01/app/11.2.0.4/grid -ocmrf /u01/patches/GI/27965497/ocm.rsp
Executing /u01/app/11.2.0.4/grid/perl/bin/perl /u01/app/11.2.0.4/grid/OPatch/crs/patch11203.pl -patchdir /u01/patches -patchn GI -oh /u01/app/11.2.0.4/grid -ocmrf /u01/patches/GI/27965497/ocm.rsp -paramfile /u01/app/11.2.0.4/grid/crs/install/crsconfig_params

This is the main log file: /u01/app/11.2.0.4/grid/cfgtoollogs/opatchauto2018-05-17_10-28-13.log

This file will show your detected configuration and all the steps that opatchauto attempted to do on your system:
/u01/app/11.2.0.4/grid/cfgtoollogs/opatchauto2018-05-17_10-28-13.report.log

2018-05-17 10:28:13: Starting Clusterware Patch Setup
Using configuration parameter file: /u01/app/11.2.0.4/grid/crs/install/crsconfig_params

Stopping CRS...
Stopped CRS successfully

patch /u01/patches/GI/27965497  apply successful for home  /u01/app/11.2.0.4/grid

Starting CRS...
Installing Trace File Analyzer
CRS-4123: Oracle High Availability Services has been started.

opatch auto succeeded.

• Login as GI software owner and verify that the one-off patch is applied successfully

[root@dm01db01 cfgtoollogs]# su - oracle

dm01db01-+ASM1 {/home/oracle}:export PATH=$PATH:/u01/app/11.2.0.4/grid/OPatch

dm01db01-+ASM1 {/home/oracle}:opatch lspatches
27965497;ACFS Interim patch for 27965497
26925255;DATABASE PATCH FOR EXADATA (Jan 2018 - 11.2.0.4.180116) : (26925255)
26609929;OCW Patch Set Update : 11.2.0.4.170814 (26609929)
23727132;

OPatch succeeded.

• Verify GI and Database status

dm01db01-+ASM1 {/home/oracle}:ps -ef|grep pmon
oracle   171755      1  0 10:40 ?        00:00:00 asm_pmon_+ASM1
oracle   173484      1  0 10:41 ?        00:00:00 ora_pmon_orcldb1
oracle   178926 176700  0 10:43 pts/0    00:00:00 grep pmon

dm01db01-orcldb1 {/home/oracle}:/u01/app/11.2.0.4/grid/bin/crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
dm01db01-orcldb1 {/home/oracle}:

SQL> select name, open_mode,database_role from gv$database;

NAME      OPEN_MODE            DATABASE_ROLE
--------- -------------------- ----------------
ORCLDB   MOUNTED              PHYSICAL STANDBY
ORCLDB   MOUNTED              PHYSICAL STANDBY
ORCLDB   MOUNTED              PHYSICAL STANDBY

***Repeat the above steps on all other Compute nodes in the Cluster***

Start the Media Recovery process if it is not started automatically.

SQL> alter database recover managed standby database using currently logfile disconnect;

Conclusion

In this article we have learned how to apply a one-off patch to Grid Infrastructure home on Exadata Database Machine.