Convert Standard ASM to Flex ASM on Exadata

About Flex ASM

• In earlier Oracle database release, ASM will be running on each node in the cluster and ASM clients such as database and ACFS can only access ASM by using an ASM on the same host.
• Oracle Database 12c introduced a new feature called Oracle Flex ASM which provides remote storage access.
• Oracle Flex ASM is an ASM instance which will be running in the cluster based on cardinality defined.
• The default cardinality is 3, but this can be modified using srvctl modify asm command.
• ASM is not required to run on each node in the cluster.
• With Oracle Flex ASM,the clients can connect to remote ASM using network connection (ie ASM network ).
• If a server running an ASM instance fails, Oracle Clusterware will start a new ASM instance on a different server to maintain the cardinality. If a 12c database instance is using a particular ASM instance, and that instance is lost because of a server crash or ASM instance failure, then the Oracle 12c database instance will reconnect to an existing ASM instance on another node.These features are collectively called Oracle Flex ASM.

Steps to Convert Standard ASM to Flex ASM

• You can convert an Oracle ASM configuration to an Oracle Flex ASM using ASMCA. This functionality is only available in an Oracle Grid Infrastructure 12c configuration.

Before you convert an Oracle ASM configuration to an Oracle Flex ASM, you must ensure the following:

• Oracle Cluster Registry (OCR) is stored in a disk group

[root@dm01db01 ~]# /u01/app/12.1.0.2/grid/bin/ocrcheck
Status of Oracle Cluster Registry is as follows :
         Version                  :          4
         Total space (kbytes)     :     409568
         Used space (kbytes)      :      10684
         Available space (kbytes) :     398884
         ID                       : 1119121028
         Device/File Name         :      +DATA
                                    Device/File integrity check succeeded

                                    Device/File not configured

                                    Device/File not configured

                                    Device/File not configured

                                    Device/File not configured

         Cluster registry integrity check succeeded

         Logical corruption check succeeded

• The server parameter file (SPFILE) is stored in a disk group.

SQL> show parameter spfile

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
spfile                               string      +DATA/dm01-cluster/ASMPARAMETE
                                                 RFILE/registry.253.934096549

SQL> show parameter spfile

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
spfile                               string      +DATA/orcl/spfileorcl.ora

• The password file (ORAPWD file) is stored in a disk group.

dm01db01-orcldb1 {/home/oracle}:dcli -g ~/dbs_group -l oracle 'ls -ltr
dm01db01: -rw-r----- 1 oracle oinstall 8192 Jan 17 03:45 /u01/app/oracle/product/12.1.0.2/dbhome/dbs/orapworcl1
dm01db01: -rw-r----- 1 oracle oinstall 8192 Jan 17 03:45 /u01/app/oracle/product/12.1.0.2/dbhome/dbs/orapworcl2
dm01db01: -rw-r----- 1 oracle oinstall 8192 Jan 17 03:45 /u01/app/oracle/product/12.1.0.2/dbhome/dbs/orapworcl3
dm01db01: -rw-r----- 1 oracle oinstall 8192 Jan 17 03:45 /u01/app/oracle/product/12.1.0.2/dbhome/dbs/orapworcl4
dm01db01: -rw-r----- 1 oracle oinstall 8192 Jan 17 03:45 /u01/app/oracle/product/12.1.0.2/dbhome/dbs/orapworcl5
dm01db01: -rw-r----- 1 oracle oinstall 8192 Jan 17 03:45 /u01/app/oracle/product/12.1.0.2/dbhome/dbs/orapworcl7
dm01db01: -rw-r----- 1 oracle oinstall 8192 Jan 17 03:45 /u01/app/oracle/product/12.1.0.2/dbhome/dbs/orapworcl6
dm01db01: -rw-r----- 1 oracle oinstall 8192 Jan 17 03:45 /u01/app/oracle/product/12.1.0.2/dbhome/dbs/orapworcl8

dm01db01-orcl1 {/home/oracle}:orapwd file='+DATA/orcl/orapworcl' entries=10 dbuniquename='orcl' format=12

Enter password for SYS:

ASMCMD> ls -l +DATA/ORCL/orapworcl
Type      Redund  Striped  Time             Sys  Name
PASSWORD  HIGH    COARSE   JAN 26 06:00:00  N    orapworcl => +DATA/ORCL/PASSWORD/pwdorcl.272.934265865

dm01db01-+ASM1 {/home/oracle}:orapwd file='+DATA/orapwASM' entries=10 dbuniquename='ASM' format=12

Enter password for SYS:

OPW-00019: Failed to update the CRS resource with DB password file location

dm01db01-+ASM1 {/home/oracle}:srvctl modify asm -pwfile +DATA/orapwASM

dm01db01-+ASM1 {/home/oracle}:srvctl config asm -detail
ASM home: <CRS home>
Password file: +DATA/orapwASM
ASM listener: LISTENER
ASM is enabled.
ASM is individually enabled on nodes:
ASM is individually disabled on nodes:

ASMCMD [+DATA] > ls -l orapwasm
Type      Redund  Striped  Time             Sys  Name
PASSWORD  HIGH    COARSE   JAN 27 04:00:00  N    orapwasm => +DATA/ASM/PASSWORD/pwdasm.399.934344779

• The disk group where the OCR, SPFILE, and ORAPWD files are stored has Oracle ASM compatibility (COMPATIBLE.ASM) set to 12.1 or higher.

SQL> col name for a40
SQL> col value for a40
SQL> set lines 200
SQL> set pages 200
SQL> select a.group_number,b.name dgname,a.name,a.value from v$asm_attribute a, v$asm_diskgroup b where a.name in ('au_size','disk_repair_time','compatible.rdbms','compatible.asm') and a.group_number=b.group_number order by b.name,a.name;

GROUP_NUMBER DGNAME                         NAME                                     VALUE
------------ ------------------------------ --------------------------               ------------------------
           3 ACFS_DG                        au_size                                  4194304
           3 ACFS_DG                        compatible.asm                           12.1.0.0.0
           3 ACFS_DG                        compatible.rdbms                         12.1.0.0.0
           3 ACFS_DG                        disk_repair_time                         3.6h
           1 DATA                           au_size                                  4194304
           1 DATA                           compatible.asm                           12.1.0.2.0
           1 DATA                           compatible.rdbms                         11.2.0.4.0
           1 DATA                           disk_repair_time                         3.6h
           2 RECO                           au_size                                  4194304
           2 RECO                           compatible.asm                           12.1.0.2.0
           2 RECO                           compatible.rdbms                         11.2.0.4.0
           2 RECO                           disk_repair_time                         3.6h

dm01db01-+ASM1 {/home/oracle}:/u01/app/12.1.0.2/grid/bin/crsctl stat res -t
--------------------------------------------------------------------------------
Name           Target  State        Server                   State details
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.ACFS_DG.ACFSVOL.advm
               ONLINE  ONLINE       dm01db01                 Volume device /dev/a
                                                             sm/acfsvol-228 is online,STABLE
               ONLINE  ONLINE       dm01db02                 Volume device /dev/a
                                                             sm/acfsvol-228 is online,STABLE
               ONLINE  ONLINE       dm01db03                 Volume device /dev/a
                                                             sm/acfsvol-228 is online,STABLE
               ONLINE  ONLINE       dm01db04                 Volume device /dev/a
                                                             sm/acfsvol-228 is online,STABLE
               ONLINE  ONLINE       dm01db05                 Volume device /dev/a
                                                             sm/acfsvol-228 is online,STABLE
               ONLINE  ONLINE       dm01db06                 Volume device /dev/a
                                                             sm/acfsvol-228 is online,STABLE
               ONLINE  ONLINE       dm01db07                 Volume device /dev/a
                                                             sm/acfsvol-228 is online,STABLE
               ONLINE  ONLINE       dm01db08                 Volume device /dev/a
                                                             sm/acfsvol-228 is online,STABLE
ora.ACFS_DG.dg
               ONLINE  ONLINE       dm01db01                 STABLE
               ONLINE  ONLINE       dm01db02                 STABLE
               ONLINE  ONLINE       dm01db03                 STABLE
               ONLINE  ONLINE       dm01db04                 STABLE
               ONLINE  ONLINE       dm01db05                 STABLE
               ONLINE  ONLINE       dm01db06                 STABLE
               ONLINE  ONLINE       dm01db07                 STABLE
               ONLINE  ONLINE       dm01db08                 STABLE
ora.DATA.dg
               ONLINE  ONLINE       dm01db01                 STABLE
               ONLINE  ONLINE       dm01db02                 STABLE
               ONLINE  ONLINE       dm01db03                 STABLE
               ONLINE  ONLINE       dm01db04                 STABLE
               ONLINE  ONLINE       dm01db05                 STABLE
               ONLINE  ONLINE       dm01db06                 STABLE
               ONLINE  ONLINE       dm01db07                 STABLE
               ONLINE  ONLINE       dm01db08                 STABLE
ora.LISTENER.lsnr
               ONLINE  ONLINE       dm01db01                 STABLE
               ONLINE  ONLINE       dm01db02                 STABLE
               ONLINE  ONLINE       dm01db03                 STABLE
               ONLINE  ONLINE       dm01db04                 STABLE
               ONLINE  ONLINE       dm01db05                 STABLE
               ONLINE  ONLINE       dm01db06                 STABLE
               ONLINE  ONLINE       dm01db07                 STABLE
               ONLINE  ONLINE       dm01db08                 STABLE
ora.RECO.dg
               ONLINE  ONLINE       dm01db01                 STABLE
               ONLINE  ONLINE       dm01db02                 STABLE
               ONLINE  ONLINE       dm01db03                 STABLE
               ONLINE  ONLINE       dm01db04                 STABLE
               ONLINE  ONLINE       dm01db05                 STABLE
               ONLINE  ONLINE       dm01db06                 STABLE
               ONLINE  ONLINE       dm01db07                 STABLE
               ONLINE  ONLINE       dm01db08                 STABLE
ora.acfs_dg.acfsvol.acfs
               ONLINE  ONLINE       dm01db01                 mounted on /acfs,STABLE
               ONLINE  ONLINE       dm01db02                 mounted on /acfs,STABLE
               ONLINE  ONLINE       dm01db03                 mounted on /acfs,STABLE
               ONLINE  ONLINE       dm01db04                 mounted on /acfs,STABLE
               ONLINE  ONLINE       dm01db05                 mounted on /acfs,STABLE
               ONLINE  ONLINE       dm01db06                 mounted on /acfs,STABLE
               ONLINE  ONLINE       dm01db07                 mounted on /acfs,STABLE
               ONLINE  ONLINE       dm01db08                 mounted on /acfs,STABLE
ora.asm
               ONLINE  ONLINE       dm01db01                 Started,STABLE
               ONLINE  ONLINE       dm01db02                 Started,STABLE
               ONLINE  ONLINE       dm01db03                 Started,STABLE
               ONLINE  ONLINE       dm01db04                 Started,STABLE
               ONLINE  ONLINE       dm01db05                 Started,STABLE
               ONLINE  ONLINE       dm01db06                 Started,STABLE
               ONLINE  ONLINE       dm01db07                 Started,STABLE
               ONLINE  ONLINE       dm01db08                 Started,STABLE
ora.net1.network
               ONLINE  ONLINE       dm01db01                 STABLE
               ONLINE  ONLINE       dm01db02                 STABLE
               ONLINE  ONLINE       dm01db03                 STABLE
               ONLINE  ONLINE       dm01db04                 STABLE
               ONLINE  ONLINE       dm01db05                 STABLE
               ONLINE  ONLINE       dm01db06                 STABLE
               ONLINE  ONLINE       dm01db07                 STABLE
               ONLINE  ONLINE       dm01db08                 STABLE
ora.ons
               ONLINE  ONLINE       dm01db01                 STABLE
               ONLINE  ONLINE       dm01db02                 STABLE
               ONLINE  ONLINE       dm01db03                 STABLE
               ONLINE  ONLINE       dm01db04                 STABLE
               ONLINE  ONLINE       dm01db05                 STABLE
               ONLINE  ONLINE       dm01db06                 STABLE
               ONLINE  ONLINE       dm01db07                 STABLE
               ONLINE  ONLINE       dm01db08                 STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       dm01db08                 STABLE
ora.LISTENER_SCAN2.lsnr
      1        ONLINE  ONLINE       dm01db07                 STABLE
ora.LISTENER_SCAN3.lsnr
      1        ONLINE  ONLINE       dm01db05                 STABLE
ora.dm01db01.vip
      1        ONLINE  ONLINE       dm01db01                 STABLE
ora.dm01db02.vip
      1        ONLINE  ONLINE       dm01db02                 STABLE
ora.dm01db03.vip
      1        ONLINE  ONLINE       dm01db03                 STABLE
ora.dm01db04.vip
      1        ONLINE  ONLINE       dm01db04                 STABLE
ora.dm01db05.vip
      1        ONLINE  ONLINE       dm01db05                 STABLE
ora.dm01db06.vip
      1        ONLINE  ONLINE       dm01db06                 STABLE
ora.dm01db07.vip
      1        ONLINE  ONLINE       dm01db07                 STABLE
ora.dm01db08.vip
      1        ONLINE  ONLINE       dm01db08                 STABLE
ora.cvu
      1        ONLINE  ONLINE       dm01db06                 STABLE
ora.oc4j
      1        OFFLINE OFFLINE                               STABLE
ora.orcl.db
      1        ONLINE  ONLINE       dm01db03                 Open,STABLE
      2        ONLINE  ONLINE       dm01db04                 Open,STABLE
      3        ONLINE  ONLINE       dm01db01                 Open,STABLE
      4        ONLINE  ONLINE       dm01db05                 Open,STABLE
      5        ONLINE  ONLINE       dm01db06                 Open,STABLE
      6        ONLINE  ONLINE       dm01db02                 Open,STABLE
      7        ONLINE  ONLINE       dm01db07                 Open,STABLE
      8        ONLINE  ONLINE       dm01db08                 Open,STABLE
ora.scan1.vip
      1        ONLINE  ONLINE       dm01db08                 STABLE
ora.scan2.vip
      1        ONLINE  ONLINE       dm01db07                 STABLE
ora.scan3.vip
      1        ONLINE  ONLINE       dm01db05                 STABLE


ASMCMD> showclustermode
ASM cluster : Flex mode disabled

dm01db01-+ASM1 {/home/oracle}:srvctl status asm -detail
ASM is running on dm01db01,dm01db03,dm01db02,dm01db05,dm01db04,dm01db07,dm01db06,dm01db08
ASM is enabled.

Conversion Steps

• Set the display and Start ASMCA

• Click on "ASM Instances" and then "Convert to Oracle Flex ASM"

• On Convert to Oracle Flex ASM page, specify:
• ASM Listener Port number: 1521
• Select network to be used for ASM: bond0 (cluster_interconnect)
• Check the box if you plan on using ASM for pre 12c databases.
• Click Ok

• Click Yes

• The processing page will be displayed.

• Open the script and view the script contents.

[root@dm01db01 ~]# cat /u01/app/oracle/cfgtoollogs/asmca/scripts/converttoFlexASM.sh  | more
#!/bin/sh

cd /u01/app/oracle/cfgtoollogs/asmca/scripts
su oracle -c "/u01/app/12.1.0.2/grid/bin/srvctl add asm -proxy"
if [ "$?" != "0" ]; then
        echo "Error adding ASM Proxy"
        exit 1
fi
/u01/app/12.1.0.2/grid/srvm/admin/srvmhelper -updateDepASM
if [ "$?" != "0" ]; then
        echo "Error updating ASM resource dependencies"
        exit 1
fi
for node in dm01db01 dm01db02 dm01db03 dm01db04 dm01db05 dm01db06 dm01db07 dm01db08
do
        /u01/app/12.1.0.2/grid/bin/crsctl stop cluster -n $node
        if [ "$?" != "0" ]; then
                echo "Error stopping Oracle Grid Infrastructure in node $node"
                exit 1
        fi
        /u01/app/12.1.0.2/grid/bin/crsctl start cluster -n $node

        stat=1
        cmd="/u01/app/12.1.0.2/grid/bin/crsctl status server $node"
        while [ $stat = 1 ]
        do
                sleep 5
                op=`$cmd`
                op=`echo $op | /bin/cut -d' ' -f2  | /bin/cut -d'=' -f2`
                if [ "$op" = "ONLINE" ]
                then
                        stat=0
                        echo "Oracle Grid Infrastructure restarted in node $node"
                fi
        done
        if [ "$node" = dm01db01 ]; then
                for lsnr in ASMNET1LSNR_ASM
                do
                        /u01/app/12.1.0.2/grid/bin/srvctl start listener -listener $lsnr
                        lsnrret=$?
                        if [ "$lsnrret" = "2" ]; then
                                echo "ASM listener $lsnr running already"
                        elif [ "$lsnrret" != "0" ]; then
                                echo "Error starting ASM listener $lsnr"
                                exit 1
                        fi
                done
        fi
done

• Execute the script on node 1 as root user as follows:

dm01db01-+ASM1 {/dev/asm}:df -k | grep asm
/dev/asm/acfsvol-228

Stop the ACFS file system before executing the script.

dm01db01-+ASM1 {/dev/asm}:srvctl status filesystem -device /dev/asm/acfsvol-228
ACFS file system /acfs is mounted on nodes

dm01db01,dm01db02,dm01db03,dm01db04,dm01db05,dm01db06,dm01db07,dm01db08

dm01db01-+ASM1 {/home/oracle}:srvctl stop filesystem -d /dev/asm/acfsvol-228 -f

dm01db01-+ASM1 {/home/oracle}:srvctl status filesystem -d /dev/asm/acfsvol-228
ACFS file system /acfs is not mounted

dm01db01-+ASM1 {/dev/asm}:df -k | grep asm

Finally execute the script

[root@dm01db01 ~]# /u01/app/oracle/cfgtoollogs/asmca/scripts/converttoFlexASM.sh
CRS-2673: Attempting to stop 'ora.crsd' on 'dm01db01'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'dm01db01'
CRS-2673: Attempting to stop 'ora.orcldb.db' on 'dm01db01'
CRS-2673: Attempting to stop 'ttsrv_dm01db01' on 'dm01db01'
CRS-2673: Attempting to stop 'ora.acfs_dg.acfsvol.acfs' on 'dm01db01'
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'dm01db01'
CRS-2673: Attempting to stop 'ora.RECO.dg' on 'dm01db01'
CRS-2673: Attempting to stop 'ora.DATA.dg' on 'dm01db01'
CRS-2673: Attempting to stop 'ora.orcl.db' on 'dm01db01'
CRS-2673: Attempting to stop 'ora.ACFS_DG.dg' on 'dm01db01'
CRS-2677: Stop of 'ora.ACFS_DG.dg' on 'dm01db01' succeeded
CRS-2677: Stop of 'ora.RECO.dg' on 'dm01db01' succeeded
CRS-2677: Stop of 'ora.DATA.dg' on 'dm01db01' succeeded
CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'dm01db01' succeeded
CRS-2673: Attempting to stop 'ora.dm01db01.vip' on 'dm01db01'
CRS-2677: Stop of 'ora.dm01db01.vip' on 'dm01db01' succeeded
CRS-2672: Attempting to start 'ora.dm01db01.vip' on 'dm01db05'
CRS-2677: Stop of 'ora.orcldb.db' on 'dm01db01' succeeded
CRS-2677: Stop of 'ora.orcl.db' on 'dm01db01' succeeded
CRS-2676: Start of 'ora.dm01db01.vip' on 'dm01db05' succeeded
CRS-2673: Attempting to stop 'ora.ons' on 'dm01db01'
CRS-2677: Stop of 'ora.ons' on 'dm01db01' succeeded
CRS-2673: Attempting to stop 'ora.net1.network' on 'dm01db01'
CRS-2677: Stop of 'ora.net1.network' on 'dm01db01' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'dm01db01' has completed
CRS-2677: Stop of 'ora.crsd' on 'dm01db01' succeeded
CRS-2673: Attempting to stop 'ora.ctssd' on 'dm01db01'
CRS-2673: Attempting to stop 'ora.evmd' on 'dm01db01'
CRS-2673: Attempting to stop 'ora.storage' on 'dm01db01'
CRS-2677: Stop of 'ora.storage' on 'dm01db01' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'dm01db01'
CRS-2677: Stop of 'ora.ctssd' on 'dm01db01' succeeded
CRS-2677: Stop of 'ora.evmd' on 'dm01db01' succeeded
CRS-2677: Stop of 'ora.asm' on 'dm01db01' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'dm01db01'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'dm01db01' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'dm01db01'
CRS-2677: Stop of 'ora.cssd' on 'dm01db01' succeeded
CRS-2673: Attempting to stop 'ora.diskmon' on 'dm01db01'
CRS-2677: Stop of 'ora.diskmon' on 'dm01db01' succeeded
CRS-2672: Attempting to start 'ora.evmd' on 'dm01db01'
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'dm01db01'
CRS-2676: Start of 'ora.cssdmonitor' on 'dm01db01' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'dm01db01'
CRS-2672: Attempting to start 'ora.diskmon' on 'dm01db01'
CRS-2676: Start of 'ora.evmd' on 'dm01db01' succeeded
CRS-2676: Start of 'ora.diskmon' on 'dm01db01' succeeded
CRS-2676: Start of 'ora.cssd' on 'dm01db01' succeeded
CRS-2672: Attempting to start 'ora.ctssd' on 'dm01db01'
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'dm01db01'
CRS-2676: Start of 'ora.ctssd' on 'dm01db01' succeeded
CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'dm01db01' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'dm01db01'
CRS-2676: Start of 'ora.asm' on 'dm01db01' succeeded
CRS-2672: Attempting to start 'ora.storage' on 'dm01db01'
CRS-2676: Start of 'ora.storage' on 'dm01db01' succeeded
CRS-2672: Attempting to start 'ora.crsd' on 'dm01db01'
CRS-2676: Start of 'ora.crsd' on 'dm01db01' succeeded
Oracle Grid Infrastructure restarted in node dm01db01
CRS-2673: Attempting to stop 'ora.crsd' on 'dm01db02'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on 'dm01db02'
CRS-2673: Attempting to stop 'ora.acfs_dg.acfsvol.acfs' on 'dm01db02'
CRS-2673: Attempting to stop 'ora.orcl.db' on 'dm01db02'
CRS-2673: Attempting to stop 'ora.orcldb.db' on 'dm01db02'
CRS-2673: Attempting to stop 'ora.RECO.dg' on 'dm01db02'
CRS-2673: Attempting to stop 'ora.DATA.dg' on 'dm01db02'
CRS-2677: Stop of 'ora.RECO.dg' on 'dm01db02' succeeded
CRS-2677: Stop of 'ora.DATA.dg' on 'dm01db02' succeeded
CRS-2677: Stop of 'ora.orcl.db' on 'dm01db02' succeeded
CRS-2677: Stop of 'ora.orcldb.db' on 'dm01db02' succeeded
...
...

[root@dm01db01 ~]#

• Go back to ASMCA and click close.

Verify the cluster mode now.

ASMCMD> showclustermode
ASM cluster : Flex mode enabled

Conclusion
In this article we have learned about Flex ASM in Oracle database 12c and how to convert a Standard ASM to Flex ASM to take advantage of 12c new feature.