Sunday, 19 February 2017

How To Add Exadata Storage Server To Existing Exadata Cluster

Overview

You purchased a new Storage cell for expansion or removed Storage from an existing Cluster and want to add it to the new a Cluster due to space demand.

In this article I will demonstrate how to add a storage cell to an existing Exadata Database Machine.

Here we are adding new storage cell with 600 GB disks to existing Exadata database machine using 600 GB disks.


Environment

  • Exadata Database Machine X2-2 Full Rack
  • Exadata Storage Software version 12.1.2.2.0
  • Oracle Grid/Database Version 11.2.0.4


Pre Checks:

  • Create a new file named "new_cell_group" containing only the hostnames for the new cells.

[root@dm01db01 ~]# vi new_cell_group

[root@dm01db01 ~]# cat new_cell_group
dm01cel05
  • It is assumed that the file "cell_group" only contains the original cells, not the new ones.  If you have already modified "cell_group" to contain the new cells, just comment it out or remove it from the file.

[root@dm01db01 ~]# cat cell_group
dm01cel01
dm01cel02
dm01cel03
dm01cel04
dm01cel05
dm01cel06
dm01cel07
dm01cel08
dm01cel09
dm01cel10
dm01cel11
dm01cel12
dm01cel13
dm01cel14

[root@dm01db01 ~]# vi cell_group

[root@dm01db01 ~]# cat cell_group
dm01cel01
dm01cel02
dm01cel03
dm01cel04
#dm01cel05
dm01cel06
dm01cel07
dm01cel08
dm01cel09
dm01cel10
dm01cel11
dm01cel12
dm01cel13
dm01cel14
  • Here we use the root user to run the cellcli commands; ensure that user equivalency setup for root user between Storage cells and Compute nodes.

[root@dm01db01 ~]# dcli -g dbs_group -l root 'uptime'
dm01db01: 09:55:05 up 425 days, 23:16,  1 user,  load average: 3.26, 3.37, 3.39
dm01db02: 09:55:05 up 117 days, 18:20,  0 users,  load average: 1.19, 1.42, 1.56
dm01db03: 09:55:05 up 85 days, 10:07,  0 users,  load average: 6.25, 6.20, 6.22
dm01db04: 09:55:05 up 519 days, 11:33,  0 users,  load average: 1.53, 1.48, 1.47
dm01db05: 09:55:05 up 519 days, 12:45,  0 users,  load average: 1.36, 1.35, 1.47
dm01db06: 09:55:05 up 515 days, 21:40,  0 users,  load average: 1.47, 1.36, 1.36
dm01db07: 09:55:05 up 519 days, 12:03,  0 users,  load average: 1.44, 1.64, 1.71
dm01db08: 09:55:05 up 519 days, 11:29,  0 users,  load average: 1.78, 1.90, 1.78

[root@dm01db01 ~]# dcli -g cell_group -l root 'uptime'
dm01cel01: 09:55:15 up 466 days, 20:25,  0 users,  load average: 1.44, 1.35, 1.26
dm01cel02: 09:55:15 up 519 days, 14:48,  0 users,  load average: 1.49, 1.44, 1.51
dm01cel03: 09:55:15 up 519 days, 14:47,  0 users,  load average: 1.01, 0.96, 1.10
dm01cel04: 09:55:15 up 519 days, 14:47,  0 users,  load average: 1.40, 1.32, 1.24
dm01cel06: 09:55:15 up 519 days, 14:46,  0 users,  load average: 1.10, 1.25, 1.30
dm01cel07: 09:55:15 up 519 days, 14:46,  0 users,  load average: 0.99, 1.14, 1.18
dm01cel08: 09:55:15 up 466 days, 19:37,  0 users,  load average: 1.52, 1.24, 1.17
dm01cel09: 09:55:15 up 519 days, 14:46,  0 users,  load average: 1.00, 1.40, 1.56
dm01cel10: 09:55:15 up 519 days, 14:46,  0 users,  load average: 1.09, 1.22, 1.22
dm01cel11: 09:55:15 up 519 days, 14:46,  0 users,  load average: 1.24, 1.27, 1.22
dm01cel12: 09:55:15 up 519 days, 14:46,  0 users,  load average: 1.11, 1.14, 1.14
dm01cel13: 09:55:15 up 519 days, 14:46,  0 users,  load average: 1.13, 1.27, 1.16
dm01cel14: 09:55:15 up 519 days, 14:46,  0 users,  load average: 1.20, 1.12, 1.15

Ensure Cell Disks are Ready

  • First verify if any cell disks already exist:

[root@dm01db01 ~]# dcli -g new_cell_group -l root "cellcli -e list celldisk attributes name,status,freespace where disktype=harddisk"
dm01cel05: CD_00_dm01cel05       normal  0
dm01cel05: CD_01_dm01cel05       normal  0
dm01cel05: CD_02_dm01cel05       normal  0
dm01cel05: CD_03_dm01cel05       normal  0
dm01cel05: CD_04_dm01cel05       normal  0
dm01cel05: CD_05_dm01cel05       normal  0
dm01cel05: CD_06_dm01cel05       normal  0
dm01cel05: CD_07_dm01cel05       normal  0
dm01cel05: CD_08_dm01cel05       normal  0
dm01cel05: CD_09_dm01cel05       normal  0
dm01cel05: CD_10_dm01cel05       normal  0
dm01cel05: CD_11_dm01cel05       normal  0

In my case the cell disks exists.
  • If no cell disks are created. There will be no output.

[root@dm01db01 ~]# dcli -g new_cell_group -l root "cellcli -e list celldisk attributes name,status,freespace where disktype=harddisk"
[root@dm01db01 ~]#
  • If this command returns a list of cell disks, then you should verify if any grid disks exist:

[root@dm01db01 ~]# dcli -g new_cell_group -l root "cellcli -e list griddisk where disktype=harddisk"
dm01cel05: DATA_CD_00_dm01cel05          active
dm01cel05: DATA_CD_01_dm01cel05          active
dm01cel05: DATA_CD_02_dm01cel05          active
dm01cel05: DATA_CD_03_dm01cel05          active
dm01cel05: DATA_CD_04_dm01cel05          active
dm01cel05: DATA_CD_05_dm01cel05          active
dm01cel05: DATA_CD_06_dm01cel05          active
dm01cel05: DATA_CD_07_dm01cel05          active
dm01cel05: DATA_CD_08_dm01cel05          active
dm01cel05: DATA_CD_09_dm01cel05          active
dm01cel05: DATA_CD_10_dm01cel05          active
dm01cel05: DATA_CD_11_dm01cel05          active
dm01cel05: RECO_CD_00_dm01cel05          active
dm01cel05: RECO_CD_01_dm01cel05          active
dm01cel05: RECO_CD_02_dm01cel05          active
dm01cel05: RECO_CD_03_dm01cel05          active
dm01cel05: RECO_CD_04_dm01cel05          active
dm01cel05: RECO_CD_05_dm01cel05          active
dm01cel05: RECO_CD_06_dm01cel05          active
dm01cel05: RECO_CD_07_dm01cel05          active
dm01cel05: RECO_CD_08_dm01cel05          active
dm01cel05: RECO_CD_09_dm01cel05          active
dm01cel05: RECO_CD_10_dm01cel05          active
dm01cel05: RECO_CD_11_dm01cel05          active
dm01cel05: DBFS_DG_CD_02_dm01cel05      active
dm01cel05: DBFS_DG_CD_03_dm01cel05      active
dm01cel05: DBFS_DG_CD_04_dm01cel05      active
dm01cel05: DBFS_DG_CD_05_dm01cel05      active
dm01cel05: DBFS_DG_CD_06_dm01cel05      active
dm01cel05: DBFS_DG_CD_07_dm01cel05      active
dm01cel05: DBFS_DG_CD_08_dm01cel05      active
dm01cel05: DBFS_DG_CD_09_dm01cel05      active
dm01cel05: DBFS_DG_CD_10_dm01cel05      active
dm01cel05: DBFS_DG_CD_11_dm01cel05      active

In my case grid disks exists.
  • If no grid disks are created. There will be no output.

[root@dm01db01 ~]# dcli -g new_cell_group -l root "cellcli -e list griddisk where disktype=harddisk"
[root@dm01db01 ~]#

  • If any grid disks are present, then you must ensure they are not in use by looking in V$ASM_DISK and verifying the mount status is "closed" and the header status is "former" or "candidate":

dm01db01-orcldb1 {/home/oracle}:. oraenv
ORACLE_SID = [orcldb1] ? +ASM1
The Oracle base remains unchanged with value /u01/app/oracle
dm01db01-+ASM1 {/home/oracle}:sqlplus / as sysasm

SQL*Plus: Release 11.2.0.4.0 Production on Mon Oct 17 10:01:15 2016

Copyright (c) 1982, 2013, Oracle.  All rights reserved.

Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options

SQL> select name, path, state, mode_status,header_status, mount_status from v$asm_disk where header_status <> 'MEMBER' order by path;

no rows selected
  • If it is OK to drop the grid disks, you can use the following command to drop set of grid disks (by prefix like, RECO):

[root@dm01db01 ~]# dcli -g new_cell_group -l root "cellcli -e drop griddisk all harddisk prefix=RECO"
dm01cel05: GridDisk RECO_CD_00_dm01cel05 successfully dropped
dm01cel05: GridDisk RECO_CD_01_dm01cel05 successfully dropped
dm01cel05: GridDisk RECO_CD_02_dm01cel05 successfully dropped
dm01cel05: GridDisk RECO_CD_03_dm01cel05 successfully dropped
dm01cel05: GridDisk RECO_CD_04_dm01cel05 successfully dropped
dm01cel05: GridDisk RECO_CD_05_dm01cel05 successfully dropped
dm01cel05: GridDisk RECO_CD_06_dm01cel05 successfully dropped
dm01cel05: GridDisk RECO_CD_07_dm01cel05 successfully dropped
dm01cel05: GridDisk RECO_CD_08_dm01cel05 successfully dropped
dm01cel05: GridDisk RECO_CD_09_dm01cel05 successfully dropped
dm01cel05: GridDisk RECO_CD_10_dm01cel05 successfully dropped
dm01cel05: GridDisk RECO_CD_11_dm01cel05 successfully dropped

[root@dm01db01 ~]# dcli -g new_cell_group -l root "cellcli -e drop griddisk all harddisk prefix=DATA"
dm01cel05: GridDisk DATA_CD_00_dm01cel05 successfully dropped
dm01cel05: GridDisk DATA_CD_01_dm01cel05 successfully dropped
dm01cel05: GridDisk DATA_CD_02_dm01cel05 successfully dropped
dm01cel05: GridDisk DATA_CD_03_dm01cel05 successfully dropped
dm01cel05: GridDisk DATA_CD_04_dm01cel05 successfully dropped
dm01cel05: GridDisk DATA_CD_05_dm01cel05 successfully dropped
dm01cel05: GridDisk DATA_CD_06_dm01cel05 successfully dropped
dm01cel05: GridDisk DATA_CD_07_dm01cel05 successfully dropped
dm01cel05: GridDisk DATA_CD_08_dm01cel05 successfully dropped
dm01cel05: GridDisk DATA_CD_09_dm01cel05 successfully dropped
dm01cel05: GridDisk DATA_CD_10_dm01cel05 successfully dropped
dm01cel05: GridDisk DATA_CD_11_dm01cel05 successfully dropped

[root@dm01db01 ~]# dcli -g new_cell_group -l root "cellcli -e drop griddisk all harddisk prefix=DBFS_DG"
dm01cel05: GridDisk DBFS_DG_CD_02_dm01cel05 successfully dropped
dm01cel05: GridDisk DBFS_DG_CD_03_dm01cel05 successfully dropped
dm01cel05: GridDisk DBFS_DG_CD_04_dm01cel05 successfully dropped
dm01cel05: GridDisk DBFS_DG_CD_05_dm01cel05 successfully dropped
dm01cel05: GridDisk DBFS_DG_CD_06_dm01cel05 successfully dropped
dm01cel05: GridDisk DBFS_DG_CD_07_dm01cel05 successfully dropped
dm01cel05: GridDisk DBFS_DG_CD_08_dm01cel05 successfully dropped
dm01cel05: GridDisk DBFS_DG_CD_09_dm01cel05 successfully dropped
dm01cel05: GridDisk DBFS_DG_CD_10_dm01cel05 successfully dropped
dm01cel05: GridDisk DBFS_DG_CD_11_dm01cel05 successfully dropped
  • Verify Grid disks are dropped

[root@dm01db01 ~]# dcli -g new_cell_group -l root "cellcli -e list griddisk where disktype=harddisk"
[root@dm01db01 ~]#
  • Drop cell disks

[root@dm01db01 ~]# dcli -g new_cell_group -l root "cellcli -e drop celldisk all harddisk force"
dm01cel05: CellDisk CD_00_dm01cel05 successfully dropped
dm01cel05: CellDisk CD_01_dm01cel05 successfully dropped
dm01cel05: CellDisk CD_02_dm01cel05 successfully dropped
dm01cel05: CellDisk CD_03_dm01cel05 successfully dropped
dm01cel05: CellDisk CD_04_dm01cel05 successfully dropped
dm01cel05: CellDisk CD_05_dm01cel05 successfully dropped
dm01cel05: CellDisk CD_06_dm01cel05 successfully dropped
dm01cel05: CellDisk CD_07_dm01cel05 successfully dropped
dm01cel05: CellDisk CD_08_dm01cel05 successfully dropped
dm01cel05: CellDisk CD_09_dm01cel05 successfully dropped
dm01cel05: CellDisk CD_10_dm01cel05 successfully dropped
dm01cel05: CellDisk CD_11_dm01cel05 successfully dropped
  • If no cell disks exist, create cell disks on all hard disks (output will be similar to the following): 

[root@dm01db01 ~]# dcli -g new_cell_group -l root "cellcli -e create celldisk all harddisk"
dm01cel05: CellDisk CD_00_dm01cel05 successfully created
dm01cel05: CellDisk CD_01_dm01cel05 successfully created
dm01cel05: CellDisk CD_02_dm01cel05 successfully created
dm01cel05: CellDisk CD_03_dm01cel05 successfully created
dm01cel05: CellDisk CD_04_dm01cel05 successfully created
dm01cel05: CellDisk CD_05_dm01cel05 successfully created
dm01cel05: CellDisk CD_06_dm01cel05 successfully created
dm01cel05: CellDisk CD_07_dm01cel05 successfully created
dm01cel05: CellDisk CD_08_dm01cel05 successfully created
dm01cel05: CellDisk CD_09_dm01cel05 successfully created
dm01cel05: CellDisk CD_10_dm01cel05 successfully created
dm01cel05: CellDisk CD_11_dm01cel05 successfully created
  • Verify that celldisks were created (output should be similar to the following)

[root@dm01db01 ~]# dcli -g new_cell_group -l root "cellcli -e list celldisk attributes name,status,freespace where disktype=harddisk"
dm01cel05: CD_00_dm01cel05       normal  528.6875G
dm01cel05: CD_01_dm01cel05       normal  528.6875G
dm01cel05: CD_02_dm01cel05       normal  557.8125G
dm01cel05: CD_03_dm01cel05       normal  557.8125G
dm01cel05: CD_04_dm01cel05       normal  557.8125G
dm01cel05: CD_05_dm01cel05       normal  557.8125G
dm01cel05: CD_06_dm01cel05       normal  557.8125G
dm01cel05: CD_07_dm01cel05       normal  557.8125G
dm01cel05: CD_08_dm01cel05       normal  557.8125G
dm01cel05: CD_09_dm01cel05       normal  557.8125G
dm01cel05: CD_10_dm01cel05       normal  557.8125G
dm01cel05: CD_11_dm01cel05       normal  557.8125G

Create DATA and RECO grid disks at the same offsets found on the existing disks (outer tracks)

  • Verify the current sizes and offsets for existing grid disks (on the existing cells):

[root@dm01db01 ~]# dcli -g cell_group -l root "cellcli -e list griddisk attributes name,size | grep 'CD_03' | grep -i DBFS_DG"
dm01cel01: DBFS_DG_CD_03_dm01cel01      29.125G
dm01cel02: DBFS_DG_CD_03_dm01cel02      29.125G
dm01cel03: DBFS_DG_CD_03_dm01cel03      29.125G
dm01cel04: DBFS_DG_CD_03_dm01cel04      29.125G
dm01cel06: DBFS_DG_CD_03_dm01cel06      29.125G
dm01cel07: DBFS_DG_CD_03_dm01cel07      29.125G
dm01cel08: DBFS_DG_CD_03_dm01cel08      29.125G
dm01cel09: DBFS_DG_CD_03_dm01cel09      29.125G
dm01cel10: DBFS_DG_CD_03_dm01cel10      29.125G
dm01cel11: DBFS_DG_CD_03_dm01cel11      29.125G
dm01cel12: DBFS_DG_CD_03_dm01cel12      29.125G
dm01cel13: DBFS_DG_CD_03_dm01cel13      29.125G
dm01cel14: DBFS_DG_CD_03_dm01cel14      29.125G

[root@dm01db01 ~]# dcli -g cell_group -l root "cellcli -e list griddisk attributes name,size,offset | grep 'CD_03' | grep -i data"
dm01cel01: DATA_CD_03_dm01cel01          260G            32M
dm01cel02: DATA_CD_03_dm01cel02          260G            32M
dm01cel03: DATA_CD_03_dm01cel03          260G            32M
dm01cel04: DATA_CD_03_dm01cel04          260G            32M
dm01cel06: DATA_CD_03_dm01cel06          260G            32M
dm01cel07: DATA_CD_03_dm01cel07          260G            32M
dm01cel08: DATA_CD_03_dm01cel08          260G            32M
dm01cel09: DATA_CD_03_dm01cel09          260G            32M
dm01cel10: DATA_CD_03_dm01cel10          260G            32M
dm01cel11: DATA_CD_03_dm01cel11          260G            32M
dm01cel12: DATA_CD_03_dm01cel12          260G            32M
dm01cel13: DATA_CD_03_dm01cel13          260G            32M
dm01cel14: DATA_CD_03_dm01cel14          260G            32M

[root@dm01db01 ~]# dcli -g cell_group -l root "cellcli -e list griddisk attributes name,size,offset | grep 'CD_03' | grep -i reco"
dm01cel01: RECO_CD_03_dm01cel01          268.6875G       260.046875G
dm01cel02: RECO_CD_03_dm01cel02          268.6875G       260.046875G
dm01cel03: RECO_CD_03_dm01cel03          268.6875G       260.046875G
dm01cel04: RECO_CD_03_dm01cel04          268.6875G       260.046875G
dm01cel06: RECO_CD_03_dm01cel06          268.6875G       260.046875G
dm01cel07: RECO_CD_03_dm01cel07          268.6875G       260.046875G
dm01cel08: RECO_CD_03_dm01cel08          268.6875G       260.046875G
dm01cel09: RECO_CD_03_dm01cel09          268.6875G       260.046875G
dm01cel10: RECO_CD_03_dm01cel10          268.6875G       260.046875G
dm01cel11: RECO_CD_03_dm01cel11          268.6875G       260.046875G
dm01cel12: RECO_CD_03_dm01cel12          268.6875G       260.046875G
dm01cel13: RECO_CD_03_dm01cel13          268.6875G       260.046875G
dm01cel14: RECO_CD_03_dm01cel14          268.6875G       260.046875G

  • Create similar grid disks on the new cells with the same size; be sure to create them in order of offset position, from smallest to largest offset. 

[root@dm01db01 ~]# dcli -g new_cell_group -l root "cellcli -e CREATE GRIDDISK ALL HARDDISK PREFIX='DATA', size=260G"
dm01cel05: GridDisk DATA_CD_00_dm01cel05 successfully created
dm01cel05: GridDisk DATA_CD_01_dm01cel05 successfully created
dm01cel05: GridDisk DATA_CD_02_dm01cel05 successfully created
dm01cel05: GridDisk DATA_CD_03_dm01cel05 successfully created
dm01cel05: GridDisk DATA_CD_04_dm01cel05 successfully created
dm01cel05: GridDisk DATA_CD_05_dm01cel05 successfully created
dm01cel05: GridDisk DATA_CD_06_dm01cel05 successfully created
dm01cel05: GridDisk DATA_CD_07_dm01cel05 successfully created
dm01cel05: GridDisk DATA_CD_08_dm01cel05 successfully created
dm01cel05: GridDisk DATA_CD_09_dm01cel05 successfully created
dm01cel05: GridDisk DATA_CD_10_dm01cel05 successfully created
dm01cel05: GridDisk DATA_CD_11_dm01cel05 successfully created

[root@dm01db01 ~]# dcli -g new_cell_group -l root "cellcli -e CREATE GRIDDISK ALL HARDDISK PREFIX='RECO', size=268.6875G"
dm01cel05: GridDisk RECO_CD_00_dm01cel05 successfully created
dm01cel05: GridDisk RECO_CD_01_dm01cel05 successfully created
dm01cel05: GridDisk RECO_CD_02_dm01cel05 successfully created
dm01cel05: GridDisk RECO_CD_03_dm01cel05 successfully created
dm01cel05: GridDisk RECO_CD_04_dm01cel05 successfully created
dm01cel05: GridDisk RECO_CD_05_dm01cel05 successfully created
dm01cel05: GridDisk RECO_CD_06_dm01cel05 successfully created
dm01cel05: GridDisk RECO_CD_07_dm01cel05 successfully created
dm01cel05: GridDisk RECO_CD_08_dm01cel05 successfully created
dm01cel05: GridDisk RECO_CD_09_dm01cel05 successfully created
dm01cel05: GridDisk RECO_CD_10_dm01cel05 successfully created
dm01cel05: GridDisk RECO_CD_11_dm01cel05 successfully created

  • Validate that grid disks were created properly on the new cells (size and offset matches the original cells)

[root@dm01db01 ~]# dcli -g new_cell_group -l root "cellcli -e list griddisk attributes name,size,offset | grep -i data"
dm01cel05: DATA_CD_00_dm01cel05  260G            32M
dm01cel05: DATA_CD_01_dm01cel05  260G            32M
dm01cel05: DATA_CD_02_dm01cel05  260G            32M
dm01cel05: DATA_CD_03_dm01cel05  260G            32M
dm01cel05: DATA_CD_04_dm01cel05  260G            32M
dm01cel05: DATA_CD_05_dm01cel05  260G            32M
dm01cel05: DATA_CD_06_dm01cel05  260G            32M
dm01cel05: DATA_CD_07_dm01cel05  260G            32M
dm01cel05: DATA_CD_08_dm01cel05  260G            32M
dm01cel05: DATA_CD_09_dm01cel05  260G            32M
dm01cel05: DATA_CD_10_dm01cel05  260G            32M
dm01cel05: DATA_CD_11_dm01cel05  260G            32M

[root@dm01db01 ~]# dcli -g new_cell_group -l root "cellcli -e list griddisk attributes name,size,offset | grep -i reco"
dm01cel05: RECO_CD_00_dm01cel05  268.6875G       260.046875G
dm01cel05: RECO_CD_01_dm01cel05  268.6875G       260.046875G
dm01cel05: RECO_CD_02_dm01cel05  268.6875G       260.046875G
dm01cel05: RECO_CD_03_dm01cel05  268.6875G       260.046875G
dm01cel05: RECO_CD_04_dm01cel05  268.6875G       260.046875G
dm01cel05: RECO_CD_05_dm01cel05  268.6875G       260.046875G
dm01cel05: RECO_CD_06_dm01cel05  268.6875G       260.046875G
dm01cel05: RECO_CD_07_dm01cel05  268.6875G       260.046875G
dm01cel05: RECO_CD_08_dm01cel05  268.6875G       260.046875G
dm01cel05: RECO_CD_09_dm01cel05  268.6875G       260.046875G
dm01cel05: RECO_CD_10_dm01cel05  268.6875G       260.046875G
dm01cel05: RECO_CD_11_dm01cel05  268.6875G       260.046875G

Create the new DBFS_DG grid disks only on disks 3 through 12 of the new storage servers

  • List the size of the DBFS_DG grid disks on the original cells (they should all be the same size) - we'll call this size, "dbfs_dg_size":

[root@dm01db01 ~]# dcli -c dm01cel01 -l root "cellcli -e list griddisk attributes name,size | grep 'DBFS_DG'"
dm01cel01: DBFS_DG_CD_02_dm01cel01      29.125G
dm01cel01: DBFS_DG_CD_03_dm01cel01      29.125G
dm01cel01: DBFS_DG_CD_04_dm01cel01      29.125G
dm01cel01: DBFS_DG_CD_05_dm01cel01      29.125G
dm01cel01: DBFS_DG_CD_06_dm01cel01      29.125G
dm01cel01: DBFS_DG_CD_07_dm01cel01      29.125G
dm01cel01: DBFS_DG_CD_08_dm01cel01      29.125G
dm01cel01: DBFS_DG_CD_09_dm01cel01      29.125G
dm01cel01: DBFS_DG_CD_10_dm01cel01      29.125G
dm01cel01: DBFS_DG_CD_11_dm01cel01      29.125G

  • The DBFS_DG grid disks will be automatically placed just after the RECO grid disks on the innermost tracks.

NOTE: This command will "fail" on the first and second disks of each cell, but that is expected since there is no more free space on those disks due to the system partitions. The "error" message is similar to:  "Cell disks were skipped because they had no freespace for grid disks: CD_00_dm01cel05, CD_01_dm01cel05."

[root@dm01db01 ~]# dcli -g new_cell_group -l root "cellcli -e CREATE GRIDDISK ALL HARDDISK PREFIX='DBFS_DG', size=29.125G"
dm01cel05: Cell disks were skipped because they had no freespace for grid disks: CD_00_dm01cel05, CD_01_dm01cel05.
dm01cel05: GridDisk DBFS_DG_CD_02_dm01cel05 successfully created
dm01cel05: GridDisk DBFS_DG_CD_03_dm01cel05 successfully created
dm01cel05: GridDisk DBFS_DG_CD_04_dm01cel05 successfully created
dm01cel05: GridDisk DBFS_DG_CD_05_dm01cel05 successfully created
dm01cel05: GridDisk DBFS_DG_CD_06_dm01cel05 successfully created
dm01cel05: GridDisk DBFS_DG_CD_07_dm01cel05 successfully created
dm01cel05: GridDisk DBFS_DG_CD_08_dm01cel05 successfully created
dm01cel05: GridDisk DBFS_DG_CD_09_dm01cel05 successfully created
dm01cel05: GridDisk DBFS_DG_CD_10_dm01cel05 successfully created
dm01cel05: GridDisk DBFS_DG_CD_11_dm01cel05 successfully created

Validate that Grid Disks on all Cells are the Same Size

  • All Cells at once:

[root@dm01db01 ~]# dcli -g cell_group -l root "cellcli -e list griddisk attributes name,size,offset | grep -i data"
dm01cel01: DATA_CD_00_dm01cel01          260G            32M
dm01cel01: DATA_CD_01_dm01cel01          260G            32M
dm01cel01: DATA_CD_02_dm01cel01          260G            32M
dm01cel01: DATA_CD_03_dm01cel01          260G            32M
dm01cel01: DATA_CD_04_dm01cel01          260G            32M
dm01cel01: DATA_CD_05_dm01cel01          260G            32M
dm01cel01: DATA_CD_06_dm01cel01          260G            32M
dm01cel01: DATA_CD_07_dm01cel01          260G            32M
dm01cel01: DATA_CD_08_dm01cel01          260G            32M
dm01cel01: DATA_CD_09_dm01cel01          260G            32M
dm01cel01: DATA_CD_10_dm01cel01          260G            32M
dm01cel01: DATA_CD_11_dm01cel01          260G            32M
…..
dm01cel14: DATA_CD_00_dm01cel14          260G            32M
dm01cel14: DATA_CD_01_dm01cel14          260G            32M
dm01cel14: DATA_CD_02_dm01cel14          260G            32M
dm01cel14: DATA_CD_03_dm01cel14          260G            32M
dm01cel14: DATA_CD_04_dm01cel14          260G            32M
dm01cel14: DATA_CD_05_dm01cel14          260G            32M
dm01cel14: DATA_CD_06_dm01cel14          260G            32M
dm01cel14: DATA_CD_07_dm01cel14          260G            32M
dm01cel14: DATA_CD_08_dm01cel14          260G            32M
dm01cel14: DATA_CD_09_dm01cel14          260G            32M
dm01cel14: DATA_CD_10_dm01cel14          260G            32M
dm01cel14: DATA_CD_11_dm01cel14          260G            32M

[root@dm01db01 ~]# dcli -g cell_group -l root "cellcli -e list griddisk attributes name,size,offset | grep -i reco"
dm01cel01: RECO_CD_00_dm01cel01          268.6875G       260.046875G
dm01cel01: RECO_CD_01_dm01cel01          268.6875G       260.046875G
dm01cel01: RECO_CD_02_dm01cel01          268.6875G       260.046875G
dm01cel01: RECO_CD_03_dm01cel01          268.6875G       260.046875G
dm01cel01: RECO_CD_04_dm01cel01          268.6875G       260.046875G
dm01cel01: RECO_CD_05_dm01cel01          268.6875G       260.046875G
dm01cel01: RECO_CD_06_dm01cel01          268.6875G       260.046875G
dm01cel01: RECO_CD_07_dm01cel01          268.6875G       260.046875G
dm01cel01: RECO_CD_08_dm01cel01          268.6875G       260.046875G
dm01cel01: RECO_CD_09_dm01cel01          268.6875G       260.046875G
dm01cel01: RECO_CD_10_dm01cel01          268.6875G       260.046875G
dm01cel01: RECO_CD_11_dm01cel01          268.6875G       260.046875G
……
dm01cel13: RECO_CD_11_dm01cel13          268.6875G       260.046875G
dm01cel14: RECO_CD_00_dm01cel14          268.6875G       260.046875G
dm01cel14: RECO_CD_01_dm01cel14          268.6875G       260.046875G
dm01cel14: RECO_CD_02_dm01cel14          268.6875G       260.046875G
dm01cel14: RECO_CD_03_dm01cel14          268.6875G       260.046875G
dm01cel14: RECO_CD_04_dm01cel14          268.6875G       260.046875G
dm01cel14: RECO_CD_05_dm01cel14          268.6875G       260.046875G
dm01cel14: RECO_CD_06_dm01cel14          268.6875G       260.046875G
dm01cel14: RECO_CD_07_dm01cel14          268.6875G       260.046875G
dm01cel14: RECO_CD_08_dm01cel14          268.6875G       260.046875G
dm01cel14: RECO_CD_09_dm01cel14          268.6875G       260.046875G
dm01cel14: RECO_CD_10_dm01cel14          268.6875G       260.046875G
dm01cel14: RECO_CD_11_dm01cel14          268.6875G       260.046875G

[root@dm01db01 ~]# dcli -g cell_group -l root "cellcli -e list griddisk attributes name,size,offset | grep -i DBFS_DG"
dm01cel01: DBFS_DG_CD_02_dm01cel01      29.125G         528.734375G
dm01cel01: DBFS_DG_CD_03_dm01cel01      29.125G         528.734375G
dm01cel01: DBFS_DG_CD_04_dm01cel01      29.125G         528.734375G
dm01cel01: DBFS_DG_CD_05_dm01cel01      29.125G         528.734375G
dm01cel01: DBFS_DG_CD_06_dm01cel01      29.125G         528.734375G
dm01cel01: DBFS_DG_CD_07_dm01cel01      29.125G         528.734375G
dm01cel01: DBFS_DG_CD_08_dm01cel01      29.125G         528.734375G
dm01cel01: DBFS_DG_CD_09_dm01cel01      29.125G         528.734375G
dm01cel01: DBFS_DG_CD_10_dm01cel01      29.125G         528.734375G
dm01cel01: DBFS_DG_CD_11_dm01cel01      29.125G         528.734375G
….
dm01cel14: DBFS_DG_CD_02_dm01cel14      29.125G         528.734375G
dm01cel14: DBFS_DG_CD_03_dm01cel14      29.125G         528.734375G
dm01cel14: DBFS_DG_CD_04_dm01cel14      29.125G         528.734375G
dm01cel14: DBFS_DG_CD_05_dm01cel14      29.125G         528.734375G
dm01cel14: DBFS_DG_CD_06_dm01cel14      29.125G         528.734375G
dm01cel14: DBFS_DG_CD_07_dm01cel14      29.125G         528.734375G
dm01cel14: DBFS_DG_CD_08_dm01cel14      29.125G         528.734375G
dm01cel14: DBFS_DG_CD_09_dm01cel14      29.125G         528.734375G
dm01cel14: DBFS_DG_CD_10_dm01cel14      29.125G         528.734375G
dm01cel14: DBFS_DG_CD_11_dm01cel14      29.125G         528.734375G

  • One cell at a time:

[root@dm01db01 ~]# dcli -c dm01cel01 -l root "cellcli -e list griddisk attributes name,size,offset | grep -i data"
dm01cel01: DATA_CD_00_dm01cel01          260G            32M
dm01cel01: DATA_CD_01_dm01cel01          260G            32M
dm01cel01: DATA_CD_02_dm01cel01          260G            32M
dm01cel01: DATA_CD_03_dm01cel01          260G            32M
dm01cel01: DATA_CD_04_dm01cel01          260G            32M
dm01cel01: DATA_CD_05_dm01cel01          260G            32M
dm01cel01: DATA_CD_06_dm01cel01          260G            32M
dm01cel01: DATA_CD_07_dm01cel01          260G            32M
dm01cel01: DATA_CD_08_dm01cel01          260G            32M
dm01cel01: DATA_CD_09_dm01cel01          260G            32M
dm01cel01: DATA_CD_10_dm01cel01          260G            32M
dm01cel01: DATA_CD_11_dm01cel01          260G            32M

[root@dm01db01 ~]# dcli -c dm01cel01 -l root "cellcli -e list griddisk attributes name,size,offset | grep -i reco"
dm01cel01: RECO_CD_00_dm01cel01          268.6875G       260.046875G
dm01cel01: RECO_CD_01_dm01cel01          268.6875G       260.046875G
dm01cel01: RECO_CD_02_dm01cel01          268.6875G       260.046875G
dm01cel01: RECO_CD_03_dm01cel01          268.6875G       260.046875G
dm01cel01: RECO_CD_04_dm01cel01          268.6875G       260.046875G
dm01cel01: RECO_CD_05_dm01cel01          268.6875G       260.046875G
dm01cel01: RECO_CD_06_dm01cel01          268.6875G       260.046875G
dm01cel01: RECO_CD_07_dm01cel01          268.6875G       260.046875G
dm01cel01: RECO_CD_08_dm01cel01          268.6875G       260.046875G
dm01cel01: RECO_CD_09_dm01cel01          268.6875G       260.046875G
dm01cel01: RECO_CD_10_dm01cel01          268.6875G       260.046875G
dm01cel01: RECO_CD_11_dm01cel01          268.6875G       260.046875G

[root@dm01db01 ~]# dcli -c dm01cel01 -l root "cellcli -e list griddisk attributes name,size,offset | grep -i DBFS_DG"
dm01cel01: DBFS_DG_CD_02_dm01cel01      29.125G         528.734375G
dm01cel01: DBFS_DG_CD_03_dm01cel01      29.125G         528.734375G
dm01cel01: DBFS_DG_CD_04_dm01cel01      29.125G         528.734375G
dm01cel01: DBFS_DG_CD_05_dm01cel01      29.125G         528.734375G
dm01cel01: DBFS_DG_CD_06_dm01cel01      29.125G         528.734375G
dm01cel01: DBFS_DG_CD_07_dm01cel01      29.125G         528.734375G
dm01cel01: DBFS_DG_CD_08_dm01cel01      29.125G         528.734375G
dm01cel01: DBFS_DG_CD_09_dm01cel01      29.125G         528.734375G
dm01cel01: DBFS_DG_CD_10_dm01cel01      29.125G         528.734375G
dm01cel01: DBFS_DG_CD_11_dm01cel01      29.125G         528.734375G

Add the DBFS_DG grid disks to the existing DBFS_DG disk group

  • Update the cellip.ora file

[root@dm01db01 ~]# vi /etc/oracle/cell/network-config/cellip.ora

[root@dm01db01 network-config]# cat /etc/oracle/cell/network-config/cellip.ora
cell="192.168.2.9"
cell="192.168.2.10"
cell="192.168.2.11"
cell="192.168.2.12"
cell="192.168.2.13"
cell="192.168.2.14"
cell="192.168.2.15"
cell="192.168.2.16"
cell="192.168.2.17"
cell="192.168.2.18"
cell="192.168.2.19"
cell="192.168.2.20"
cell="192.168.2.21"
cell="192.168.2.22"

[root@dm01db01 ~]# cd /etc/oracle/cell/network-config/

[root@dm01db01 network-config]# dcli -g ~/dbs_group -l root -d /etc/oracle/cell/network-config -f cellip.ora

[root@dm01db01 network-config]# dcli -g ~/dbs_group -l root ls -l  /etc/oracle/cell/network-config/cellip.ora
dm01db01: -rwxr-xr-x 1 oracle oinstall 279 Oct 17 11:15 /etc/oracle/cell/network-config/cellip.ora
dm01db02: -rwx------ 1 oracle oinstall 279 Oct 17 11:15 /etc/oracle/cell/network-config/cellip.ora
dm01db03: -rwx------ 1 oracle oinstall 279 Oct 17 11:15 /etc/oracle/cell/network-config/cellip.ora
dm01db04: -rwx------ 1 oracle oinstall 279 Oct 17 11:15 /etc/oracle/cell/network-config/cellip.ora
dm01db05: -rwx------ 1 oracle oinstall 279 Oct 17 11:15 /etc/oracle/cell/network-config/cellip.ora
dm01db06: -rwx------ 1 oracle oinstall 279 Oct 17 11:15 /etc/oracle/cell/network-config/cellip.ora
dm01db07: -rwx------ 1 oracle oinstall 279 Oct 17 11:15 /etc/oracle/cell/network-config/cellip.ora
dm01db08: -rwx------ 1 oracle oinstall 279 Oct 17 11:15 /etc/oracle/cell/network-config/cellip.ora

dm01db01-orcldb1 {/home/oracle}:. oraenv
ORACLE_SID = [orcldb1] ? +ASM1
The Oracle base remains unchanged with value /u01/app/oracle
dm01db01-+ASM1 {/home/oracle}:sqlplus / as sysasm

SQL*Plus: Release 11.2.0.4.0 Production on Mon Oct 17 11:18:34 2016

Copyright (c) 1982, 2013, Oracle.  All rights reserved.

Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options

SQL> alter diskgroup DBFS_DG add disk 'o/*/DBFS_DG*dm01cel05'  rebalance power 32 NOWAIT;

Diskgroup altered.

Add the DATA and RECO grid disks to the existing DATA and RECO disk groups

  • From one DB node add the new DATA disks to the DATA disk group (replace the pattern below " 'o/*/DATA_DM01*new_cel01', 'o/*/DATA_DM01*new_cel02', 'o/*/DATA_DM01*new_cel03'" with list of actual new cells):

SQL> alter diskgroup DATA add disk 'o/*/DATA*dm01cel05' rebalance power 32 NOWAIT;

Diskgroup altered.

  •  From another DB node add the new RECO disks to the RECO disk group (replace the pattern below, " 'o/*/RECO_DM01*new_cel01','o/*/RECO_DM01*new_cel02','o/*/RECO_DM01*new_cel03' " with list of actual new cells):

SQL> alter diskgroup RECO add disk 'o/*/RECO*dm01cel05' rebalance power 32 NOWAIT;

Diskgroup altered.

  • Monitor the progress of the rebalance activities using this query (you can proceed with the rest of the steps while this is continuing):

SQL> select * from gv$asm_operation;

   INST_ID GROUP_NUMBER OPERA STAT      POWER     ACTUAL      SOFAR   EST_WORK   EST_RATE EST_MINUTES ERROR_CODE
---------- ------------ ----- ---- ---------- ---------- ---------- ---------- ---------- ----------- --------------------------------------------
         7            1 REBAL WAIT         32
         3            1 REBAL WAIT         32
         2            1 REBAL WAIT         32
         1            1 REBAL WAIT         32
         6            1 REBAL WAIT         32
         8            1 REBAL WAIT         32
         4            1 REBAL WAIT         32
         5            1 REBAL RUN          32         32      41666      45014      17263           0

8 rows selected.

SQL> select * from gv$asm_operation;

no rows selected

Validate that DATA, RECO, and DBFS_DG contain disks from new cells

Before:
dm01db01-orcldb1 {/home/oracle}:. oraenv
ORACLE_SID = [orcldb1] ? +ASM1
The Oracle base remains unchanged with value /u01/app/oracle
dm01db01-+ASM1 {/home/oracle}:sqlplus / as sysasm

SQL*Plus: Release 11.2.0.4.0 Production on Mon Oct 17 10:43:16 2016

Copyright (c) 1982, 2013, Oracle.  All rights reserved.

Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options

SQL> set lines 200
SQL> set pages 200
SQL> select d.failgroup,dg.name, sum(d.total_mb) cell_total_mb, count(1) cell_disk_count from v$asm_disk d, v$asm_diskgroup dg where d.group_number = dg.group_number group by d.failgroup, dg.name order by d.failgroup, dg.name;

FAILGROUP                      NAME                           CELL_TOTAL_MB CELL_DISK_COUNT
------------------------------ ------------------------------ ------------- ---------------
DM01CEL01                      DATA                                 3194880              12
DM01CEL01                      RECO                                 3301632              12
DM01CEL01                      DBFS_DG                              298240              10
DM01CEL02                      DATA                                 2928640              12
DM01CEL02                      RECO                                 3026496              12
DM01CEL02                      DBFS_DG                              268416              10
DM01CEL03                      DATA                                 3194880              12
DM01CEL03                      RECO                                 3301632              12
DM01CEL03                      DBFS_DG                              298240              10
DM01CEL04                      DATA                                 3194880              12
DM01CEL04                      RECO                                 3301632              12
DM01CEL04                      DBFS_DG                              298240              10
DM01CEL06                      DATA                                 3194880              12
DM01CEL06                      RECO                                 3301632              12
DM01CEL06                      DBFS_DG                              298240              10
DM01CEL07                      DATA                                 2928640              11
DM01CEL07                      RECO                                 3026496              11
DM01CEL07                      DBFS_DG                              268416              10
DM01CEL08                      DATA                                 3194880              12
DM01CEL08                      RECO                                 3301632              12
DM01CEL08                      DBFS_DG                              298240              10
DM01CEL09                      DATA                                 2396160               12
DM01CEL09                      RECO                                 2476224               12
DM01CEL09                      DBFS_DG                              238592               10
DM01CEL10                      DATA                                 3194880              12
DM01CEL10                      RECO                                 3301632              12
DM01CEL10                      DBFS_DG                              298240              10
DM01CEL11                      DATA                                 2928640              12
DM01CEL11                      RECO                                 3026496              12
DM01CEL11                      DBFS_DG                              298240              10
DM01CEL12                      DATA                                 3194880              12
DM01CEL12                      RECO                                 3301632              12
DM01CEL12                      DBFS_DG                              298240              10
DM01CEL13                      DATA                                 3194880              12
DM01CEL13                      RECO                                 3301632              12
DM01CEL13                      DBFS_DG                              298240              10
DM01CEL14                      DATA                                 3194880              12
DM01CEL14                      RECO                                 3301632              12
DM01CEL14                      DBFS_DG                              298240              10

39 rows selected.

After
SQL> set lines 200
SQL> set pages 200
SQL> select d.failgroup,dg.name, sum(d.total_mb) cell_total_mb, count(1) cell_disk_count from v$asm_disk d, v$asm_diskgroup dg where d.group_number = dg.group_number group by d.failgroup, dg.name order by d.failgroup, dg.name;

FAILGROUP                      NAME                           CELL_TOTAL_MB CELL_DISK_COUNT
------------------------------ ------------------------------ ------------- ---------------
DM01CEL01                      DATA                                 3194880              12
DM01CEL01                      RECO                                 3301632              12
DM01CEL01                      DBFS_DG                              298240              10
DM01CEL02                      DATA                                 2928640              12
DM01CEL02                      RECO                                 3026496              12
DM01CEL02                      DBFS_DG                              268416               10
DM01CEL03                      DATA                                 3194880              12
DM01CEL03                      RECO                                 3301632              12
DM01CEL03                      DBFS_DG                              298240              10
DM01CEL04                      DATA                                 2928640              12
DM01CEL04                      RECO                                 3026496              12
DM01CEL04                      DBFS_DG                              268416               10
DM01CEL05                      DATA                                 3194880              12
DM01CEL05                      RECO                                 3301632              12
DM01CEL05                      DBFS_DG                              298240              10
DM01CEL06                      DATA                                 3194880              12
DM01CEL06                      RECO                                 3301632              12
DM01CEL06                      DBFS_DG                              298240              10
DM01CEL07                      DATA                                 2928640              12
DM01CEL07                      RECO                                 3026496              12
DM01CEL07                      DBFS_DG                              268416               10
DM01CEL08                      DATA                                 3194880              12
DM01CEL08                      RECO                                 3301632              12
DM01CEL08                      DBFS_DG                              298240              10
DM01CEL09                      DATA                                 2396160               12
DM01CEL09                      RECO                                 2476224               12
DM01CEL09                      DBFS_DG                              238592               10
DM01CEL10                      DATA                                 3194880              12
DM01CEL10                      RECO                                 3301632              12
DM01CEL10                      DBFS_DG                              298240              10
DM01CEL11                      DATA                                 2928640              12
DM01CEL11                      RECO                                 3026496              12
DM01CEL11                      DBFS_DG                              298240              10
DM01CEL12                      DATA                                 3194880              12
DM01CEL12                      RECO                                 3301632              12
DM01CEL12                      DBFS_DG                              298240              10
DM01CEL13                      DATA                                 3194880              12
DM01CEL13                      RECO                                 3301632              12
DM01CEL13                      DBFS_DG                              298240              10
DM01CEL14                      DATA                                 3194880              12
DM01CEL14                      RECO                                 3301632              12
DM01CEL14                      DBFS_DG                              298240              10

42 rows selected.

Ensure that there are no offline disks in any disk group:

SQL> select name,total_mb,free_mb,offline_disks from v$asm_diskgroup;

NAME                             TOTAL_MB    FREE_MB OFFLINE_DISKS
------------------------------ ---------- ---------- -------------
DATA                             42864640   41713708             0
RECO                             44296896   44293768             0
DBFS_DG                          4026240    4023232             0

Conclusion
In this article we have learned how to add a storage cell to an existing Exadata Database machine of same disk size.

No comments:

Post a Comment

Comparing Oracle Database Appliance X8-2 Model Family

September 2019 Oracle announced Oracle Database Appliance X8-2 (Small, Medium and HA). ODA X8-2 comes with more computing resources com...