Saturday, June 10, 2017

Migrating from ASMLib to Oracle ASM Filter Driver (ASMFD)

Starting 12.1.0.2, ASMFD can be used instead of ASMLib for the ASM disks management. In this article I will explain how to migrate from ASMLib to ASMFD. I already have ASMLib configured on my server and I want to move to using ASMFD. For new GI installation, we can configure ASMFD for our ASM disks from GI installation wizard. Method explained here works for both standalone GI installations as well as for RAC.

As root user, get list of all current disks being used by ASM.
[root@salman11 ~]# oracleasm scandisks
Reloading disk partitions: done
Cleaning any stale ASM disks...
Scanning system for ASM disks...

[root@salman11 ~]# oracleasm listdisks
DATA
[root@salman11 ~]# oracleasm querydisk -p DATA
Disk "DATA" is a valid ASM disk
/dev/sdb1: LABEL="DATA" TYPE="oracleasm"

So I have only one disk DATA (/dev/sdb1) right now that I will migrate to ASMFD.
Next step is to check the state of ASMFD, then we will reset the DISKSTRING parameter by using asmcmd command. As grid user, execute following set of commands. Please note that for my ASM disks I am using discovery path as ‘ORCL*’, in your case, you might be using full disk path (/dev/sd*) instead.
For RAC, perform this on all nodes.
[grid@salman11 bin]$ export ORACLE_HOME=/u01/app/12.2.0/grid/
[grid@salman11 bin]$ export PATH=$ORACLE_HOME/bin:$PATH
[grid@salman11 bin]$ export ORACLE_SID=+ASM

-- Check state of ASMFD
[grid@salman11 bin]$ asmcmd afd_state
ASMCMD-9530: The AFD state is 'NOT SUPPORTED'

-- Check current diskstring parameter setting. For me, diskstring set is ORCL:*. For other cases, it may be something like “/dev/oracleasm/disks/*”.
 [grid@salman11 bin]$ asmcmd dsget
parameter:ORCL:*
profile:ORCL:*

-- Add ‘AFD:*’ to the diskstring. It should contain old diskstring path “ORCL:*” (or if your disksting is /dev/oracleasm/disks/*), and also new disk string path for ASMFD which is “AFD:*”
[grid@salman11 bin]$ asmcmd dsset 'ORCL:*','AFD:*'
[grid@salman11 bin]$ asmcmd dsget
parameter:ORCL:*,  AFD:*
profile:ORCL:*,AFD:*

-- Above command will also set disk_String parameter in ASM instance to point to current disk discovery path and also new AFD path.
[grid@salman11 bin]$ sqlplus

SQL*Plus: Release 12.2.0.1.0 Production on Tue Mar 21 13:09:50 2017

Copyright (c) 1982, 2016, Oracle.  All rights reserved.

Enter user-name: sys as sysasm
Enter password:

Connected to:
Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production

SQL> show parameter string

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
asm_diskstring                       string      ORCL:*, AFD:*


As root, stop the “has” services or CRS.
[root@salman11 ~]# /u01/app/12.2.0/grid/bin/crsctl stop has

-- For RAC, use following command to stop CRS. Execute this on all nodes
[root@salman11 ~]# /u01/app/12.2.0/grid/bin/crsctl stop crs

Stop oracleasm driver and acfsload. Execute following as root (on all nodes in case of RAC)
[root@salman11 ~]# oracleasm exit
Unmounting ASMlib driver filesystem: /dev/oracleasm
Unloading module "oracleasm": oracleasm

[root@salman11 ~]# /u01/app/12.2.0/grid/bin/acfsload stop

As root, configure ASMFD. This will automatically de-configure ASMLib also. For RAC, perform this on one node and once it completes, check the state of ASMFD on all nodes. If it is not configured on all nodes, then execute this step on all nodes.
[root@salman11 ~]# /u01/app/12.2.0/grid/bin/asmcmd afd_configure
AFD-627: AFD distribution files found.
AFD-634: Removing previous AFD installation.
AFD-635: Previous AFD components successfully removed.
AFD-636: Installing requested AFD software.
AFD-637: Loading installed AFD drivers.
AFD-9321: Creating udev for AFD.
AFD-9323: Creating module dependencies - this may take some time.
AFD-9154: Loading 'oracleafd.ko' driver.
AFD-649: Verifying AFD devices.
AFD-9156: Detecting control device '/dev/oracleafd/admin'.
AFD-638: AFD installation correctness verified.
Modifying resource dependencies - this may take some time.

-- Check the status of ASMFD (on all nodes)
[root@salman11 ~]# /u01/app/12.2.0/grid/bin/asmcmd afd_state
ASMCMD-9526: The AFD state is 'LOADED' and filtering is 'ENABLED' on host 'salman11.salman.com'

Only if you are using 12.1.0.2, as root user, migrate all your disk(s) to ASMFD. Perform following step for this.
[root@salman11 ~]# /u01/app/12.2.0/grid/bin/asmcmd afd_label DATA /dev/sdb1 --migrate

For versions above 12.1.0.2, if you execute this step, you will see an error as bellow because this migration is automatically performed during ASMFD driver configuration and above step for migration is not needed.
[root@salman11 ~]# /u01/app/12.2.0/grid/bin/asmcmd afd_label DATA /dev/sdb1 --migrate

label name DATA is already in use
ASMCMD-9513: ASM disk label set operation failed.


List the disks to confirm if disk(s) have been migrated successfully. Disk(s) will also be listed under /dev/oraclefd/disk directory (If you can recall, for ASMLib; disks were listed under /dev/oracleasm/disks)

[root@salman11 ~]# /u01/app/12.2.0/grid/bin/asmcmd afd_lsdsk
--------------------------------------------------------------------------------
Label                     Filtering   Path
====================================================================
DATA                        ENABLED   /dev/sdb1

[root@salman11 ~]# ls -l /dev/oracleafd/disks/
total 4
-rwxrwx---. 1 grid oinstall 10 Mar 22 16:30 DATA
[root@salman11 ~]#

As root user, start asfs first, and then “has” for standalone GI installations.  Start “CRS” for RAC installations (On all nodes)
[root@salman11 ~]# /u01/app/12.2.0/grid/bin/acfsload start
ACFS-9391: Checking for existing ADVM/ACFS installation.
ACFS-9392: Validating ADVM/ACFS installation files for operating system.
ACFS-9393: Verifying ASM Administrator setup.
ACFS-9308: Loading installed ADVM/ACFS drivers.
ACFS-9154: Loading 'oracleoks.ko' driver.
ACFS-9154: Loading 'oracleadvm.ko' driver.
ACFS-9154: Loading 'oracleacfs.ko' driver.
ACFS-9327: Verifying ADVM/ACFS devices.
ACFS-9156: Detecting control device '/dev/asm/.asm_ctl_spec'.
ACFS-9156: Detecting control device '/dev/ofsctl'.
ACFS-9322: completed


[root@salman11 ~]# /u01/app/12.2.0/grid/bin/crsctl start has

-- For RAC
/u01/app/12.2.0/grid/bin/crsctl start crs

It is better to deinstall oracleasm (Form all nodes in case of RAC)
[root@salman11 ~]# rpm -q oracleasm-support
oracleasm-support-2.1.8-3.1.el7.x86_64
[root@salman11 ~]# rpm -e oracleasm-support-2.1.8-3.1.el7.x86_64
warning: /etc/sysconfig/oracleasm saved as /etc/sysconfig/oracleasm.rpmsave

Since ASMFD will be used now, we can set the diskstring to use only ‘AFD:*’, and remove ‘ORCL:*’ from diskstring.
As grid user, execute following command
[grid@salman11 ~]$ asmcmd dsget
parameter:ORCL:*, AFD:*
profile:ORCL:*,AFD:*

[grid@salman11 ~]$ asmcmd dsset 'AFD:*'

[grid@salman11 ~]$ asmcmd dsget
parameter:AFD:*
profile:AFD:*



No comments: