Colocation to Virtualization

The following is a walk through of installing EMC PowerPath software on RedHat based Linux hosts (CentOS/Fedora). This is required to fully utilize multiple paths to EMC SANs. The test server used here is a Cisco UCS B250-M1 blade running FCOE over 10gb Ethernet. The configuration steps work for ISCSI, Fiber Channel, and FCOE connectivity to Clariion systems.

First, copy the RPM installation package over to the server. Below shows the package to be installed.

[root@test_server01 user01]# ll
total 7036
-rw-r--r-- 1 user01 user01 7191661 Apr 27 09:24 EMCpower.LINUX-5.3.1.00.00-111.rhel5.x86_64.rpm

Install the package via “rpm -i”.

[root@test_server01 user01]# rpm -i EMCpower.LINUX-5.3.1.00.00-111.rhel5.x86_64.rpm
All trademarks used herein are the property of their respective owners.
NOTE:License registration is not required to manage the CLARiiON AX series array.

Before powerpath can be used, a license key must be installed.

[root@test_server01 user01]# emcpreg -list
unable to open license key file: No such file or directory

Overview of the “emcpreg -add” syntax.

[root@test_server01 user01]# emcpreg -add
Missing option parameter.
Usage:
    emcpreg [opts] -add key [key ...]
    emcpreg [opts] -remove key [key ...]
    emcpreg [opts] -check key [key ...]
    emcpreg [opts] -list
    emcpreg [opts] -edit
    emcpreg [opts] -install
Options:
    -f file     license file

Now we add the license key to powerpath. The following key is fake! You must obtain yours from EMC.

[root@test_server01 user01]# emcpreg -add AGE4-DFD3-89842-DSAF-JIJ0-WKG50
1 key(s) successfully added.

Make sure the license was installed correctly.
[root@test_server01 user01]# emcpreg -list

Key AGE4-DFD3-89842-DSAF-JIJ0-WKG50
  Product: PowerPath
  Capabilities: All
[root@test_server01 user01]#

Next, start the Power Path service.

[root@test_server01 user01]# /etc/init.d/PowerPath start
Starting PowerPath:  done

Display the current paths to storage via “powermt”. Since this server is booting from SAN and just being installed, there is currently only one path to storage.

[root@test_server01 ~]# powermt display dev=all
Pseudo name=emcpowera
CLARiiON ID=AXE00515480482 [test_server01_ucs]
Logical device ID=15618646804648SDSDFW84FW4894949 [test_server01_ucs_boot]
state=alive; policy=CLAROpt; priority=0; queued-IOs=0
Owner: default=Unknown, current=SP A    Array failover mode: 1
==============================================================================
---------------- Host ---------------   - Stor -   -- I/O Path -  -- Stats ---
###  HW Path                I/O Paths    Interf.   Mode    State  Q-IOs Errors
==============================================================================
   0 fnic                      sda       SP A0     active  alive      0      0

Now that powerpath is installed, we need to edit fstab to boot off of the Power Path device.

Origional fstab using labels for “/boot”.

/dev/lvm/root           /                       ext3    defaults        1 1
/dev/lvm/usr            /usr                    ext3    defaults        1 2
/dev/lvm/app            /app                    ext3    defaults        1 2
/dev/lvm/home           /home                   ext3    defaults        1 2
/dev/lvm/var            /var                    ext3    defaults        1 2
/dev/lvm/vartmp         /var/tmp                ext3    defaults        1 2
/dev/lvm/UsrLocal       /usr/local              ext3    defaults        1 2
LABEL=/boot             /boot                   ext3    defaults        1 2
tmpfs                   /dev/shm                tmpfs   defaults        0 0
devpts                  /dev/pts                devpts  gid=5,mode=620  0 0
sysfs                   /sys                    sysfs   defaults        0 0
proc                    /proc                   proc    defaults        0 0
LABEL=SWAP-sda3         swap                    swap    defaults        0 0

Below is the edited fstab with “LABEL=/boot” commented out and /boot changed to use “/dev/emcpowera1″

[root@test_server01 ~]# vi /etc/fstab
/dev/lvm/root           /                       ext3    defaults        1 1
/dev/lvm/usr            /usr                    ext3    defaults        1 2
/dev/lvm/app            /app                    ext3    defaults        1 2
/dev/lvm/home           /home                   ext3    defaults        1 2
/dev/lvm/var            /var                    ext3    defaults        1 2
/dev/lvm/vartmp         /var/tmp                ext3    defaults        1 2
/dev/lvm/UsrLocal       /usr/local              ext3    defaults        1 2
/dev/emcpowera1         /boot                   ext3    defaults        0 0
#LABEL=/boot             /boot                   ext3    defaults        1 2
tmpfs                   /dev/shm                tmpfs   defaults        0 0
devpts                  /dev/pts                devpts  gid=5,mode=620  0 0
sysfs                   /sys                    sysfs   defaults        0 0
proc                    /proc                   proc    defaults        0 0
LABEL=SWAP-sda3         swap                    swap    defaults        0 0

Filesystem options were changed to “0 0″ on emcpowera due to RedHat trying to do filesystem scans before the Power Path driver is started.

All paths now need to be zoned in the fiber switch, initiators registered in Navisphere, and paths added to the host in it’s storage group. This will not be covered here.

After zoning both paths on one switch, “powermt” now shows a path to both Clariion SPA and SPB. If not, try either stopping and restartaring “/etc/init.d/PowerPath” or restarting the server.

[root@test_server01 ~]# powermt display dev=all
Pseudo name=emcpowera
CLARiiON ID=AXE00515480482 [test_server01_ucs]
Logical device ID=15618646804648SDSDFW84FW4894949 [test_server01_ucs_boot]
state=alive; policy=CLAROpt; priority=0; queued-IOs=0
Owner: default=SP B, current=SP A       Array failover mode: 1
==============================================================================
---------------- Host ---------------   - Stor -   -- I/O Path -  -- Stats ---
###  HW Path                I/O Paths    Interf.   Mode    State  Q-IOs Errors
==============================================================================
   0 fnic                      sdc       SP B1     active  alive      0      0
   0 fnic                      sdd       SP A0     active  alive      0      0

Configuration has now been completed on fiber switch 2 and both SPA and SPB in the Clariion. Reboot the server. Again, “powermt” is used to list the paths.

[root@test_server01 ~]# powermt display dev=all
Pseudo name=emcpowera
CLARiiON ID=AXE00515480482 [test_server01_ucs]
Logical device ID=15618646804648SDSDFW84FW4894949 [test_server01_ucs_boot]
state=alive; policy=CLAROpt; priority=0; queued-IOs=0
Owner: default=SP B, current=SP B       Array failover mode: 1
==============================================================================
---------------- Host ---------------   - Stor -   -- I/O Path -  -- Stats ---
###  HW Path                I/O Paths    Interf.   Mode    State  Q-IOs Errors
==============================================================================
   0 fnic                      sdc       SP B1     active  alive      0      0
   0 fnic                      sdd       SP A0     active  alive      0      0
   1 fnic                      sde       SP B0     active  alive      0      0
   1 fnic                      sdf       SP A1     active  alive      0      0

From above, you can see that we now have 4 paths definied. Both fnic interfaces can see SPA and SPB. Each fnic is attached to a seperage fiber switch, so we have redundant paths to both Clariion heads (SP’s). Once rebooted, the server should load fine with no issues and see all paths via powermt.

Notes:  “/boot” is the storage label used in this example.  If your mount point is different, modify it’s entry instead.  “/dev/emcpowera1″ is used since there is only one LUN mapped to this host.  Like anything else, if there are more than one, each would have it’s own device.

~ by Kevin Goodman on May 4, 2010.

Posted in EMC, Filesystems, Linux, SAN (Storage Area Network)
Tags: boot from san, centos, EMCpower, EMCpower.LINUX, emcpowera1, emcpreg, fedora, fnic, key, license, LinkedIn, Linux, path, paths, Power Path, powermt, powerpath, redhat, san, storage