Once we got over the fear of FCoE reliability, almost everything is now being ordered with 10gbE. The only reason of worry was during this implementation over a year ago, FCoE was not in the main stream yet. It was almost “bleeding edge” in the market.

Where are we a year later?

One of the big killer of ports in our environment is having to tie the Gigabit Ethernet switches into them. This is wasting 4 ports that could be 10gb pre switch. Hopefully that will be resolved with a future purchase of Nexus 7000 switches.  We also have a pair of fabric extenders connected into the 5010s.  This also takes up two 10gbE ports per 5010.

Below is a port listing from one of our Nexus 5010 switches. This is after we removed 3 ports (per switch) from physical servers that were tied into the 10gbE infrastructure. As you can see from below, we only have 5 ports per switch left (10 total if you count the redundant switch)! That is only 5 more devices at the most.

--------------------------------------------------------------------------------
Ethernet      VLAN   Type Mode   Status  Reason                   Speed     Port
Interface                                                                   Ch #
--------------------------------------------------------------------------------
Eth1/1        x     eth  trunk  up      none                       1000(D) 1
Eth1/2        x     eth  trunk  up      none                       1000(D) 1
Eth1/3        x     eth  trunk  up      none                       1000(D) 1
Eth1/4        x     eth  trunk  up      none                       1000(D) 1
Eth1/5        x     eth  trunk  up      none                        10G(D) --
Eth1/6        x     eth  trunk  up      none                        10G(D) --
Eth1/7        x     eth  access up      none                        10G(D) --
Eth1/8        x     eth  access up      none                        10G(D) --
Eth1/9        x     eth  trunk  down    Link not connected          10G(D) --
Eth1/10       x     eth  trunk  down    Link not connected          10G(D) --
Eth1/11       x     eth  trunk  down    Link not connected          10G(D) --
Eth1/12       x     eth  access down    SFP not inserted            10G(D) --
Eth1/13       x     eth  access up      none                        10G(D) --
Eth1/14       x     eth  access down    SFP not inserted            10G(D) --
Eth1/15       x     eth  trunk  up      none                        10G(D) --
Eth1/16       x     eth  trunk  up      none                        10G(D) --
Eth1/17       x     eth  fabric up      none                        10G(D) --
Eth1/18       x     eth  fabric up      none                        10G(D) --
Eth1/19       x     eth  trunk  up      none                        10G(D) 2
Eth1/20       x     eth  trunk  up      none                        10G(D) 2

At the 6120′s end (UCS switches) we are fine. Currently have 14 ports on each switch available. That should give us the ability to wire in 7 more chassis with 2 – 10gbE uplinks per IOM. A total of 4 links / 40gb throughput per chassis. Our CIFS and NFS access is a lot faster running through the EMC NX4′s 10gbE.

So if you are looking into 10gbE for our network, it would help to have at least a rough draft of where you want your datacenter to be a few years from now. Pretty much all major storage companies are doing some sort of 10gbE connectivity. Using it makes administration a lot easier. FCoE simplifies things even more, including eliminating the need for separate fiber switches. To sum it up, I love 10gbE and would recommend it even if not planning on incorporating FCoE or UCS in your environment.

DD 670 Front

We received our new DataDomain 670 last week and it has definitely had a face lift from the 500 series.  First, it is a 2u versus a 3u unit.  Also, the 600 series no longer have the annoying third power supply.  The drives are also horizontal instead of vertical.  Even slots in the back are different.  I wish we had a multi-shelf unit one.  Below are a few pictures from the unpacking and racking.

Notes: Only complaint is the rails.  I really wish they rails on the server were pegged style – drop into extended rack rail types (IBM).  Instead, the rails on the servers are fixed and have to be lined up perfectly with the ones in the rack.  They then slide straight into the rack rails and lock.  I find it a lot hard with this kind of rails to rack heavy equipment alone.

Below are a few videos of the equipment.  Be careful with the sound!  These were taken in the datacenter and you will hear the typical rumbling of servers and AC units.

Video of the NetApp FAS 3020 once removed

Video of the EMC NX4 waiting to go in

DD01# filesys show compression /backup/db/xxx/xxx/xx6e_1_1
Total files: 1;  bytes/storage_used: 59.1
       Original Bytes:       16,252,249,684
  Globally Compressed:          954,839,915
   Locally Compressed:          272,055,847
            Meta-data:            3,009,088

bytes/storage_used: 59.1 =	Compression and Deduplication ratio
Globally Compressed Factor = 	Pre-Comp / (Size after de-dupe)
Locally Compressed Factor = 	(Size after de-dupe) / Post-Comp

From above you should be getting an idea of the deduplication, but lets follow DataDomain’s formula

Reduction % = ((Pre-Comp – Post-Comp) / Pre-Comp) * 100

Now put in values from our above example

16,252,249,684 – 272,055,847 / 16,252,249,684 * 100 = 98.326%

That gives us a 98.326% reduction!

Basically what I am trying to say is that if you are not doing some sort of deduplication + compression on your Oracle backups, you should look into it.

Note(s): The above information is from a system running Data Domain OS 4.8.1.0-175623.  Also, remember that deduplication savings grow as the same types of blocks are written.  This system has been in production for over a year.  We have about 1tb raw of Oracle data being written to the DataDomain which helps the dedupe ratio.

Now all that we need to do is get some data over to it.  We have a matching unit a few racks down that will be moved to our Tier 2 site once replication has been completed.

Notes: One thing to note is that the rails are an all in one unit.  You can not change the order in which each device is racked.  Also, the controllers (SPs) are built into the drive tray.  Best part is that this will be wired up via 10gbE connectivity over fiber.

On the source, I have already setup a directory tree for “/backup/europe_data”. All files destined for our Euroupe office will be placed here. On the DataDomain devices, replicated folders are added manually. By default none are replicated.
*********************
Be aware that if you are viewing this through a web browser that a lot of the command might be hidden due to formatting of the WordPress theme.  It is best viewed by an RSS reader.  If you would like the full text, email or leave a comment and I can send you a PDF version.
*********************

Below is the help output for “replication add” command.

user01@us-dd565# replication add  ?
Commands matching "replication add":
  replication add source  destination
                 Add a replication pair

I wish the syntax was easy as what was stated above, but below is the actual command used to setup replication on the source DataDomain

user01@us-dd565# replication add source dir://us-dd565.testdomain.com/backup/europe_data destination dir://eu-dd01.testdomain.com/backup/europe_data

Once the source is setup, the destination endpoint/DataDomain also must be configured. The command is identical to what was entered at the source DataDomain

user01@eu-dd01# replication add source dir://us-dd565.testdomain.com/backup/europe_data destination dir://eu-dd01.testdomain.com/backup/europe_data

Once replication is setup on both DataDomains, log back into the source and start the replication.

user01@us-dd565# replication initialize
Incorrect syntax "replication initialize", showing help:
  replication initialize
                 Initialize replication on the source
                 (configure both source and destination
                 first)

Below is the actual syntax to startup replication for the directory that we just configured.

user01@us-dd565# replication initialize dir://eu-dd01.testdomain.com/backup/europe_data
(00:04) Waiting for initialize to start...
(00:06) Initialize started.
Use 'replication watch dir://eu-dd01.testdomain.com/backup/europe_data' to monitor progress.

Since I am impatient and want to know for sure this is working, I use “replication watch”. This shows the current progress of the replication between DataDomains.

user01@us-dd565# replication watch dir://eu-dd01.testdomain.com/backup/europe_data
    Use Control-C to stop monitoring.

(00:57) Replication initialize started...
(00:59) initializing 3/3:
(10:35)     : pre-comp:   0%    27 KB/s, network    26 KB/s, 0 of 6 files

When new data is written to the mount point, replication automatically starts. Below is what shows if no data is currently being replicated.

user01@us-dd565# replication watch dir://eu-dd01.testdomain.com/backup/europe_data
**** Replication initialize or recover already completed.

“replication show config” is used to display all the configured replication points.

user01@us-dd565# replication show config
CTX   Source                                                    Destination                                                 Connection Host and Port                       Enabled
---   -------------------------------------------------------   ---------------------------------------------------------   --------------------------------------------   -------
11    dir://us-dd565.testdomain.com/backup/mysql-rep       	dir://us-t2-dd510.testdomain.com/backup/mysql-rep	us-t2-dd510.testdomain.com   (default)     yes
12    dir://us-dd565.testdomain.com/backup/europe_data        	dir://eu-dd510.testdomain.com/backup/europe_data	eu-dd510.testdomain.com   (default)     yes
---   -------------------------------------------------------   ---------------------------------------------------------   --------------------------------------------   -------

Sorry for all of the spacing being off. All the tabs did not come over when copying data from the console.

user01

DataDomain Web Alert

SSH to the DataDomain and add my Linux systems IP address

admin1@DataDomain01# nfs add /ddvar 172.16.100.6
Mount the DataDomain /ddvar export on the Linux system
root@tstbox02:~# mount DataDomain01:/ddvar /mnt/DataDomain01/

Move into the core sub-directory under the ddvar mount point

root@tstbox02:~# cd /mnt/DataDomain01/core/

Check the directory for files. As you can see below, the core directory contained 47 gig worth of data

root@tstbox02:/mnt/DataDomain01/core# ls -lah
total 47G
drwxrwsr-x  2 root group 4.0K 2009-01-14 22:43 .
drwxr-xr-x 12 root group 4.0K 2009-04-02 10:28 ..
-rw-r--r--  1 root group 2.2M 2008-04-10 14:23 cmdb.core.26956.1207851782.gz
-rw-r--r--  1 root group 3.8M 2008-04-10 15:06 core.13124
-rw-r--r--  1 root group 121K 2008-04-08 07:01 ddfs.core.14547.1207652507.gz
-rw-------  1 root group 352K 2008-04-07 22:56 ddfs.core.15098.1207618361.gz
-rw-r--r--  1 root group 167M 2008-04-09 09:30 ddfs.core.15100.1207747799.gz
-rw-r--r--  1 root group 167M 2008-04-08 10:37 ddfs.core.15125.1207665449.gz
-rw-r--r--  1 root group 167M 2008-04-08 10:38 ddfs.core.15505.1207665497.gz
-rw-r--r--  1 root group 167M 2008-04-09 09:31 ddfs.core.15506.1207747848.gz
-rw-r--r--  1 root group 527M 2008-09-10 16:27 ddfs.core.1560.1221078099.gz
-rw-r--r--  1 root group 166M 2008-04-08 07:32 ddfs.core.15675.1207654304.gz
-rw-r--r--  1 root group 167M 2008-04-08 10:39 ddfs.core.15795.1207665554.gz
-rw-r--r--  1 root group 167M 2008-04-09 09:32 ddfs.core.15813.1207747905.gz
-rw-r--r--  1 root group 636M 2008-07-28 15:20 ddfs.core.15991.1217272516.gz
-rw-r--r--  1 root group 167M 2008-04-08 10:40 ddfs.core.16055.1207665605.gz
-rw-r--r--  1 root group 167M 2008-04-09 09:33 ddfs.core.16095.1207747957.gz
-rw-r--r--  1 root group 167M 2008-04-08 07:32 ddfs.core.18047.1207654349.gz
-rw-r--r--  1 root group 167M 2008-04-08 07:33 ddfs.core.18231.1207654404.gz
-rw-r--r--  1 root group 167M 2008-04-08 07:34 ddfs.core.18591.1207654458.gz
-rw-r--r--  1 root group 548M 2008-09-20 02:56 ddfs.core.22290.1221893385.gz
-rw-r--r--  1 root group 167M 2008-04-09 10:23 ddfs.core.22705.1207750958.gz
-rw-r--r--  1 root group 167M 2008-04-09 10:23 ddfs.core.22875.1207751006.gz
-rw-r--r--  1 root group 167M 2008-04-09 10:24 ddfs.core.23008.1207751055.gz
-rw-r--r--  1 root group 553M 2008-08-19 15:26 ddfs.core.23183.1219173638.gz
-rw-r--r--  1 root group 167M 2008-04-09 10:25 ddfs.core.23269.1207751113.gz
-rw-r--r--  1 root group 167M 2008-04-09 09:06 ddfs.core.28347.1207746385.gz
-rw-r--r--  1 root group 167M 2008-04-09 09:07 ddfs.core.28507.1207746432.gz
-rw-r--r--  1 root group 167M 2008-04-09 09:08 ddfs.core.28694.1207746490.gz
-rw-r--r--  1 root group 167M 2008-04-09 09:09 ddfs.core.28989.1207746540.gz
-rw-r--r--  1 root group 167M 2008-04-09 09:14 ddfs.core.29883.1207746837.gz
-rw-r--r--  1 root group 213M 2008-04-07 22:05 ddfs.gdb.15098.1207618361

Since I verified that we do not need the archived (.gz) files, they were removed

root@tstbox02:/mnt/DataDomain01/core# rm *.gz

By only retaining the newer files in this directory, the size is now down to 1.6 gigs.

root@tstbox02:/mnt/DataDomain01/core# ls -lah
total 1.6G
drwxrwsr-x  2 root group 4.0K 2009-04-15 14:29 .
drwxr-xr-x 12 root group 4.0K 2009-04-02 10:28 ..
-rw-r--r--  1 root group 3.8M 2008-04-10 15:06 core.13124
-rw-r--r--  1 root group 213M 2008-04-07 22:05 ddfs.gdb.15098.1207618361

Once this was completed, the alert in this DataDomain cleared in the web interface.

Data Domain Web Alert Cleared

Notes: Always verify files that are being removed are not needed. If there are core files present, it generally means the system has hit an error. Verify with your DataDomain support before modifying the ddvar filesystem.

So here is the really bad part. The inodes on ext2 and ext3 (Linux default type) are statically set when the file system is formatted. You can not go back and change the max inode settings. The exceptions to this that I know of are as follows:

- Reiser4
- VxFS
- XFS
- JFS
- WAFL (NetApp proprietary)
- XZFS

If you are running one of the above and have max inodes issue, you can correct it.

I have been working with computers for over 15 years and have only ran into this problem once. Luckily, it occurred on a NetApp NAS device that had the ability to increase this value on the live file system. The main killer here are tons of small files. In this case, the file system for that NFS share was 40 gigabytes in size and default was ~1 million inode limit. The quick fix for the issue was to increase this to 3 million.

As far as a ext2 and 3 go, the following shows how to query a file system for relevant inode information

root@testbox:~# tune2fs -l /dev/sda1
tune2fs 1.41.3 (12-Oct-2008)
Filesystem volume name:   <none>
Last mounted on:          <not available>
Filesystem UUID:          56161dd8-9d1d-4c54-851d-938bb88ce6d4
Filesystem magic number:  0xEF53
Filesystem revision #:    1 (dynamic)
Filesystem features:      has_journal ext_attr resize_inode dir_index filetype needs_recovery sparse_super large_file
Filesystem flags:         signed_directory_hash
Default mount options:    (none)
Filesystem state:         clean
Errors behavior:          Continue
Filesystem OS type:       Linux
Inode count:              4685824
Block count:              18731782
Reserved block count:     936589
Free blocks:              15534374
Free inodes:              4459463
First block:              0
Block size:               4096
Fragment size:            4096
Reserved GDT blocks:      1019
Blocks per group:         32768
Fragments per group:      32768
Inodes per group:         8192
Inode blocks per group:   256
Filesystem created:       Mon Sep 29 16:25:20 2008
Last mount time:          Fri Jan 23 14:27:02 2009
Last write time:          Fri Jan 23 14:27:02 2009
Mount count:              4
Maximum mount count:      33
Last checked:             Thu Jan 15 09:00:37 2009
Check interval:           15552000 (6 months)
Next check after:         Tue Jul 14 10:00:37 2009
Reserved blocks uid:      0 (user root)
Reserved blocks gid:      0 (group root)
First inode:              11
Inode size:	          128
Journal inode:            8
First orphan inode:       2908742
Default directory hash:   tea
Directory Hash Seed:      a6544c5xxxxxxxxxxxxxxxxx
Journal backup:           inode blocks

The above is good to know, especially to check an un-mounted file system. The command below shows a friendlier formatted output

root@testbox:~# df -i
Filesystem            Inodes   IUsed   IFree IUse% Mounted on
/dev/sda1            4685824  226361 4459463    5% /
tmpfs                 222201       4  222197    1% /lib/init/rw
varrun                222201      64  222137    1% /var/run
varlock               222201       5  222196    1% /var/lock
udev                  222201    5142  217059    3% /dev
tmpfs                 222201       5  222196    1% /dev/shm
/dev/sdb1            61063168    1116 61062052    1% /media/disk

As you can see, there are no issues to be worried about on this test computer. Most systems administrators perform centralized monitoring of disk usage at a disk space level (capacity). On highly used servers that utilize locally stored and/or direct attached storage, it is a good idea to have a script check and report on the available inodes.

PDF Version

Notes: This information is provided for a high level overview concerning inodes. More in-depth information will be provided in up-coming posts.

I was outside hooking the kid bike trailer to the 12 speed when my wife came out holding my cell phone

Wife> Eric just called you twice
Me> Startup my laptop for me
... Call Eric
Eric> Sorry to bother you but  .... .

Luckily he noticed a couple of VMs dropped off the face of the earth. To be exact, we lost 4 virtual machines. When told to power on, they would error with “Can not open disk: PathToVmdk.. The parent virtual disk has been modified since the child was created“.

I never trust the GUI, so I went strait to the ESX console. Sure enough, things failed there also.

Per /var/log/vmkernel

Mar  5 23:02:59 vmsrv06 vmkernel: 28:09:39:57.271 cpu6:1215)<6>Debug scsi underrun
Mar  5 23:02:59 vmsrv06 vmkernel: 28:09:39:57.342 cpu6:1183)<6>Debug scsi underrun
Mar  5 23:02:59 vmsrv06 vmkernel: 28:09:39:57.413 cpu6:1098)<6>Debug scsi underrun
Mar  5 23:06:49 vmsrv06 vmkernel: 28:09:43:46.898 cpu15:1074)<6>qla24xx_abort_command(1): handle to abort=1497
Mar  5 23:06:49 vmsrv06 vmkernel: 28:09:43:46.898 cpu15:1074)<6>qla24xx_abort_command(1): handle to abort=1501
Mar  5 23:06:49 vmsrv06 vmkernel: 28:09:43:46.899 cpu15:1074)<6>qla24xx_abort_command(1): handle to abort=1499
Mar  5 23:06:49 vmsrv06 vmkernel: 28:09:43:46.899 cpu15:1074)<6>qla24xx_abort_command(1): handle to abort=1504
Mar  5 23:06:49 vmsrv06 vmkernel: 28:09:43:46.900 cpu15:1074)<6>qla24xx_abort_command(1): handle to abort=1496
Mar  5 23:06:49 vmsrv06 vmkernel: 28:09:43:46.901 cpu15:1074)<6>qla24xx_abort_command(1): handle to abort=1494
Mar  5 23:06:49 vmsrv06 vmkernel: 28:09:43:46.901 cpu15:1074)<6>qla24xx_abort_command(1): handle to abort=1502
Mar  5 23:10:06 vmsrv06 vmkernel: 28:09:47:04.375 cpu2:1051)WARNING: SCSI: 119: Failing I/O due to too many reservation conflicts
Mar  5 23:10:06 vmsrv06 vmkernel: 28:09:47:04.375 cpu2:1051)WARNING: FS3: 4785: Reservation error: SCSI reservation conflict

Next we will look at the NetApp. Below is from a console session. As you can see, the CPUs are hit hard.
From the NetApp

NAS>stats show system
system:system:nfs_ops:536/s
system:system:cifs_ops:0/s
system:system:http_ops:0/s
system:system:dafs_ops:0/s
system:system:fcp_ops:481/s
system:system:iscsi_ops:97/s
system:system:net_data_recv:136KB/s
system:system:net_data_sent:2323KB/s
system:system:disk_data_read:63752KB/s
system:system:disk_data_written:51640KB/s
system:system:cpu_busy:50%
system:system:avg_processor_busy:50%
system:system:total_processor_busy:100%
system:system:num_processors:2
system:system:time:1236313212s
system:system:uptime:9164385s

The conclusion is as follows: When VMware creates a snapshot, a new vmdk is created that references the new snapshot. The main vmx files will then be modified to link to the snapshot as well. When a snapshot is deleted, VMware is not just deleting a file from the file system. VMware gathers all the snapshots for that VM, collapses all their contained data into the main vmdk file, modifies the vmx files, then deletes the selected snapshots files. So in essence, when going down the list of all VMs that had snapshots(9-12) and deleting them, this caused the NetApp to overload. During this, the ESX hosts intermittently lost connectivity to the VMFS LUNS. This happened at such a fast rate, that no alerts were generated, but caused the VMs that were currently writing the snapshot to the main vmdk files to corrupt their configurations.

Moral of the story: Rapidly deleting snapshots can DOS older/slower storage systems.

From the CLI
# alerts show current
Alert Time         Description
—————-   —————————————————–
Mon Dec  8 08:38   Encl 1 A power supply module has failed.
—————-   —————————————————–
There is 1 active alert

I needed to go over to that colocation anyway so I made sure remote hands didn’t mess anything up.  So this alert is definitely hung.  The new power supply had good status indicators, and the power cable was replaced.  Removing this alert has to be done through the CLI in SE mode.  To do this, I SSHed into the Data Domain and escalated my privileges as follows.

Retrieve the serial number
# system show serialno
Serial number: XXXXXXXXXXX

Escalate to SE privileges
# priv set SE
Enter system password:[enter serial number]

Once in SE mode, the prompt will change to something resembling the following
SE@hostname##

Now, the ‘reg’ command is extended to its full feature set
SE@hostname## reg
Commands matching “reg”:

reg removekey <key>           Remove a key from the configuration
reg set reg-default <key>     Set the specified key to the default value
reg set unset-defaults        Set unset defaults.  Useful for upgrades
reg set <key> = <value>       Set a configuration key to the given value
reg setraw <key> = <value>    Set an encoded reg string
reg show defaults             Show registry defaults
reg show obsolete             Show obsolete keys
reg show stats                Show registry stats
reg showraw [nokey]           Show encoded registry data
reg show [nokey] <key>        Show a value for a configuration key

The autosupport contains the registry key that needs to be removed
dynamic.ems.701.x.1000.x.9.1.5 = Encl 1 A power supply

Cut out the part of the key that is needed
dynamic.ems.701.x.1000.x.9.1.5

The following removes the hung alert from the registry
SE@hostname## reg removekey dynamic.ems.701.x.1000.x.1.5

Note: Unfortunately, the alert does not tell which of the 3 power supplies have failed.  This was not a huge problem, since the remote hands just eyed the power supply that had no status lights on.

• ISCSI
• NFS/NAS
• VMotion

This information is provided when going through “Add Networking” under the configuration tab in ESX3.x.

VMkernel Add Networking

In ESX 3.x VMware has the ability to create a VMkernel Port on each vSwitch.  So far, we have had no luck utilizing NFS or software ISCSI through two different VMkernel Ports.  Example:

There is a NAS on both a private VLAN and the normal LAN network.  Each goes through a different vSwitch.  After adding a VMkernel Port on both vSwitches, NFS and ISCSI worked fine to the first defined connection.  After VMware first connects to a server on either subnet, in this example 172.0.1.2, the next connection to 172.0.2.1 will fail. It will fail no matter if trying to connect through the other vSwitch using the same protocol (NFS) or ISCSI.  Whichever connection is established first, VMWare seems not to be able to route through the other VMkernel Port

Dual NFS Mount Errors

Below shows a vSwitch configuration with a VMkernel Port

vSwitch With VMkernel

Note: I am unsure if this would be an issue when using an hardware ISCSI (ISCSI card). As far as NFS is concerned, I do not have a work around for this currently.  If you will need to mount NFS shares from two different sources, through different vSwitches, better find the answer to this problem first.  I hope that ESX 4 solves this issue.

nas01> route -s
Routing tables

Internet:

Example local ping
nas> ping 172.16.100.1
172.16.100.1 is alive

Pinging the other locations gateway would return with no route to host
ping 172.16.0.1

Add the default gateway in
nas> route add default 172.16.100.1 1

Access restored
nas> ping 172.16.0.1
172.16.0.1 is alive

Verify the routing table
nas01> route -s
Routing tables

Internet:

Note: So it seems that the web GUI just reads the /etc/dgateways from the filesystem.  When I destroyed the vif that acts as the path from the default gateway, it looks to have removed the gateway from the active routing table.  Normally I reboot the whole filer to be safe, but again this was to be transparent to the ISCSI data stores.  Everything worked fine after I manually re-entered the route.  Also, ‘netstat -r’ will also display the routing table.

Check to make sure the licensces are installed
NAS> license
nearstore_option xxxxxZH
a_sis xxxxxCG

Enable sis (deduplication) on the volume
NAS> sis on /vol/testVol
SIS for “/vol/testVol” is enabled.
Already existing data could be processed by running “sis start -s /vol/testVol”.

Initiate deduplication service
NAS> sis start -s /vol/testVol
The file system will be scanned to process existing data in /vol/testVol.
This operation may initialize related existing metafiles.
Are you sure you want to proceed with scan (y/n)? y

Thu Nov 13 10:01:38 EST [wafl.scan.start:info]: Starting SIS volume scan on volume testVol.
The SIS operation for “/vol/testVol” is started.

NAS> Thu Nov 13 10:01:42 EST [wafl.snap.delete:info]: Snapshot copy sis.93057f3c-aed8-11dd-a6d9-00a09804ad86 on volume testVol NetApp was deleted by the Data ONTAP function dense_delete_snapshot. The unique ID for this Snapshot copy is (44, 156).

Sis status will display the state and running status on all volumes
NAS> sis status

View the status on a specific volume
NAS> sis status /vol/testVol

After sis completes, use df to show the amount saved and deduplication percentage
NAS> df -sh /vol/testVol

Verify that sis is scheduled to run frequent enough at the times you want
NAS> sis config

Below is the help output of sis config
NAS> sis help config
sis config [ [ -s schedule ] <path> | <path> … ]
- Sets up, modifies, and retrieves the schedule of SIS volumes.

Reschedule for nightly at 11pm
NAS> sis config -s sun-sat@23 /vol/testVol

Verify the new configuration

NAS> sis config /vol/testVol

Note: ASIS is not real time deduplication. It is ran on a schedule instead of as the data is coming in. Also, when setting the schedule for a volume, you can only specifiy the hour to start running, in this case trying to specify sun-sat@23:45 did not work when testing.

Command structure utilizing the VM name as source
vcbmounter -h <virtual center server> -u <username> -p <password> -a name:<name of vm> -r <path to store the files> -t fullvm -m san

Use vcbmounter to pull down a full backup of a VM.
C:Program FilesVMwareVMware Consolidated Backup Framework>vcbmounter -h viServer.x.com -u vcb -p vcbuserPass -a name:TestVM -r Z:vcprepotestTestVM -t fullvm -m san

Copying “[NetApp] TestVM//vmware-938.log”:
0%=====================50%=====================100%
**************************************************

Copying “[NetApp] TestVM//vmware-940.log”:
0%=====================50%=====================100%
**************************************************

Copying “[NetApp] TestVM//vmware-942.log”:
0%=====================50%=====================100%
**************************************************

Copying “[NetApp] TestVM//vmware.log”:
0%=====================50%=====================100%
**************************************************

[2008-11-12 15:53:34.093 'BaseLibs' 3100 warning] SSLVerifyCertAgainstSystemStore: Subject mismatch: VMware vs viServer.x.com
[2008-11-12 15:53:41.730 'BaseLibs' 3100 warning] SSLVerifyIsEnabled: failed to read registry value. Assuming verification is disabled. LastError = 0
[2008-11-12 15:53:41.730 'BaseLibs' 3100 warning] SSLVerifyCertAgainstSystemStore: Certificate verification is disabled, so connection will proceed de
spite the error
Converting “Z:vcprepotestTestVMscsi0-0-0-TestVM.vmdk” (compact file):
0%=====================50%=====================100%
**************************************************

Above is the truncated output of the command.  It rambled on for about two pages.  In this example, VCB is pulling down a full backup of the VM.  TestVM storage is connected via ISCSI on a NetApp.  Vcbmounter works fine on our VMs whose storage is on an EMC.  Z: data store is actually a mapped drive connected to an ISCSI point.

Below is an image showing the files that were created from vcbmounter.exe

vcbmounter files

Note:  Pasting in the information above messed up the alignments of the *s, the transfers were actually 100% complete.

Edit the fstab file.  In this example, 192.168.1.100 is the NFS, the NFS export to be mounted is /vol/dir1, and local directory to be mapped to is /dir1.  Next is the mount point type.  For details on the rest of the options, use ‘man nfs’ from the command line.
# vi /etc/fstab
192.1.1.100:/vol/dir1  /dir1                   nfs     rw,bg,hard,nointr,rsize=32768,wsize=32768,tcp,nfsvers=3,timeo=600,actimeo=0 0 0

Create the local directory
# mkdir /dir1/

Mount always searches /etc/fstab if just a label (/dir1) is passed to it.  Here, it found the entry in fstab and mounted it.
# mount /dir1/

Using df to verify that mount did connect the NFS point
# df -h | grep dir
192.1.1.100:/vol/dir1
32G   96K   32G   1% /dir1

Move into the mount point
# cd /dir1

In this example, the local group luser needs to have full access to the share.  Below shows the changing of ownership from root to luser.
# chgrp luser .

Switch to the user luser1, who is also in the luser group
# su – luser1

Move into the new NFS mount point
[luser1@x ~]$ cd /dir1/

Create an empty file to test write access
[luser1@x dir1]$ touch asdf

Use ls to make sure the file exists.
[luser1@x dir1]$ ls
asdf

Logout of luser since testing is done.
[luser1@x  /]$ exit
logout

Note: The title of this is NFS (NetApp) Fstab Mount because technically this should work for any NFS mount point, not just a NetApp one.