DataDomain/NAS/Filesystems/Linux: Remove Files From A DataDomain’s /ddvar

A few days ago we received alerts for the /ddvar fileystem from one of our Data Domain units. Normally, I would not manually remove any files from this filesystem due to it being mainly used by the underlining OS and not for NAS storage. In this case, the problem was the “core” subdirectory. I tried to remove the files from a Windows machine, but due to permission issues, I was unable to do so. Even through Windows (CIFS) to the DataDomain, I was able to modify the permissions of the file(s) and still would receive a permission denied issue when trying to remove. The quick solution here was to add my Linux box to the NFS access list, mount “/ddvar”, and remove the files as the root user. Below details the process that worked for me.

DataDomain Web Alert

DataDomain Web Alert

SSH to the DataDomain and add my Linux systems IP address

admin1@DataDomain01# nfs add /ddvar 172.16.100.6
Mount the DataDomain /ddvar export on the Linux system
root@tstbox02:~# mount DataDomain01:/ddvar /mnt/DataDomain01/

Move into the core sub-directory under the ddvar mount point

root@tstbox02:~# cd /mnt/DataDomain01/core/

Check the directory for files. As you can see below, the core directory contained 47 gig worth of data

root@tstbox02:/mnt/DataDomain01/core# ls -lah
total 47G
drwxrwsr-x  2 root group 4.0K 2009-01-14 22:43 .
drwxr-xr-x 12 root group 4.0K 2009-04-02 10:28 ..
-rw-r--r--  1 root group 2.2M 2008-04-10 14:23 cmdb.core.26956.1207851782.gz
-rw-r--r--  1 root group 3.8M 2008-04-10 15:06 core.13124
-rw-r--r--  1 root group 121K 2008-04-08 07:01 ddfs.core.14547.1207652507.gz
-rw-------  1 root group 352K 2008-04-07 22:56 ddfs.core.15098.1207618361.gz
-rw-r--r--  1 root group 167M 2008-04-09 09:30 ddfs.core.15100.1207747799.gz
-rw-r--r--  1 root group 167M 2008-04-08 10:37 ddfs.core.15125.1207665449.gz
-rw-r--r--  1 root group 167M 2008-04-08 10:38 ddfs.core.15505.1207665497.gz
-rw-r--r--  1 root group 167M 2008-04-09 09:31 ddfs.core.15506.1207747848.gz
-rw-r--r--  1 root group 527M 2008-09-10 16:27 ddfs.core.1560.1221078099.gz
-rw-r--r--  1 root group 166M 2008-04-08 07:32 ddfs.core.15675.1207654304.gz
-rw-r--r--  1 root group 167M 2008-04-08 10:39 ddfs.core.15795.1207665554.gz
-rw-r--r--  1 root group 167M 2008-04-09 09:32 ddfs.core.15813.1207747905.gz
-rw-r--r--  1 root group 636M 2008-07-28 15:20 ddfs.core.15991.1217272516.gz
-rw-r--r--  1 root group 167M 2008-04-08 10:40 ddfs.core.16055.1207665605.gz
-rw-r--r--  1 root group 167M 2008-04-09 09:33 ddfs.core.16095.1207747957.gz
-rw-r--r--  1 root group 167M 2008-04-08 07:32 ddfs.core.18047.1207654349.gz
-rw-r--r--  1 root group 167M 2008-04-08 07:33 ddfs.core.18231.1207654404.gz
-rw-r--r--  1 root group 167M 2008-04-08 07:34 ddfs.core.18591.1207654458.gz
-rw-r--r--  1 root group 548M 2008-09-20 02:56 ddfs.core.22290.1221893385.gz
-rw-r--r--  1 root group 167M 2008-04-09 10:23 ddfs.core.22705.1207750958.gz
-rw-r--r--  1 root group 167M 2008-04-09 10:23 ddfs.core.22875.1207751006.gz
-rw-r--r--  1 root group 167M 2008-04-09 10:24 ddfs.core.23008.1207751055.gz
-rw-r--r--  1 root group 553M 2008-08-19 15:26 ddfs.core.23183.1219173638.gz
-rw-r--r--  1 root group 167M 2008-04-09 10:25 ddfs.core.23269.1207751113.gz
-rw-r--r--  1 root group 167M 2008-04-09 09:06 ddfs.core.28347.1207746385.gz
-rw-r--r--  1 root group 167M 2008-04-09 09:07 ddfs.core.28507.1207746432.gz
-rw-r--r--  1 root group 167M 2008-04-09 09:08 ddfs.core.28694.1207746490.gz
-rw-r--r--  1 root group 167M 2008-04-09 09:09 ddfs.core.28989.1207746540.gz
-rw-r--r--  1 root group 167M 2008-04-09 09:14 ddfs.core.29883.1207746837.gz
-rw-r--r--  1 root group 213M 2008-04-07 22:05 ddfs.gdb.15098.1207618361

Since I verified that we do not need the archived (.gz) files, they were removed

root@tstbox02:/mnt/DataDomain01/core# rm *.gz

By only retaining the newer files in this directory, the size is now down to 1.6 gigs.

root@tstbox02:/mnt/DataDomain01/core# ls -lah
total 1.6G
drwxrwsr-x  2 root group 4.0K 2009-04-15 14:29 .
drwxr-xr-x 12 root group 4.0K 2009-04-02 10:28 ..
-rw-r--r--  1 root group 3.8M 2008-04-10 15:06 core.13124
-rw-r--r--  1 root group 213M 2008-04-07 22:05 ddfs.gdb.15098.1207618361

Once this was completed, the alert in this DataDomain cleared in the web interface.

Data Domain Web Alert Cleared

Data Domain Web Alert Cleared

Notes: Always verify files that are being removed are not needed. If there are core files present, it generally means the system has hit an error. Verify with your DataDomain support before modifying the ddvar filesystem.


~ by Kevin Goodman on April 16, 2009.

Leave a Reply