Colocation to Virtualization » Miscellaneous

Networking / Colocation: Baetech ATS (Automatic Transfer Switch) testing for Cisco/single Powered Devices

Kevin Goodman — Mon, 06 Apr 2009 14:37:35 +0000

Below details the testing of an Baetech ATS-11 series (ATS18A-30) ATS (Automatic Transfer Switch). The networking rack houses some devices that contain only one power supply. The main focus of testing this unit was to provided failover power capabilities to our stacked network switch cluster (Cisco 3750G). A few options were out there for lower (Amps) but would not work for our implementation. Cisco Redundant Power System were also evaluated, but were limited in their abilities. From what I remember, 6 devices could be plugged in, but the RPS would only be able to power 3 of the 6 devices. This would mean we would have to use 2-3 of these units to actually be fully backed up.

230 VAC, 30 Amp ATS with RS232 access

ATS18A-30 Specifications

Number of inputs:	2
Circuit breakers/input:	2-30A
Total output receptacles:	2
Total rebootable receptacles:	0
Phases:	1
Vertical unit:	NO
Current sensors:	1
Voltage sensors:	2
Power Measurment:	YES
nternal temperature sensor:	YES
RS232 (serial) access:	YES
Ethernet access:	NO

The power input (2) on this unit (to the floor) are NEMA L6-30P (twist lock) plugs. We already had APC vertical PDUs in our racks and wanted to be able to integrate them with the ATS solution. This model contains NEMA L6-30R (2 female sockets) that allowed the APC metered PDUs to be plugged directly in.

Testing information
Due to limited resources for testing, only a 6 amp load was placed on the ATS switch. The following devices were used:

Quantity	Device
2	APC vertical 208V Metered PDUs Model AP7841
5	IBM System X servers
1	Cisco 3750G
2	Dell PowerConnect 5224 switches

Initial login to the ATS system.

ATS-11 Series
(C) 2003 by BayTech

F3.04
Option(s) Installed:
True RMS Current
Internal Temperature
True RMS Voltage

Unit ID: ATS18A-30
Input Power Source:  1
     Average Power:  10 Watts 

True RMS Voltage 1:   209.1 Volts
True RMS Voltage 2:   208.3 Volts
  True RMS Current:     0.0 Amps
  Maximum Detected:     0.1 Amps

Internal Temperature:  29.5 C

Type "Help" for a list of commands

Below lists the commands that can be used while the serial mode is set to console on the ATS unit.

ATS-11>help

Status    --ATS-11 Status
Config    --Enter configuration mode
Source n  --Select the input power source, n=1 or 2
Current   --Display True RMS Current
Clear     --Reset the maximum detected current
Temp      --Read current temperature
Voltage   --Display True RMS Voltage
Logout    --Logoff
Logoff    --Logoff
Exit      --Logoff

Password  --Changes the current user password
Whoami    --Displays the current user name
Unitid    --Displays the unit ID
Help      --This Command

Type "Help" for a list of commands

Information before attaching the devices

ATS-11>status
Input Power Source:  1
     Average Power:       6 Watts
True RMS Voltage 1:   209.1 Volts
True RMS Voltage 2:   208.3 Volts
  True RMS Current:     0.1 Amps
  Maximum Detected:     0.1 Amps

Internal Temperature:  29.5 C

After attaching devices

ATS-11>
Unit ID: ATS18A-30

Input Power Source:  1
     Average Power:    1101 Watts 

True RMS Voltage 1:   208.7 Volts
True RMS Voltage 2:   208.8 Volts
  True RMS Current:     5.9 Amps
  Maximum Detected:     6.0 Amps

Internal Temperature:  30.0 C

Start of testing

ATS-11>current

  True RMS Current:     5.7 Amps
  Maximum Detected:     6.0 Amps

Below shows that we are currently running on power source 1

ATS-11>status

Input Power Source:  1
     Average Power:    1034 Watts
True RMS Voltage 1:   208.8 Volts
True RMS Voltage 2:   208.9 Volts
  True RMS Current:     5.7 Amps
  Maximum Detected:     6.0 Amps

Internal Temperature:  30.0 C

After failover test of source 1, below shows that the unit was failed over to the second power leg. This was completely transparent to ALL devices.

ATS-11>status

Input Power Source:  2
     Average Power:    1014 Watts 

True RMS Voltage 1:     0.4 Volts
True RMS Voltage 2:   208.2 Volts
  True RMS Current:     5.7 Amps
  Maximum Detected:     6.0 Amps

Internal Temperature:  29.5 C

Power source 1 was reconnected. The unit continued to run on source 2. For testing, source 2 was failed. As shown below, the unit automatically failed back over to source 1.

ATS-11>status

Unit ID: ATS18A-30

Input Power Source:    1
     Average Power:    1032 Watts 

True RMS Voltage 1:   208.8 Volts
True RMS Voltage 2:     0.6 Volts
  True RMS Current:     5.7 Amps
  Maximum Detected:     6.0 Amps

Internal Temperature:  30.0 C

Details one more time of current Amps.

ATS-11>current

  True RMS Current:     5.6 Amps
  Maximum Detected:     6.0 Amps

This unit has the ability to manually failover the power source via the console. Below confirms this ability. The input power source is currently 1.

ATS-11 Series

(C) 2003 by BayTech

F3.04
Option(s) Installed:

True RMS Current
Internal Temperature
True RMS Voltage

Unit ID: ATS18A-30

Input Power Source:    1
     Average Power:    1027 Watts 

True RMS Voltage 1:   208.7 Volts
True RMS Voltage 2:   208.9 Volts 

  True RMS Current:     5.6 Amps
  Maximum Detected:     6.0 Amps

Internal Temperature:  30.0 C

Using the “source” command, the unit is manually failed over to the other power source.

ATS-11>source 2

Unit ID: ATS18A-30

Input Power Source:  2
     Average Power:     956 Watts 

True RMS Voltage 1:   209.6 Volts
True RMS Voltage 2:   208.1 Volts 

  True RMS Current:     5.7 Amps
  Maximum Detected:     6.0 Amps

Internal Temperature:  30.0 C

Results:

All testing was completed without error. None of the devices reported any power problems or errors. The following were monitored:
- Each server via the RSA logging interface
- Cisco catalyst logs
- PDU interface and logs
- Visual inspection to check for booting indicators

The ATS was failed over from power source 1 to 2 without issue. Next source 1 was brought back online, and source 2 was failed. The switch failed back to source 1. Source 2 was brought back up and source 1 was failed again. This was performed approximately 5 times for verification. Also, manual failover utilizing the ATS serial console was performed without issue.

Notes: So this really works. I was somewhat sceptical initially, but now I am ready to put it into production. This is a great option for providing redundant power paths for servers or networking equipment that only has one power supply. I know that the single power supply units are still not truly redundantly powered, but at least this gives that single power fed unit a better chance. In this implementation, the power sources are redundant pathed under the floor at the colocation. This would work great also for a small office. Just feed one PDU to commercial building power and the other to a UPS unit.

Posted in Colocation, Miscellaneous, Networking

NetApp Active Directory DNS issues. dns.update.enable

Kevin Goodman — Thu, 23 Oct 2008 17:15:40 +0000

There are 2 IPs assigned to each of the NetApp units here. One for the regular network VLAN and one on the ISCSI network. When querying the DNS for the IP information, both IPs would come back. Turns out that under options in the NetApp command line, dns.update.enable was the issue. This option told the NetApp to send DNS updates to the DNS servers, thus auto updating the records with both its public IP and ISCSI VLAN IP. This was causing an intermittent problem for all servers and clients that were trying to access NFS or CIFS (windows) shares on the NetApp.

Incorrect DNS information
$ nslookup netapphn

Name: netapphn.x.com
Address: 127.x.1.85
Name: netapphn.x.com
Address: 127.x.200.85

Our other NetApp off site was not having this issue. It only resolved to one IP

From the Netapp CLI, it shows that DNS updating is turned on
NAS> options dns.update.enable
dns.update.enable on

Trun it off
NAS> options dns.update.enable off

Now, verify that the change took
NAS> options dns.update.enable
dns.update.enable off

The change did take. Now time to check the DNS
$ nslookup netapphn
Name: netapphn.x.com
Address: 127.x.1.85

Great, DNS now resolves correctly. No more hading out two IPs.

Note: Will need to flush the DNS cache on the server or delete the incorrect “A” record out of the DNS server to see immediate results.

Posted in Miscellaneous, NAS, Networking

Data Domain

Kevin Goodman — Thu, 23 Oct 2008 03:44:51 +0000

I have been managing Data Domain units for about a year now. Currently, there are 3 running on the network. Initially, some issues came up that made me a little “weary” of them. First was that we had a few occurrences of one unit disabling its file system. The biggest problem was when it hit a “less than 1 percent chance” error. The hash function used in the RAID software to isolate corruption had a problem slip through. This rendered the unit worthless for about 3 days worth of debugging from Data Domain. Since then, there have been no major problem with either 3 of the units. Also, the compression that we are getting out of the units is amazing. The following information is coming from a report I did when we were only running Data Domain 510s. Backups started to surpass the designated backup window and this is the argument for the upgrade.

3.75 TB Raw

1.95 TB Usable

290 GB/hr throughput

Problems

We are maxed out in capacity (number of disks) on the DD510
Currently using ~70% of the available storage
Can not meet a backup window of 8hrs
We are maxing out the processor during the backup window

Backup window calculations
Using the last 24 hour pre-compression write rate, we are pushing 2810.8 gigs of data to the Data Domain. Per DD benchmarking and documentation, the max throughput for the DD510 is 290GB an hour. Using an 8 hour window, the calculations would be as follows:

2810.8/8 = 351.35 GB /hr

This shows that we are already trying to push more data than the DD 510 can handle in the time window we are trying to achieve. Below is from Data Domains documentation showing a comparison between their models

Future expansion
Plans are in the works for the IT department to implement NetBackup for Exchange and also move their active directory backups to the Data Domain. Forecasting of the space required for exchange backups would need to be ~1000 gigs. Currently, the Exchange data store is around 350 gigs. Added into the current setup that would be

3160.8/8 = 395.1 GB /hr

That would put us over the limit of the DD530 appliance and up to the DD565 in just throughput alone. The DD565 would give us the ability to scale up to 3 shelves of disks, whereas the DD510 and 530 can not expand beyond one enclosure (shelves).

Conclusion
Best case for achieving a backup window of ~8 hours or below would be to purchase a DD565 for use at the primary site. This would allow for over double the hourly throughput and up to 6 times as much raw storage capacity. The costs and compression capabilities of using a Data Domain system in comparison of an EMC or NetApp solution seems to be the most financially reasonable route. I do hear that EMC Avamar is coming down in price! Also, NetApp free deduplication license for the NetApp does reasonable compression as well. Although with NetApp, that means owning one first and that would cost a lot more than the Data Domain.

Since this initial writing, there is a new Data Domain 565 in the process of being deployed. Once this is up and running, I will update the information here with the backup window results.

Note: This is destination based agentless deduplication, not source based. All the data is pushed from the servers, through the network, and to the datadomain. The data is then deduped on the Data Domain. EMC Avamar is agent based, allowing the server to only push the changed blocks or data to the Avamar deduplicaiton appliance. This will definitely take a load off of a network. Only problem with that is you have to install agents on all servers, as well as update that software. You better have at least a true gigabit network when pushing upwards of 5 terrabytes a night to the data domain. All in all, I am very happy with data domain and recommend them.
Update: Per a conversation with a Data Domain sales team today, they show in the labs that restore speed (read from the Data Domain) is ~85% of the write speed to the device.

So, if a backup takes 1 hour, retrieval of that data should take approximately 1 hour and 11 minutes (70.588 minutes). Now, this does not take into account the overhead of the application doing the restore. Hope I did my math correctly.

Posted in Data Domain, Miscellaneous, NAS, SAN (Storage Area Network)

Laptop.. Scratch That. Ultimate Mobile Workstation is more like it

Kevin Goodman — Tue, 21 Oct 2008 03:40:18 +0000

Ok, budget time is here at work and I have been tasked with the decision between a new workstation or laptop. Being a server / linux / networking guy, I am a huge fan of high performance laptops. They have always been missing one thing, Raid (Redundant Array of Inexpensive Disks). Alienware released their first RAIDed laptop a while back and Lenovo which took over IBMs laptop division has followed. This thing is a beast! That is in features and in the wallet. The base unit of their ThinkPad W700 STARTS at $2,729.00. Specs are as follows:

Intel Core 2 Extreme processor QX9300 (2.53GHz 1066MHz 12MBL2)
17″ WUXGA 400NIT TFT
NVIDIA Quadro FX 3700M 128-core CUDA parallel computing processor 1GB (dedicated)
4 GB PC3-8500 DDR3 SDRAM 1067MHz SODIMM Memory (2 DIMM)
Ultranav + Fingerprint Reader + Pantone Color Sensor
1.3 Megapixel Integrated Camera
Internal RAID – Configured by Lenovo
Primary SATA RAID 1
Dual 200GB, 7200rpm RAID Enabled Hard Disk Drives with Disk Encryption
DVD Recordable 8x Max Ultrabay Enhanced (Serial ATA)
Compact Flash + Express Card(34mm)
Intel WiFi Link 5300 (AGN)
Integrated UWB WPAN

Now, that is not completely configured to the max specification and its costing $3,344.00. I definitely do not need a Blue Ray reader in a laptop, but I do want raid, disk encryption, and 7200 rpm drives. Getting that to work with Linux is another thing, which would happen as soon as it was received. Now for the bad, it weighs around 8 pounds.

Almost forgot, you can get a built in digitizer with pen for digital editing or taking notes like a tablet.

http://shop.lenovo.com/ISS_Static/merchandising/US/PDFs/W700_Datasheet.pdf

Posted in Miscellaneous