A Fresh Approach to the Data Center

An infrastructure optimization project saves time, money and energy at Queensborough Community College.

When the servers in Queensborough Community College's data center began to show their considerable age, the college decided it was time to take a comprehensive look at how the entire data center operated, says Bryan Farr, IT senior associate at the New York City educational institution in Bayside, Queens.

“When Y2K was coming down, we went to a client-server architecture, so every application had its own physical server,” says Farr, who manages the hardware in the QCC data center.

“That was in the late 1990s or early 2000. In the past few years, as equipment wore out and we started to have to replace a lot of servers individually, we realized that we had to optimize. We needed something better for management, cost savings and power – all the main drivers.”

In order to meet those needs, QCC has implemented an optimization strategy that includes server virtualization and consolidation, storage virtualization, and substantial upgrades to the power protection and management systems in the data center.

Key elements of the revamped IT infrastructure are an IBM BladeCenter chassis with 14 IBM HS21 blade servers, VMware vSphere Enterprise virtualization software, an HP LeftHand SAN and an APC Symmetra power management system.

The Benefits of Going Green

Reducing energy usage and saving taxpayer money were other important reasons for the optimization, says George Sherman, QCC's executive director of information technology and CIO.

“The green aspect was important. Mayor [Michael] Bloomberg wants all city institutions to reduce their carbon footprints,” Sherman says. “But in the process of cutting power consumption, we're also improving our computing capabilities. We're consolidating server resources and making them more quickly available, as well as creating automatic redundancy.”

QCC, a campus of the City University of New York, has about 15,000 students and 1,400 faculty and staff members. In addition to maintaining network resources for traditional classrooms as well as business and administrative offices, the IT and Academic Computing Center staff supports the wide range of online and hybrid courses offered by QCC.

The rapidly changing needs of the academic computing environment provided an extra push toward optimizing the IT infrastructure, says Farr.

“The academic world right now changes a lot with initiatives based on grant-funding cycles,” he explains. “There's always a new project coming up.”

Farr and Raj Vaswani, a senior IT associate for the QCC Academic Computing Center, had been exploring the possibilities of virtualization since 2006. But the first long stride toward optimization came the following year when QCC deployed an IBM BladeCenter as a platform for server consolidation. As with all the hardware purchased for the data center, primary selection criteria were vendor support and durability, says Farr.

“We asked every manufacturer that came in, ‘What is the life expectancy of that product line?'” he says. “You have to make sure that something you buy is going to be around and sustainable for a while. One main reason we chose the IBM BladeCenter was that the model we bought was second generation, but it fits first-generation blades and now third generation. That continuity was important to us.”

Synergy: Blades and Virtualization

Switching to blade servers saved space and power, but using the blades in conjunction with VMware is the heart of the optimization, says Vaswani. Virtualization allows five or six virtual servers to run on a single physical blade, and vSphere Enterprise provides a complete management platform for the virtual environment.

“As great as the blades are by themselves, the real power comes in running them with VMware,” says Vaswani. “VMware with the BladeCenter is the driving force that made the optimization successful. Without VMware, the blades are very good servers, but they're just servers.

“On the other hand, using the BladeCenter really maximizes VMware features like VMotion [which enables the live migration of virtual machines],” Vaswani continues. “Using the two technologies together meant that we could just wipe out all our dying servers and move them onto the BladeCenter.”

QCC already owned an HP LeftHand SAN, which was well suited to meet increased high-availability storage needs for the virtualized environment, Farr says. The college initially bought the iSCSI-based SAN to provide storage and backup for an e-mail server. But the SAN performed so well that the IT staff realized it could do much more, and do it at substantial savings over a Fibre Channel SAN, says Farr.

“We also didn't have to pay for Fibre Channel switches and a dedicated fiber network for storage,” he says. “That was a big cost savings, and we were able to put that money toward actual storage.”

QCC is now using 17 terabytes of the SAN's 30TB storage capacity. The HP LeftHand SAN offers storage clustering capabilities that make it easier to increase capacity. Thin provisioning features allocate space as data is stored rather than requiring that storage be allocated in advance.

Backup and disaster recovery are simplified by features that allow administrators to access snapshots of virtual machines at a single point in time, as well as to roll back entire volumes of backup files. The HP LeftHand SAN has also decreased maintenance hassles and is easier to use compared with the Fibre Channel-based storage area network previously used at QCC, says Farr.

“With iSCSI, we take care of just about everything ourselves,” he says. “With the Fibre Channel system, every time we had a problem, we had to call support and wait. It was so complicated we couldn't maintain it without a lot of help.”

Powering Up

Initially, QCC ran two standalone UPS units to power the IBM BladeCenter, which contains 14 IBM HS21 blade servers, eight of which are used to support a total of 60 virtual servers. One of the UPS units was connected to the electrical grid that normally serviced the building, while the other was hooked up to a backup generator circuit.

In late 2009, the college implemented APC Symmetra power protection, a modular system that is easily scalable as data center needs grow. The  power management system protects the blades, along with other standard servers, the SAN and all the hardware appliances in the data center.

“Our power here had been a nightmare for a couple of years. We've had transformers blow and major power outages time and time again, including everything going down on campus,” says Vaswani. “It was scary before we got the APC system.”

The Symmetra system also acts as a power filter and transformer, ensuring that the data center gets only “clean” power, which protects and extends the life of the hardware, says Farr.

Implementation of the data center optimization project took place in phases over two and a half years and went relatively smoothly. Backing up the virtual server environment was, however, a major challenge, Farr says.

There was no problem backing up data from the virtual machines, but the QCC staff initially had trouble recovering point-in-time snapshots of individual VMs from the SAN. The difficulty was resolved with the help of esxRanger, backup software designed to be used with VMware.

Minor integration issues with some applications in the virtual environment (most notably, with Microsoft SharePoint) as they were moved to virtual servers were resolved by tweaking VMware settings. The transition was remarkably easy, however, due in large part to VMware Converter, which duplicates physical server configurations on virtual servers, Vaswani says.

“Nineteen out of 20 applications, if not more, were perfect right away using the VMware on the blades,” he says. “And the converter was huge. We didn't have to rebuild servers; we just converted them to virtual machines. It was especially great for old servers with configurations we hadn't touched in so long we could hardly remember them.”

ROI: Productivity, Time and Reliability

QCC has invested approximately $300,000 in the technology and support services necessary for the QCC data center optimization (covering the BladeCenter with eight blade servers, VMware with Virtual Center, the HP LeftHand SAN and APC Symmetra).

Hard numbers for return on investment are difficult to calculate, in part because so much of the infrastructure was at the end of its life, making it difficult to use baseline figures for comparisons, Farr says.

“The main return on investment we see is in management time,” Farr says. “We can get things done a lot faster. Our workload has increased because of it, but that means we're doing more for more people. We can be much more responsive to administrators. Because we're a college, we don't really do ROI metrics, but we know how much better our data center works.”

Increased reliability and a decreased chance of catastrophic failure are also part of the return on the retooled data center, says Tony Hong, an IT senior associate at QCC's Office of Information Technology.

“With the new system, there's less worry that systems won't come up. If there is a problem, we can usually solve it in a couple of hours,” he says. “In the old data center, when something didn't come up, we started praying.”

QCC now does all of its application testing and troubleshooting on virtual machines at substantial time savings, says Farr. He jokes that the VMs are so useful that he looks forward to dealing with virtual server sprawl.

“That will mean that we're actually using our hardware at something close to capacity,” he says.

Maximum Capacity

Operating close to capacity – configuring the data center to make the maximum efficient use of space as well as computing and power resources – is key to data center optimization, along with having a strategy for planned growth, says Dave Cappuccio, research vice president at Gartner. 

“The trick with VMware, or all virtualization, for example, is not just to run it for its convenience but to look at optimal performance levels of the hardware and run it as close to that as you can,” he says. “The average x86 server in the United States runs at between 10 and 12 percent [capacity], but the optimal level may be more like 60 to 65 percent. So virtualize and put a few images, or 30 images if that works, on a physical server.”

Using SANs, virtualizing storage and deduplicating the data going into the repository are all ways to increase efficiency and decrease wasted space and energy in the data center, explains Cappuccio.

New data centers should be designed for “vertical scalability,” which focuses on rack space rather than floor space, and emphasizes building the IT infrastructure for maximum density and efficiency. “It's more difficult, but the same principles can be applied to optimizing an existing data center,” Cappuccio says.

“First I ask, ‘How efficient is the space you have?'” Cappuccio continues. “One calculation I did recently with a client [showed that] with increased rack density and virtualization, he could increase the computing power in his data center by 350 percent. And he would get that increase with just a 29 percent boost in power consumption. That represents the optimization of all resources, including energy, which is increasingly a big deal.”

By looking at all major aspects of the IT infrastructure, QCC is “headed in the right direction” to maximize performance, says Cappuccio.

“Infrastructure optimization is usually people taking a look at things and deciding to do what they should have done all along,” he says. “You have to ask, ‘Is there a better way to run things?' It involves fundamental infrastructure design, laying things out and running them logically and efficiently.”