InfiniBand Gets Its Groove Back by Todd Bundy It never really went away. But InfiniBand—envisioned in the early 2000s as the foundational, system-wide connectivity protocol in enterprise networking—faded out of mainstream IT managers’ plans for much of the decade. Today, the technology is back firmly in the conversation. The growth of latency-intolerant, bandwidth-intensive applications such as grid computing, supercomputing and clustering is driving InfiniBand deployment. The TOP500—a ranking of the most powerful computer systems according to the Linpack benchmark rating system—has quantified InfiniBand’s emergence as a critical enabler of system interconnection. More than a quarter of the June 2007 TOP500 list of the world’s most powerful supercomputers (http://www.top500.org/lists/2007/06/highlights/general)—a total of 127—rely on InfiniBand as the internal system interconnect. TOP500’s previous list (November 2006) included only 78 such interconnects. That’s more than 62-percent growth for InifiniBand in six months. Cost-effective clustering applications are especially popular for enterprise mainframe and storage connectivity. With these services evolving from limited, niche usage (such as in the research-and-education community) to mainstream business applications across a wide range of industries, so is growing enterprise reliance on InfiniBand as an interconnect technology. When transported across optical networks enhanced with protocol-agnostic Wavelength Division Multiplexing (WDM), InfiniBand-enabled services such as mainframe clustering and virtualized storage can be consolidated along with existing Fibre Channel-based storage applications and Ethernet-based business applications. The enterprise gains not only unbridled throughput for its most demanding applications but also appealing configuration simplicity over geographic distances. This approach enhances a corporate disaster-recovery plan. Clustering’s Rapid Rise More servers, storage and network bandwidth than ever are necessary today for an enterprise to conduct its business efficiently and compete effectively. Applications that leverage “unstructured” data—video, sounds, images, etc.—are simply overwhelming the capacity and capabilities of traditional systems. As an alternative, many enterprise Information Technology (IT) managers are choosing to cost-effectively aggregate servers and processing power across geographically dispersed data centers. InfiniBand is becoming one of a variety of interconnect options for an enterprise to choose from, including Gigabit Ethernet, Fibre Channel and proprietary solutions. Very low-latency Ethernet is another emerging competitor. Comparative performance and latency levels, power consumption and cost are among the factors that IT managers weigh in making their choices.  As indicated by the TOP500 trend, more and more enterprises are electing to base their server and storage clusters on 5Gbit/s and 10Gbit/s InfiniBand-based fabrics. Instead of deploying a large database on one high-end server, for example, the enterprise can more cost-effectively support the application by clustering industry-standard servers and distributing workload across the cluster. Computing processes are streamlined and expedited across multiple resources. Overall system resiliency is enhanced because, if one particular server fails, other servers within the cluster shoulder the workload. The benefits include greatly enhanced performance, reliability, scalability and overall workload management. Meeting Business Needs Not long ago, clustering was mostly limited in use for research and education applications, but, today, more companies are clustering servers, over different protocols depending on the business need. Web servers, for example, can be cost-effectively clustered over Fibre Channel-based storage area networks (SANs) or higher-performance InfiniBand to support 24-hour-a-day, seven-day-a-week, Internet-based business. The uniquely high performance and low latency of InfiniBand are expected to complement future offerings from industry-leading server clusters, such as Geographically Dispersed Parallel Sysplex Server Time Protocol (GDPS STP) environments. IBM’s GDPS STP is often used to support an enterprise’s synchronous storage, data-replication applications for business continuity and disaster recovery. Enterprises cluster servers across distributed networking environments while guaranteeing no loss of data and 99.999-percent reliability. “Fortune 1000 companies around the world rely on GDPS STP to ensure the resiliency of their operations, and InfiniBand provides the high performance and low latency that these sophisticated storage services demand,” said Dr. Casimer DeCusatis, distinguished engineer of the IBM System and Technology Group. “Deploying WDM among data centers enables the enterprise to cost-effectively and reliably converge the InfiniBand-based clustering applications with the rest of its network traffic.”   Lacing InfiniBand Across WDM For the sake of configuration simplicity, IT managers seek to cluster both their application servers and data storage. This represents another significant architectural shift, and it’s already underway across a wide range of industries including finance, life sciences, energy, media and manufacturing. Consolidating mainframe and storage services onto a 5Gbit/s or 10Gbit/s InfiniBand fabric offers the enterprise tremendous speed and low latency and renders the construction of clusters much easier and more affordable. Other enterprise protocols such as Fibre Channel, Fiber Connection (FICON), Enterprise System Connectivity (ESCON) and Internet Protocol (IP) will continue to have their place in enterprise networking, but they simply cannot provide the appealing levels of latency and performance that InfiniBand delivers and the emerging clustering applications require. Flexible, WDM-based optical networking platforms from qualified vendors provide the application transparency to collapse InfiniBand (or another interconnect technology) and other protocols on the same foundation. WDM creates protocol-agnostic “virtual channels” across optical fiber networks, each capable of transporting a different standard protocol (InfiniBand, Ethernet, ESCON, FICON, Fibre Channel, GDPS STP, etc.) at its native speed. At one end of an optical strand, traffic is multiplexed, and, at the other, de-multiplexed. There is no performance degradation for the applications. With storage servers—or “controllers”—clustered over InfiniBand and traffic transported across the network via WDM, storage can be cost-effectively virtualized. In storage virtualization, information of varying sensitivity and importance levels are assigned to different storage technologies and protocols. The enterprise ensures its most critical data is continuously available regardless of isolated failures along the network, but the high-end technologies are not wasted on little-accessed data and there’s no complexity associated with managing multiple interfaces. The total cost of ownership for storing, protecting and recovering information is controlled, and IT processes are brought into closer sync with business priorities. Storage virtualization is one of the possible manifestations of the trend toward processing power and storage capabilities evolving into on-demand utilities, as opposed to static resources. InfiniBand and WDM are important enabling elements to this long-range vision. Conclusion To effectively compete, enterprises have grown increasingly reliant on bandwidth-intensive network services carrying unstructured data. They have adopted steadily more powerful business-continuity and disaster-recovery capabilities. And enterprise IT managers are under unprecedented pressure to simplify infrastructures and keep operational complexity and costs in check. These trends have made IT managers intrigued with the possibilities of cost-effectively consolidating servers and processing power in flexible server and storage clusters via InfiniBand, transported across WDM-enhanced optical networks. InfiniBand—once a mere niche player for academic and research applications—is carving itself a more important role across enterprise networking.  ### Todd Bundy is Director of Global Alliances with ADVA Optical Networking (www.advaoptical.com).