University of Alberta researchers achieve data transfer land speed record

Edmonton-based Obsidian Strategics Inc., in collaboration with researchers from the University of Alberta, managed to demonstrate a record-breaking data transfer speed of 928 MB/s at the Supercomputing 2015 (SC15) conference in Austin, Texas recently, using Canada’s National Research and Education Network.

The conference, attended by more than 10,000 specialists from around the world, was the site for the data transfer, using Obsidian’s Longbow E100 unit, an enterprise class range extender that spreads InfiniBand across a wide area network.

“When you use a wide area network, a network that spans a large geographical distance beyond a city for example, the performance can drop off greatly,” says Dr. Paul Lu, a professor in the University of Alberta’s Department of Computing Science. “There are more pieces of equipment in the way—middleboxes, routers—and every one of those pieces slows down the ability to transfer data quickly.”

InfiniBand, which has been in widespread use for supercomputing since 2014, was developed for very high-speed data transfer, but over short distances, usually within a data centre.

The demonstration between Edmonton and Austin represents a first-of-its-kind demonstration of InfiniBand technology over large distances.

Typically, long-distance file sharing travels over standard Internet protocols, or TCP/IP, which represent bottlenecks in the system.

Researchers working on big files have often found it easier to ship hard drives through the mail or physical delivery services.

The file being transferred was 300 GB of genetics research data provided by the University’s Department of Biological Sciences relating to gut microbiome, including associated next-generation sequencing data.

“Next generation sequencing machines, in which DNA or cDNA copies of RNA are sequenced with ever-increasing speed and throughput, and ever-decreasing costs, generate sequence data of the specific bacteria from a patient’s sample,” said Lu. “Frequently we have to transfer the sequence data from where it originates to a compute centre. The datasets of known sequences, often larger than 500 GB, may have to be transferred as well.”

The demonstration was facilitated by Dr. Lu, a Computer Science professor, with the goal of accomplishing a large data transfer at record-breaking speeds between Edmonton and Austin, using InfiniBand, a computer-networking communications standard for supercomputers.

Each transfer site, one in Edmonton and the other in Austin, was configured with a dedicated server and solid state driver rated to transfer data at a speed of 900 MB/s, by Dr. David Southwell, Obsidian’s director and chief visionary officer, and Jason Gunthorpe, director and chief technology officer.

The National Research and Education Network (NREN) is a digital infrastructure connecting 12 provincial and territorial partners, the backbone of which is CANARIE, Canada’s Advanced Research and Innovation Network, which connects Canada to more than 100 similar advanced networks.

CyberaNet, the Alberta component of NREN, also collaborated on the data transfer, along with the International Center for Advanced Research (iCAIR) and Compute Canada.

“With the participation of iCAIR, they routed the network from Austin to Chicago, CANARIE routed it to the province of Alberta, and Cybera picked it up as our Alberta network provider,” says Lu. “We received a very high performance network connection between those two sites.”

Facilitated by a high speed (10Gbit/s) network connection on either end, the researchers were able to move the 300 GB file across a distance of 3,500 kilometres, from Edmonton to the convention floor in Austin, in less than six minutes.

“In fairness it could have gone faster; the bottleneck was the disk drive,” said Lu.

For scientists, being able to work with large data sets in as close to real time as possible, using low-latency and high-bandwidth connections, increases the potential for real gains in research.

“This allows our researchers to be involved in global research programs no matter where they occur; our physical location is immaterial to their ability to participate,” says Scott Delinger, director of research computing in the University’s Information Services & Technology department.