Supercomputers: Share and Share Alike

Wandering the halls of the NCSA (National Center for Supercomputing Applications), one can just imagine the prideful voice of HAL 9000 computer from “2001: A Space Odyssey” boasting that it became operational at the HAL Plant, located at the University of Illinois at Urbana-Champaign’s Coordinated Science Laboratory, which is just 500 ft. away. The massive computing power of the HAL 9000’s artificial intelligence remains steeped in the lore of science fiction, but the echoes of its voice are a reminder of the enormous computing power available at the NCSA.

NCSA, which is supported by the state of Illinois, the University of Illinois and the National Science Foundation (NSF), is willing to share the power of its supercomputing resources with external researchers, private manufacturers and engineering companies in an effort to democratize access to supercomputing resources and fuel collaboration. That willingness to share has spawned some impressive collaborative efforts that have helped private companies ranging from the Fortune 500 to small and midsized businesses overcome significant engineering challenges.

Democratizing Supercomputing

The star of the show is the NCSA’s Blue Waters supercomputer, which is built on the latest technologies from Cray Inc. In its current configuration, Blue Waters uses hundreds of thousands of computational cores to achieve peak performance of more than 13 quadrillion calculations per second.

“If you could multiply two numbers together every second, it would take you millions of years to do what Blue Waters does each second,” says Daniel Bodony, Blue Waters engineering professor. NCSA’s published specs on Blue Waters, claim that the supercomputer offers:

• More than 1.5 petabytes of memory, enough to store 300 million images from your digital camera.

• More than 25 petabytes of disk storage, enough to store all of the printed documents in all of the world’s libraries.

• As much as 500 petabytes of tape storage, enough to store 10% of all of the words spoken in the existence of humankind.

Pairing that computing power with private sector collaboration has spawned some impressive results. In November of 2014, NCSA announced that that they had scaled ANSYS Fluent (a computational fluid dynamics simulation software package) to 36,000 compute cores. That feat, NCSA claims, is an industry first that could lead to greater efficiencies and increased innovation throughout manufacturers’ product development processes.

“NCSA is unique in connecting the industrial users, the hardware and software vendors and the domain expertise of our staff,” says Ahmed Taha, project leader, NCSA. “In addition, this level of scalability for a commercial fluid dynamics solver is unprecedented on our system, especially considering the complexity of the model physics with transient, turbulent flow, chemical species transport and multiple non-reacting flows.”

“While most organizations don’t have access to 36,000 cores today, it won’t be long before these extreme core counts are commonplace. And even today’s users who are running at much lower core counts will see direct benefits through considerably greater efficiencies. The results will be more amazing products delivered to customers much faster than ever,” says Wim Slagter, product manager, High-Performance Computing (HPC), ANSYS.

While the results of the ANSYS and NCSA’s collaboration are impressive, that project proves to be just one example of NCSA’s ability to collaborate with the private sector, as evidenced by two other projects being undertaken with private sector participation:

1. Epistatic Interactions for Brain eGWAS in Alzheimer’s disease: This project is dedicated to analyzing data from about 400 human subjects, measuring levels from approximately 24,000 transcripts that were part of an expression array. The brain gene expression levels of these transcripts were previously tested for association with 300,000 genetic variants. The gene expression measurements were done in two different brain regions: temporal cortex and cerebellum.

2. Scalability Analysis of Massively Parallel Linear Solvers on the Sustained Petascale Computing System of Blue Waters: This project, in collaboration with Anshul Gupta of IBM, involves porting the Watson Sparse Matrix Package (WSMP) solver to Blue Waters and performing full-scale benchmarking tests using assembled global stiffness matrices and load vectors ranging from 1 to 20 million unknowns extracted from commercial and academic implicit finite element analysis applications. The results will open the door to solving large multiphysics problems on petascale machines, both in academia and industry.

Those projects, as well as many others, would not have been possible without the computational power offered by Blue Waters. However, there is much more to NCSA’s collaborative efforts than hardware. The NCSA offers other resources in the form of highly trained professionals, operators and program managers, working under the careful guidance of a group of the university’s senior professors.

The NCSA has demonstrated that private sector collaboration via the NCSA’s Private Sector Program (PSP) can lead to some impressive results that benefit all involved, as well as the public at large.

Private Sector Access

Although, most of the NCSA’s supercomputing resources’ users are researchers from universities and academic research centers, the supercomputers (including Blue Waters) are available to more than just academia. The NCSA’s PSP grants access to private sector partners, boasting some 26 partners from sectors such as manufacturing, oil & gas, finance, retail/wholesale, biomedical, life sciences, astronomy, agriculture and technology. Additionally, access isn’t just limited to those onsite, NCSA offers researchers the ability to access the computers remotely.

For the private sector, the PSP’s core mission is to help its partner companies gain a competitive edge through expert use of modern, high-performance digital and human resources. The PSP creates a collaborative environment where traditional sponsored research projects are complemented by:

• Dedicated, non-government high-performance computing resources

• A high-tech, mobile consulting team

• Software/hardware benchmarking and development in production environments

• Code-performance teams

• Public-private partner leadership

• Blended partner applied research and development

The PSP also hosts annual meetings, which are designed to bring together representatives from industry (including but not limited to current partner organizations), University of Illinois research faculty and the NCSA’s staff for presentations and discussions of shared challenges and new paths to innovation.

One of PSP’s primary goals is to fuel the collaboration and cooperation between academia, industry and government. “We are now able to bring together industrial, academic and government scientist to do stupendous things. Working with our PSP clients has given us ample evidence of what multi-physics and multi-scale HPC applications are capable of,” says Merle Giles, director of PSP. “American OEMs (original equipment manufacturers) that understand HPC are pushing the envelope and are solving problems involving high-end science across multiple disciplines.”

NCSA’s PSP offers several benefits to the private sector and HPC computing in general, including:

• Industry-driven science and engineering

• High-performance software, data and hardware

• Development, demonstration and deployment

• Accelerated faculty research

• Partnerships, services and consulting

According to Giles, some 40% of the Fortune 50 and 60% of Fortune 100 manufactures have partnered with NCSA via the PSP. To meet the demands placed upon NCSA’s supercomputing resources, NCSA brought iForge online, a supercomputer for industry. iForge was purposely developed without any government funding, allowing it to be dedicated to the PSP’s work with industry and to address the ever growing queues that are present on the NSF-funded Blue Waters. iForge offers over 3500 cores and 7700TB of DDN storage allowing it to function as a development and production workhorse.

“[The] PSP helps to eliminate barriers to supercomputing, as well as code development, by allowing ISVs (independent software vendors) to scale their off-the-shelf code for high core densities,” says Giles. He says PSP partners with several technology vendors, such as Dell, Intel and Microsoft, but adds, “We tell every vendor that wants to partner with us that they must take off their vendor hat and think about what we can do together instead.” Simply put, collaboration is fuel that drives a project.

Private sector businesses looking to access PSP’s resources will need to fill out an application and provide a PDF of their proposal. For Blue Waters, that application can be found at http://goo.gl/cg46wO. For info on accessing the iForge Cluster, interested parties can visit ncsa.illinois.edu/industry/iforge to gather more information.

More Info

• ANSYS
• National Center for Supercomputing Applications