Supporting global collaborations

The Challenge:
QuakeCoRE’s earthquake simulations and mapping led to international partnerships with the United States and South Korea. But this partnership meant adapting existing data pipeline from two different high-performance computing (HPC) facilities to QuakeCoRE’s model.

The Solution:
NeSI’s designed its HPC architecture to global standards. This made porting to international supercomputing centres, TACC and KISTI, easier.

The Outcome:
Using NeSI's HPC platform helped QuakeCoRE acquire international recognition and collaboration opportunities. Particularly with South Korea, QuakeCoRE obtained access to the one of world’s largest supercomputers by sharing QuakeCoRE-developed technology and research to kick-start seismic activities in Korean Peninsula.

QuakeCoRE, the NZ Centre of Earthquake Resilience, has made a global impact with earthquake-induced ground-motion simulations using high performance computing. In 2020, Professor Brendon Bradley and a scientific software team led by Dr Sung Eun Bae were granted two resource allocations at major international supercomputing centres.

Brendon and Sung are long-standing NeSI users. Their team relies on powerful computing resources to simulate earthquake ground motions for the Cybershake NZ project. Cybershake simulates possible future earthquakes from each known fault in New Zealand. This would predict the earthquake hazard across New Zealand so government and industry can prepare and improve infrastructure. A poster showcasing some highlights and context of that work can be viewed here (also featured at the top of this page). 

Previously, NeSI allocated several million core hours to Brendon and Sung for the Cybershake NZ project. Using NeSI allowed the QuakeCoRE team to study over 500 faults across New Zealand, with over 20,000 possible earthquake scenarios. This includes the Hikurangi subduction interface, the largest fault source in New Zealand and the Alpine fault whose next rupture is estimated to be imminent.

"The Alpine fault ruptured roughly every 300 years producing an earthquake of magnitude 8. The last end-to-end rupture was in 1717 (303 years ago), and such an event is expected to have significant impacts throughout the South Island and lower North Island" said Brendon.

The team’s work on the Cybershake NZ has propelled them to the world stage. Now they are working with international supercomputer centres in South Korea (KISTI) and Texas, USA (TACC).

“Extensive use of NeSI’s HPC resources helped QuakeCoRE to produce world-class research output, which made our research internationally visible. They helped us test the workability of our projects and showed us we could scale them up to higher resolutions and larger data sets,” Sung said.

The Texas Advanced Computing Centre (TACC)’s Stampede2 is ranked 21st in the top500 global supercomputer list. The Korean Institute of Science and Technology Information (KISTI) is ranked 17th. Brendon and Sung were allocated nearly 12 million core hours at TACC and KISTI combined.

The QuakeCoRE team collaborated with the United States Geological Survey scientist, Dr Rob Graves. Rob was the developer of the ground-motion simulation code that forms a key component of the Cybershake workflow that QuakeCoRE used and improved.

“Our own contribution to the code included automation and creating the computing pipeline. We also fine-tuned the code and its inputs to better work for New Zealand’s geology. This enabled us to run thousands of ground motion simulations on the NeSI platform in a massively automated fashion,” Sung said.

The 2016 Gyeonju and 2017 Pohang earthquakes in South Korea were a wake-up call for research community and authorities. Combined, the earthquakes caused 90 injuries and damaged thousands of properties.

“South Korea has been believed to be relatively safe from earthquakes, but these events proved that belief was wrong.”

Sung, a Korean-born Kiwi, explored a collaboration opportunity with his home country and brought a partnership agreement with KISTI from which both parties can benefit.

“South Korea is relatively new to this line of research, but they have a world-class HPC facility. It is a win-win for both countries. We can help KISTI to use our scientific expertise and computational workflows to study potential earthquake scenarios in Korea. In return, QuakeCoRE gets the access to their HPC facilities that will enable us to run Cybershake NZ in higher resolution and increase the number of earthquake scenarios,” said Sung.

There are also several New Zealand-trained earthquake researchers active in South Korea, who are expected to play important role in the on-going collaboration between two countries.

“Getting our workflow to run on KISTI’s Nurion was not completely straightforward, but Nurion is essentially Mahuika, a Cray supercomputer, with a lot more cores.”

Sung said the QuakeCoRE team’s experience on NeSI’s Mahuika supercomputer gave them a huge advantage working with KISTI.

“Because the ground-motion simulation was new to KISTI, we had to build the environment from scratch. We also had to restructure our workflow to adapt to Nurion’s scheduler (PBS), which could have been a major task if we didn’t have an in-depth understanding of Mahuika.”

Through QuakeCoRE's use of NeSI, Sung and Brendon had experience with different high-performance computing architectures and they were familiar with porting through generations of NeSI clusters. They started on BlueGene/P, then moved to Fitzroy, Kupe, and finally Māui and Mahuika today.

NeSI remains vital to QuakeCoRE’s international collaboration. There are advantages to local high-powered computing that the world’s biggest players can’t match.

“NeSI support is world-class. They’re always keen to accommodate our needs and being in the same time zone make this very easy. NeSI’s subscription queue is the most responsive and its two-factor authentication is the most user-friendly of all we’ve used too,” said Sung.

These international collaborations bring value back to New Zealand. Large-core international supercomputers help improve QuakeCoRE models back home.

Meanwhile, a partnership with South Korea and New Zealand means local researchers are proving their global expertise in earth sciences.

NeSI’s goal is always to support New Zealand research. But the QuakCoRE team’s work shows that, by keeping its architecture globally competitive, NeSI can help catapult users to the world stage.

Do you have an example of how NeSI support or platforms have supported your work? We’re always looking for projects to feature as a case study. Get in touch by emailing support@nesi.org.nz.