Using 3D models to improve health outcomes

The below case study shares some of the technical details and outcomes of the scientific and HPC-focused programming support provided to a research project through NeSI’s Consultancy Service.

This service supports projects across a range of domains, with an aim to lift researchers’ productivity, efficiency, and skills in research computing. If you are interested to learn more or apply for Consultancy support, visit our Consultancy Service page.

Research background

At the University of Auckland, researchers Prof. James Sneyd, Dr. John Rugis and Shan Su have been developing a multiscale model of saliva secretion as part of an international collaboration that is funded by the National Institutes of Health (USA). Saliva plays a vital role in oral health, serving to lubricate and protect the oral mucosa and facilitate mastication and swallowing, as well as contributing to the initial digestion of food. Insufficient saliva secretion results in dry mouth syndrome (Xerostomia), which can be quite debilitating. Therefore, understanding how the salivary gland works could have a significant impact on improving the quality of life of a large population of Xerostomia patients.

Currently, the research group is focussing on two main areas of work. Firstly on the development of an accurate 3D model of a salivary gland that will be used to run simulations of saliva secretion under different conditions. The 3D model is a complex structure, with a number of different types of biological entities, which must be reconstructed from a stack of microscopy images. 

Salivary gland duct outline traced through a microscopy image stack in Blender.

The second main area of work is in developing and implementing the mathematical model of saliva secretion. The researchers usually prototype their code in MATLAB on a local workstation and, once it is working well there, port the code to C++ for better performance and ease of running on the NeSI HPC.

Project challenges

Some steps of the method used for constructing the 3D model of the salivary gland require significant computational power and can benefit from simple parallelisation (running many independent calculations at the same time) and, therefore, could benefit from running on NeSI. However, the software stack used for this step is fairly complex and not trivial to get up and running on the NeSI HPC.

Creating a high performance implementation of the prototype code also has some challenges, including the need to verify that the results produced by the new implementation match the original results. Also, there is often a need to rewrite code in a lower level language which can require more effort. Once the cluster version of the code is written, it will be used to perform parameter sweeps, with many independent simulations running at once, which would be very difficult and time consuming to manage and analyse manually, thus the need to develop some scripting and automation to handle this.

What was done

NeSI Research Software Engineer Chris Scott and Data Engineer Maxime Rio collaborated with the research group during a NeSI Consultancy project to help address the challenges above.

The generation of synthetic meshes of a salivary gland is done with Blender, an open source 3D graphics software, using its physics engine to adjust the position of cells in a natural looking way. This time-consuming process depends on the random starting position of the cells and parameters of the physics simulation. We adapted the mesh generating code to be run non-interactively on NeSI using Blender as a Python package, as part of a Slurm job array. These modifications allow the researchers to generate more candidate meshes and reduce the time to experiment with different sets of parameters.

With respect to the simulation code, we have ported the Matlab mesh processing and setup code to C++ and confirmed that the results match. The “Striated Duct” implementation was also ported to C++ and the SUNDIALS CVODE solver was implemented for solving the ODE system. Results between the Matlab and C++ implementations were shown to be close. Python scripts and Jupyter notebooks were also created for managing running the code on NeSI and analysing and comparing results between the different implementations.

Comparing MATLAB (MAT) and C++ (CXX) results for a specific simulation.

During the project we have also switched to using a standard file format for parameter input (INI file). The benefit of doing this was that third party file readers exist for all languages we are using (Matlab, C++ and Python), which means the same input file can be used everywhere, which greatly decreases the likelihood of errors related to copying input parameters around. We have also switched to using HDF5 for outputting results of the simulations and loading initial conditions if resuming a simulation. HDF5 is a standard, portable file format, which is very good at storing structured data. HDF5 file readers and writers are available for Matlab, C++ and Python too, which means we can output the results from Matlab or C++ in the same format and then compare them in Python very easily.

Main outcomes

• Blender Python module installed and running on NeSI

• First part of the Matlab code has been ported to C++ and comparisons are favourable

• Standard parameter file format used to improve robustness

• Standard, HDF5, format used for storing input and output data

Researcher feedback

"This project is heavily reliant on advanced computer infrastructure for numerical modelling, simulation, physical modelling, mesh construction and 3D visualisation. We could not have undertaken work of this complexity and scale without support from the NeSI Consultancy team. We would like to specifically acknowledge the valued expertise of Chris and Maxime."

- Dr. John Rugis, University of Auckland

Do you have a research project that could benefit from working with NeSI Research Software Engineers or our Data Engineer? Learn more about what kind of support they can offer and get in touch by emailing support@nesi.org.nz.