Andrew Chen is developing an algorithm that will identify unnecessary video frames, such as static background shots in surveillance camera footage, faster and more accurately than current methods.

Supporting new innovations in image processing

“Having access to NeSI computing resources means that we can keep up with the state-of-the-art research in this area, otherwise we simply would not be able to run our code in a reasonable time.”

One of the most essential and important tasks in image processing and computer vision research is background estimation. This is the task of determining what parts of a video are static background, and therefore not of interest, so that those parts can be ignored to save computation time.

Consider a surveillance camera that is monitoring a hallway.

“Most of the time, there is nothing happening, and so it would be wasteful to run a computationally expensive person detection algorithm across every frame of the video,” says Andrew Chen, a PhD Candidate in the department of Electrical and Computer Engineering at the University of Auckland.

Instead, he explains, they can use comparatively lighter background estimation to detect when there has been a change (e.g. a moving object), and then run person detection on that part of the frame to see if it is a person or not. Using NeSI supercomputer resources, he is developing and refining a new superpixel-based algorithm that completes the background estimation process faster and more accurately.

Andrew Chen NeSI case study research figure
The figure represents how the SuperBE algorithm isolates the background and foreground in an image.

To develop this algorithm, Andrew and his colleagues had to run the algorithm against a large benchmark dataset, which takes about three to four hours on a standard desktop computer. The need for NeSI came when they wanted to take their development and analysis a step further and find parameter values for the algorithm.

Even though they did discrete sampling of the values (i.e. not checking every possible value), they still needed to run the algorithm a few hundred times in order to identify the relationship between each parameter and accuracy, as well as compare accuracies across the various parameters. They wanted to ensure they had the best possible accuracy in order to publish their best results in papers.

“This algorithm has five to six parameters, and there are a lot of different combinations of these parameters that need to be checked,” says Andrew. “Multiply three to four hours by a couple of hundred and it would have taken a really long time on a single desktop PC.”

Having access to NeSI meant that they were able to run their jobs on multiple cores at once in parallel. Due to the size of the job, they figured it would take a few days to complete.

“We were surprised when the scheduler allocated more than a hundred cores, and allowed us to complete in one day what otherwise would have taken weeks on a PC in the lab,” says Andrew.

Along the way, NeSI staff were helping in the background. For example, at one point, when Andrew was having trouble with some of his code, a NeSI staff member even spotted a bug in one line of the code.

“All of the NeSI staff members that I interacted with were super helpful,” he says. “I attended a tutorial that was expertly delivered and gave me the necessary skills to access and use the supercomputer cluster, and then when we needed specific software (OpenCV) to be installed on the cluster, the NeSI staff did it very quickly.”

Andrew and his team used NeSI for the later stages of their project as well, to run their benchmarking tests comparing their algorithm with others in the literature. They’ve published their findings in the Journal of Real Time Processing.

With much of the current groundbreaking work in computer vision reliant on access to powerful computing resources, NeSI has become an important tool for researchers such as Andrew to keep pace with the field.

“In the computer vision academic community, no one blinks an eye at the thought of needing hundreds or thousands of cores in order to run really complex deep convolutional neural networks, or spending thousands of dollars on GPUs,” he says. “Having access to NeSI computing resources means that we can keep up with the state-of-the-art research in this area, otherwise we simply would not be able to run our code in a reasonable time.”


Do you have an example of how NeSI platforms have supported your work? We’re always looking for projects to feature as a case study. Get in touch by emailing

Next Case Study

Understanding environmental processes on a microbial level

"High performance computing is essential for researchers working with large and complex genomic datasets. "