Files and Data
Within each facility there are high performance parallel file systems to support data input/output when running computational jobs along with hierarchical storage management to support transparent access to nearline storage.
High speed file transfers are available between facilities.
We operate a national file storage and sharing system that allows researchers to share files and datasets of up to terabytes in size. Access controls give access to sensitive files. The system has high availability providing resilience from failure and while the presentation of data looks to be logically one system, it has physically distributed storage resources across facilities nationally with files replicated for resilience, providing optimal performance during HPC processing from being able to access file replicas locally.
Note that NeSI does not currently provide a long term archive service.
A summary of file and data storage systems supported by each facility follows:
| Parallel file system | Hierarchical storage | File transfers | National file system | |
| BlueFern, University of Canterbury | x | x | x | x |
| Centre for eResearch, University of Auckland | x | x | x | x |
| FitzRoy High Performance Computing Facility, NIWA | x | x | x |
Several facilities nationally are also planning to implement file transfers and the national file system.
Security and Data Ownership
NeSI has stringent privacy and storage policies that conform to international standards and best practices.
Security requirements specific to a proposal should be listed during the application process. This includes instructions for how to deal with processing proprietary or sensitive data.
Note that NeSI does not provide a long term archive service.
