For details on how to back up data to either TigerData or the cloud, see 🏦Data Backups. Everyone must have a backup plan in place.
Princeton HPC File Systems
Title
Shared Home
Shared Scratch
Shared Data Storage
Tiger
/home/ROSENGROUP
/scratch/gpfs/ROSENGROUP
/tigerdata/ROSENGROUP/work/tiger
Della
/home/ROSENGROUP
/scratch/gpfs/ROSENGROUP
/tigerdata/ROSENGROUP/work/della
Neuronic
N/A
/scratch/rosengroup
/n/fs/rosengroup
Adroit
N/A
/scratch/network/ROSENGROUP
/tigerdata/ROSENGROUP/work/adroit
To view file quotas, use checkquota. If we are in danger of running low, inform Andrew as soon as possible so we can take prompt action.
Home Directories
You have a /home/$USER directory, which is the same as doing ~/ and is where you log into when accessing the machine. On Tiger and Della, we also have a shared group home directory that contains group software and module files that are accessible to all group members. The /home directory — both your own and that of the research group — is limited in terms of file storage but is backed up by Research Computing. In general, /home should not be used to store or interact with research data. It is much slower than /scratch/gpfs and relatively small. Calculations should not be run in home.
On Tiger and Della, anything in /home/ROSENGROUP/software/common/bin is automatically added to the PATH of all group members. Anything in /home/ROSENGROUP/Modules/modulefiles-shared is automatically available as a module.
Scratch Directories
The scratch directory is where your I/O-based calculations should be run. It is a fast disk space with a generous storage capacity. The scratch space — by definition — is not backed up and is not meant for long-term data storage. On Tiger and Della, the scratch space is not purged, but if you accidentally delete the data or the system dies, the data will be gone forever. On Neuronic, data in the /scratch directory is automatically purged after two weeks.
Since scratch has virtually unlimited file space, you should also use scratch to store large datasets that are too big to store elsewhere or are publicly available and therefore do not need to be backed up.
If you haven't done so already, it is helpful to define your scratch directory as an environment variable named SCRATCH in your ~/.bashrc such that you can easily do cd $SCRATCH as a shorthand:
exportSCRATCH=/scratch/gpfs/ROSENGROUP/$USER
Data Storage Directories
Sometimes, you may accidentally copy files you don't need in the data storage directory. Let's say you copied a bunch of CHG files you realized you don't need to store in a given directory. If you run find . -name "CHG", you will find all CHG files in all subdirectories from your current working directory. Removing them is as easy as find . -name "CHG" -delete. Being diligent about what you need vs. don't need will help us save space.
When calculations finish, files need to eventually be copied or moved from the scratch directory to a semi-permanent data storage directory. On Tiger and Della, the data storage directory is the TigerData service (found at /tigerdata), which is backed up but is less performant than the scratch directory. I/O-based calculations should never be run in the TigerData directory itself. We currently have 10 TB of space in TigerData, but we can request more if needed. Please be mindful of your data usage but don't jeopardize data security in the name of being frugal. Note that the compute nodes cannot access the /tigerdata directory.