CoreHPC
Jump to navigation
Jump to search
We are still learning about FAC/coreHPC.
You need one account on coreHPC. You access "FAC" via a disk mounted on coreHPC only: /mnt/fac/CX900007_DS1/
- We have 50TB on FAC for permanent storage. Use this for software, dockfiles, databases that you need to last for more than a few weeks.
- coreHPC is where you run jobs, using Slurm.
- your home directory is tiny. Use it for scripts and notes. It won't hold much else.
- there are two kinds of scratch space. /tmp on each node allows you to write data locally while the job is running. You must repariate it back to longer-term storage because /tmp gets wiped when your job ends.
- there is /mnt/scratch/user/<yourid>/ scratch space. This is very fast disk, but not as fast as /tmp. The data will likely persist for weeks or months, but there is no guarantee.
- the FAC storage is your best bet for long term, but we "only" have 50 TB. So you cannot store big chunks of ZINC for a long time. Please get in the habit of compressing and moving your data off FAC back to our clusters when your project is complete.
- to make a request to the CoreHPC managers, use this link: [1]
- We collectively refer to FAC/coreHPC as "Cluster 8".
- There is an open question of how to work efficiently with coreHPC. We are currently in an experimental phase.
- generally, the idea is: Our clusters in Byers Hall are persistent storage.
- move data you need to coreHPC, run the jobs, collect the output, and then move/delete to shrink your footprint when done.
- ZINC thus lives on our cluster (currently hosted on cluster 2, cross mounted read only to cluster 7).
- a copy of 50% of ZINC lives on Wynton/cluster 3.
- The pipe to FAC is currently at best 10 Gbps. We are working on getting more.
- to get to corehpc
ssh <user>@chpc-ucsf-bastion-vm1.corehpc.ucsf.edu
- to get to slurm head node
ssh chpc-ucsf-login-vm1
- disks are
/mnt/scratch/user/<user>. /mnt/home/<user> /home/remote/<user> ; you land here when you login
- To check disk usage
- For /home/remote and /scratch
quota -s
- For /mnt/fac/CX900007_DS1
df -h /mnt/fac/CX900007_DS1
Learn more
https://wiki.library.ucsf.edu/spaces/CHPC/pages/720396955/CoreHPC+Access+Primer https://it.ucsf.edu/service/corehpc https://wiki.library.ucsf.edu/spaces/CHPC/pages/736476672/CoreHPC+Submission+Examples https://wiki.library.ucsf.edu/spaces/CHPC/pages/751928275/CoreHPC+Technical+Overview
Anyone who uses coreHPC, please take notes, update this and other pages, and teach the rest of the lab how to use it. Thank you.
how to do remote mounts
https://wiki.library.ucsf.edu/spaces/CHPC/pages/720396955/CoreHPC+Access+Primer
https://it.ucsf.edu/how-to/fac-drive-mapping-instructions?check_logged_in=1#linux
smb://capstorage.fac.ucsf.edu/CX900007_DS1. lab storage /mnt/fac/CX900007_DS1 fac
- tiny.ucsf.edu/getfac - this is how you file tickets for FAC.
our shared storage is on /home/remote/jirwin3/fac
It is 1 TB. It is group writable for our group and our group alone.