Curation of ZINC-22 3D: Difference between revisions
Jump to navigation
Jump to search
m (→Once per day - 10 mins?: asdf) |
m (→color coding: asdf) |
||
| (2 intermediate revisions by the same user not shown) | |||
| Line 9: | Line 9: | ||
* check for disk full conditions (our cluster and Wynton) | * check for disk full conditions (our cluster and Wynton) | ||
df -h /nfs/ex? /nfs/db? | sort -k 5 -r |head | df -h /nfs/ex? /nfs/db? | sort -k 5 -r |head | ||
beegfs-ctl --getquota --gid bks | |||
* check that the queues on our cluster and Wynton are full | * check that the queues on our cluster and Wynton are full | ||
* check whether any preparation processes have finished (and move on to next step) | * check whether any preparation processes have finished (and move on to next step) | ||
| Line 20: | Line 22: | ||
* blue = 3D build running | * blue = 3D build running | ||
* yellow = attention required when resources allow | * yellow = attention required when resources allow | ||
* bold = unpack/repack in process. | * bold = unpack/repack in process. | ||
* orange = on hold /suspended. | |||
== Copies == | == Copies == | ||
Latest revision as of 19:10, 17 June 2022
These are notes from John to Max to help guide the curation of ZINC-22 3D. Each topic has its own subject heading.
General remarks
- I suggest using screens on gimel and then logging in to disk-hosting machine.
- /nfs/exd is where files live and README describing the layers
Once per day - 10 mins?
- check for disk full conditions (our cluster and Wynton)
df -h /nfs/ex? /nfs/db? | sort -k 5 -r |head beegfs-ctl --getquota --gid bks
- check that the queues on our cluster and Wynton are full
- check whether any preparation processes have finished (and move on to next step)
Maintain current info in spreadsheet
color coding
- green = complete
- blue = 3D build running
- yellow = attention required when resources allow
- bold = unpack/repack in process.
- orange = on hold /suspended.
Copies
- we make one copy to AWS
- one copy to OCI
- one copy to Wynton
- we have been retaining 50-mers. We should delete them.
Unpack/repack process
- ls -ld /nfs/ex?/zinc-22?/H??/H??[PM]???* - this tells you where the 50-mers live
- ls /nfs/ex?/zinc-22?/H??/H??[PM]???*/out - this tells you how many top level dirs.
- look at the last one. ls -ld * | wc -l helps you check finished / fully transferred
- csh cmd-? H??P???... unpack
- sdi6.script H?? sdi
- csh master-f? H?? <a-z>. repack
- mv H?? H??.old when repack complete
- rm -rf H??.old when sure complete
- start another one.