Difference between revisions of "Curation of ZINC-22 3D"

From DISI
Jump to navigation Jump to search
m (asdf)
Line 8: Line 8:
 
== Once per day - 10 mins? ==
 
== Once per day - 10 mins? ==
 
* check for disk full conditions (our cluster and Wynton)
 
* check for disk full conditions (our cluster and Wynton)
 +
df -h /nfs/ex? /nfs/db? | sort -k 5 -r |head
 +
 
* check that the queues on our cluster and Wynton are full
 
* check that the queues on our cluster and Wynton are full
* check whether any preparation processes have finished (and move on to next step)  
+
* check whether any preparation processes have finished (and move on to next step)
  
 
== Maintain current info in spreadsheet ==  
 
== Maintain current info in spreadsheet ==  

Revision as of 18:59, 17 June 2022

These are notes from John to Max to help guide the curation of ZINC-22 3D. Each topic has its own subject heading.


General remarks

  • I suggest using screens on gimel and then logging in to disk-hosting machine.
  • /nfs/exd is where files live and README describing the layers

Once per day - 10 mins?

  • check for disk full conditions (our cluster and Wynton)
df -h /nfs/ex? /nfs/db? | sort -k 5 -r |head
  • check that the queues on our cluster and Wynton are full
  • check whether any preparation processes have finished (and move on to next step)

Maintain current info in spreadsheet

https://docs.google.com/spreadsheets/d/14QpCiB78feWPQKT5LkpfKDNR6VCAMbEs7Md-QMX0glc/edit#gid=298497589

color coding

  • green = complete
  • blue = 3D build running
  • yellow = attention required when resources allow
  • bold = unpack/repack in process.

Copies

  • we make one copy to AWS
  • one copy to OCI
  • one copy to Wynton
  • we have been retaining 50-mers. We should delete them.


Unpack/repack process

  • ls -ld /nfs/ex?/zinc-22?/H??/H??[PM]???* - this tells you where the 50-mers live
  • ls /nfs/ex?/zinc-22?/H??/H??[PM]???*/out - this tells you how many top level dirs.
  • look at the last one. ls -ld * | wc -l helps you check finished / fully transferred
  • csh cmd-? H??P???... unpack
  • sdi6.script H?? sdi
  • csh master-f? H?? <a-z>. repack
  • mv H?? H??.old when repack complete
  • rm -rf H??.old when sure complete
  • start another one.