Difference between revisions of "Curation of ZINC-22 3D"

From DISI
Jump to navigation Jump to search
 
Line 22: Line 22:
 
* blue = 3D build running
 
* blue = 3D build running
 
* yellow = attention required when resources allow
 
* yellow = attention required when resources allow
* bold = unpack/repack in process.  
+
* bold = unpack/repack in process.
 +
* orange = on hold /suspended.
  
 
== Copies ==  
 
== Copies ==  

Latest revision as of 19:10, 17 June 2022

These are notes from John to Max to help guide the curation of ZINC-22 3D. Each topic has its own subject heading.


General remarks

  • I suggest using screens on gimel and then logging in to disk-hosting machine.
  • /nfs/exd is where files live and README describing the layers

Once per day - 10 mins?

  • check for disk full conditions (our cluster and Wynton)
df -h /nfs/ex? /nfs/db? | sort -k 5 -r |head
beegfs-ctl --getquota --gid bks
  • check that the queues on our cluster and Wynton are full
  • check whether any preparation processes have finished (and move on to next step)

Maintain current info in spreadsheet

https://docs.google.com/spreadsheets/d/14QpCiB78feWPQKT5LkpfKDNR6VCAMbEs7Md-QMX0glc/edit#gid=298497589

color coding

  • green = complete
  • blue = 3D build running
  • yellow = attention required when resources allow
  • bold = unpack/repack in process.
  • orange = on hold /suspended.

Copies

  • we make one copy to AWS
  • one copy to OCI
  • one copy to Wynton
  • we have been retaining 50-mers. We should delete them.


Unpack/repack process

  • ls -ld /nfs/ex?/zinc-22?/H??/H??[PM]???* - this tells you where the 50-mers live
  • ls /nfs/ex?/zinc-22?/H??/H??[PM]???*/out - this tells you how many top level dirs.
  • look at the last one. ls -ld * | wc -l helps you check finished / fully transferred
  • csh cmd-? H??P???... unpack
  • sdi6.script H?? sdi
  • csh master-f? H?? <a-z>. repack
  • mv H?? H??.old when repack complete
  • rm -rf H??.old when sure complete
  • start another one.