IFP Filtering on Wynton

2024-10-02T22:11:06Z

Sevigneron: Created page with "Seth Vigneron Oct 2024 IFP filtering on wynton proceeds in almost the exact same way as on our gimel cluster. Note the conda environment sourced uses the same older versions of python and LUNA as JK's original scripts on gimel, so wynton is not using any updated versions of any software and still runs the same IFP protocol. 1. Run getposes script following your screen. 2. Make split mol2 files to be run in parallel ls $PWD/poses_extract_for_getposes_parallel_*mol2 >"

How to do parameter scanning

2024-08-02T15:35:01Z

Sevigneron:

== Manually Generating ES/LD Combinations ==

written by Reed Stein, updated 12/2019 so users can specify different low dielectric and ligand desolvation sphere radii

To generate different sized low dielectric and ligand desolvation thin sphere combinations, you must first run blastermaster on your protein/ligand of interest. Once this is finished, make a new directory. Change into the new directory
and run the following command:

python ~rstein/zzz.scripts/DOCK_prep_scripts/new_0001_generate_ES_LD_generation.py -p ${FULL_ORIGINAL_BLASTERMASTER_PATH}

The full path to your original blastermaster path is the only required argument. However, you can modify other default values including:
1) Radius of low dielectric spheres you want to run: use the -es flag.
Default radius for electrostatic spheres is [1.0, 1.3, 1.5, 1.7, 1.9]
2) Radius of ligand desolvation spheres you want to run: use the -ld flag.
Default radius for ligand desolvation spheres is [0.1, 0.2, 0.3, 0.4, 0.5]
3) DMS density: use the -d flag. Default is 10.0 to ensure that the surface is sufficiently coated with spheres when using
a smaller sphere radii (typically <0.5 Å). This can be reduced to 1.0 if only larger sphere radii are used.
4) Distance from crystallographic ligand: use the -xd flag. Default is 2 Å.

If you wanted to change all, you could use:

python ~rstein/zzz.scripts/DOCK_prep_scripts/new_0001_generate_ES_LD_generation.py -p ${FULL_ORIGINAL_BLASTERMASTER_PATH}
-es 1.1 1.2 1.3 -ld 0.1 0.3 -d 10 -xd 4

This script will submit ligand desolvation jobs to the queue. This should take 15-30 minutes. Once finished, check your spheres and grids by following the steps here:

http://wiki.docking.org/index.php/Protein_Target_Preparation_Updated

Then run the following script to combine the ES/LD grids in all combinations:

python ~rstein/zzz.scripts/DOCK_prep_scripts/new_0002_combine_es_ld_grids_into_combos.py -p ${FULL_ORIGINAL_BLASTERMASTER_PATH}

This will create a new directory called "combo_directories" with docking-ready directories inside. The directories will have the name format "es_{ES_sphere_radius}_ld_{LD_sphere_radius}". "def" refers to grids taken from your ${FULL_ORIGINAL_BLASTERMASTER_PATH} directory.

If you are going to be docking with DOCK3.8, then the INDOCK files must be inside dockfiles. To set up these directories, run this script instead:

~svigneron/work_exj/scripts/new_0002_combine_es_ld_grids_into_combos_dock38.py -p ${FULL_ORIGINAL_BLASTERMASTER_PATH}

These directories are now ready for docking.

== Blastermaster Parameter Scanning ==

Written by Jiankun Lyu, 2017/01/18

4/16/2019 - this needs to be updated and will result in discontinuous thin spheres at low radii!
Use the tutorial above instead. Or use this link to run blastermaster manually:

http://wiki.docking.org/index.php/Using_thin_spheres_in_DOCK3.7

The hierarchy of the directories:

thin_spheres_parameter_scanning----- std_dockprep
|
|------ dockfiles
| |
| |----- working
| |
| ------ rec.pdb, xtal-lig.pdb, INDOCK and other files generated balstermaster.py
|
------- script ------ dockprep_thin_spheres_in_batches.csh
|
|------ submit_dockprep_thin_spheres.csh
|
|------ dockprep_thin_spheres.csh
|
|------ lig-decoy_enrichment.csh
|
|------ combineScoresAndPoses.csh
|
|------ AUCplot_of-lig-decoys.csh
|
|------ mk_matrix_logAUC.py
|
|------ sph_lib.py
|
|------ pdb_lib.py
|
------- close_sph.py

1) Make those directories above.
mkdir thin_spheres_parameter_scanning
cd thin_spheres_parameter_scanning
mkdir std_dockprep
mkdir script

2) Run blastermaster.py in std_dockprep. This will generate two directories: working and dockfiles

3) Download sph_lib.py, pdb_lib.py and close_sph.py files into the script directory
cd script
curl http://docking.org/~tbalius/code/for_dock_3.7/sph_lib.py > sph_lib.py
curl http://docking.org/~tbalius/code/for_dock_3.7/pdb_lib.py > pdb_lib.py
curl http://docking.org/~tbalius/code/for_dock_3.7/close_sph.py > close_sph.py

4) Copy scripts from my path, and modify as necessary.

cd script

cp /mnt/nfs/ex5/work/jklyu/large_scale_docking/DRD2/struct_20180322/A122I_add_polarH_mini_HID/thin_spheres_parameter_scanning/scripts/*dockprep* .

cp /mnt/nfs/reshwork/jklyu/D2R/scripts/lig-decoy_enrichment_submit.csh .
cp /mnt/nfs/reshwork/jklyu/D2R/scripts/combineScoresAndPoses.csh .
cp /mnt/nfs/reshwork/jklyu/D2R/scripts/mk_matrix_logAUC.py .

5) Run parameter scanning.
cd ../ # go back to thin_spheres_parameter_scanning folder
csh /path/to/script/dockprep_thin_spheres_in_batches.csh /path/to/script/ /path/to/std_dockprep

Note:- you can edit dockprep_thin_spheres_in_batches.csh to include more CPUs in Job Bound

6) make the following subfolders

mkdir ligands-decoys
cd ligands-decoys
mkdir ligands
mkdir decoys

now copy your decoys.db2.gz to decoys
now copy your ligands.db2.gz to ligands
now copy decoys.smi to the folder
now copy ligands.smi to the folder

7) Submit DOCK and enrichment calculation.
csh /path/to/script/lig-decoy_enrichment.csh

8) Combine and analyze the docking results.
csh /path/to/script/combineScoresAndPoses.csh #1st change the path inside the script to your own dir
csh /path/to/script/AUCplot_of-lig-decoys.csh #1st change the path inside the script to your own dir

9) Visualize the logAUC by heatmap.
python /path/to/script/mk_matrix_logAUC.py

Generating decoys (Reed's way)

2024-07-30T18:59:39Z

Sevigneron:

Written by Reed Stein on April 3, 2018.

updated 5/3/2019

updated 8/15/2019

updated 3/6/2020

updated 5/18/2020

This pipeline will generate property-matched decoys for a set of ligand SMILES. To build ligands yourself, see "ligand prep" in:
http://wiki.docking.org/index.php/DOCK_3.7_tutorial_%28Anat%29

All scripts for this tutorial can be found in:
/mnt/nfs/home/rstein/zzz.scripts/new_DUDE_SCRIPTS/

Before running any scripts, make sure to source the current version of Python
source /nfs/soft/python/envs/complete/current/env.csh

Additionally, JChem needs to be sourced in your ~/.cshrc file with the command:
source /nfs/soft/jchem/current/env.csh

If the below script still not run:
source /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk/env.csh
== Querying ZINC for Protomers ==

This procedure generates decoys for your input ligands by searching through 3D conformers that are already built in ZINC. This procedure is advised if you want decoys to be charge-matched to ligands.

=== Step 1) Setting up directories for Protomers ===

Before starting, you need a SMILES file with the format (SMILES first, unique ID second):
S(Nc1c(O)cc(C(=O)O)cc1)(c2c(scc2)C(=O)O)(=O)=O 116

You also need an input file named "decoy_generation.in" with the following lines:

PROTONATE YES
MWT 0 125
LOGP 0 3.6
RB 0 5
HBA 0 4
HBD 0 3
CHARGE 0 2
LIGAND TC RANGE 0.0 0.35
MINIMUM DECOYS PER LIGAND 20
DECOYS PER LIGAND 50
MAXIMUM TC BETWEEN DECOYS 0.8
TANIMOTO YES

If your input ligand SMILES file is already protonated as you want it, set "PROTONATE NO".

If you want your input ligand SMILES protonated, only protomer SMILES with unique properties will be kept for generating decoys. Therefore, if you have one ligand that exists in 4 tautomers, all of which have identical molecular weight, cLogP, # rotatable bonds, # H-bond acceptors and donors, and net charge, only one will be maintained for decoy matching. This doesn't apply if you set "PROTONATE NO".

This file specifies that for each ligand protomer, at least 20 decoys will be retrieved with the following properties:
- within +/- 125 Daltons
- within +/- 3.6 logP
- within +/- 5 rotatable bonds
- within +/- 4 hydrogen bond acceptors
- within +/- 3 hydrogen bond donors
- within +/- 2 charge
- 0.35 or less Tanimoto
- minimum 20 decoys per ligand protomer, if available
- preferred 50 decoys per ligand protomer, if available
- the maximum TC between decoy molecules should be 0.8
- "TANIMOTO" refers to whether a Tanimoto calculation should be performed - see step 3 for when this is necessary

These are arbitrary, and you can input your desired minimum and maximum values that decoys can differ by, relative to the ligands.

Once you have created this file, run the following command to create the decoy generation directory:

python /mnt/nfs/home/rstein/zzz.scripts/new_DUDE_SCRIPTS/0000_protonate_setup_dirs.py {SMILES_FILE} {NEW_DIR_NAME}

Provide a directory name that you want in place of {NEW_DIR_NAME}. This will create the directory with subdirectories named
"ligand_${number}" for each of the ligands in the SMILES file you input.

=== Step 2) Retrieving protomer decoys from ZINC15 ===

If you have edited the "decoy_generation.in" file which is now located in {NEW_DIR_NAME} as you want, you can run the following command:

python /mnt/nfs/home/rstein/zzz.scripts/new_DUDE_SCRIPTS/0001_qsub_generate_decoys.py {NEW_DIR_NAME}

This should take 15 minutes to an hour, depending on how many ligands you input.

=== Step 3) Assigning accepted protomer decoys to each ligand protomer ===

We can assign the property-matched decoys to the ligand protomers. Make sure you have the "decoy_generation_input.in" file from before in {NEW_DIR_NAME}.

To filter the decoys, run the following command:
python /mnt/nfs/home/rstein/zzz.scripts/new_DUDE_SCRIPTS/0002_qsub_filter_decoys.py {NEW_DIR_NAME}

This will run on the queue. A log file called "FILTER_DECOYS.log" will be generated in {NEW_DIR_NAME} with information and any errors.

If you don't get enough decoys, the "decoy_generation.in" file can be modified by changing "MAXIMUM TC BETWEEN DECOYS", "MINIMUM DECOYS PER LIGAND", etc.
To not run the time-consuming Tanimoto calculation between all decoys again, simply add/change this in the "decoy_generation.in" file:

TANIMOTO NO

If you set Tanimoto to "NO", make sure that your {NEW_DIR_NAME} still has the original files:

"test_ligdecoy_smiles.smi"
"cluster_head.list"

Otherwise, this step will not run.

If these original files still remain, this will skip the Tanimoto calculation step, and filter property matched decoys based on the new parameters in the "decoy_generation.in" file.

If this has completed successfully, you should see files in your {NEW_DIR_NAME} with the format "{LIGAND_ID}_final_property_matched_decoys.txt". These files contain the ligands and their properties, as well as property-matched decoys that have been assigned to them. These files have the format "SMILES", "ZINC ID", "logP", "#Rotatable Bonds", "# Hydrogen Bond Donors", "# Hydrogen Bond Acceptors", "Charge", "Protomer SMILES", and "Tanimoto Coefficient to Ligand".

There should also be files with the format "{LIGAND_ID}_replacements.txt", which include extra property-matched decoys that were assigned to that ligand.

If you still cannot get enough decoys for your ligands, consider reducing the number of ligands you have by clustering, for example, or using the SMILES decoy generation below, which is not limited to only molecules that are already built in ZINC15.

=== Step 4) Copying decoy .db2.gz files into your directories ===

To copy property-matched decoys into your own directory of choice, run the following command:

python /mnt/nfs/home/rstein/zzz.scripts/new_DUDE_SCRIPTS/0003_copy_decoys_to_new_dir.py {NEW_DIR_NAME} {COPY_TO_DIR}

where {COPY_TO_DIR} is a new directory that will be created where your decoys will be copied into. In this directory, two subdirectories will be created:
"ligands" - this includes the input ligands for which there are X number property matched decoys (these are all ligands with "{LIGAND_ID}_final_property_matched_decoys.txt" files in {NEW_DIR_NAME})
"decoys" - this will include the decoy .db2.gz files for docking and "decoys.smi" which contains all the SMILES strings for property matched decoys

IMPORTANT: It is possible that all of your ligand protomers were not matched to property-matched decoys. The "ligands.smi" file in {COPY_TO_DIR} will not include these. Make
sure you do not dock these if you calculate enrichment values.

== Querying ZINC for SMILES ==

This procedure generates decoys for your input ligand SMILES by finding decoy SMILES in ZINC that are property-matched. This procedure will provide decoy SMILES that you can build yourself into 3D models for docking. If you would like to query ZINC for decoy SMILES so that you can build decoys yourself or if your ligands are >400 Da, start here. If not, go to "Querying ZINC for Protomers" to generate decoys that already have 3D models.

=== Step 1) Setting up SMILES directory ===

Before starting, you need a SMILES file with the format (SMILES first, ID second):
S(Nc1c(O)cc(C(=O)O)cc1)(c2c(scc2)C(=O)O)(=O)=O 116

You also need an input file named "decoy_generation.in" with the following lines:

SMILES YES
PROTONATE YES
MWT 0 125
LOGP 0 3.6
RB 0 5
HBA 0 4
HBD 0 3
CHARGE 0 2
LIGAND TC RANGE 0.0 0.35
MINIMUM DECOYS PER LIGAND 20
DECOYS PER LIGAND 50
MAXIMUM TC BETWEEN DECOYS 0.8
TANIMOTO YES
GENERATE DECOYS 750

If your input ligand SMILES file is already protonated as you want it, set "PROTONATE NO". "SMILES" tells the function you want to query ZINC for SMILES, not built protomers.

This file specifies that for each ligand protomer, {MINIMUM DECOYS PER LIGAND} to {DECOYS PER LIGAND} decoys will be retrieved with the following properties:
- within +/- 125 Daltons
- within +/- 3.6 logP
- within +/- 5 rotatable bonds
- within +/- 4 hydrogen bond acceptors
- within +/- 3 hydrogen bond donors
- within +/- 2 charge
- 0.35 or less Tanimoto

"GENERATE DECOYS" specifies how many potential decoys you want to check for property matching with your ligands. A smaller number results in faster decoy generation, but a smaller pool of potential decoys to compare your ligand against. A larger number results in slower decoy generation, and greater likelihood of property-matched decoys for all your ligands.

As with protomers, "MINIMUM DECOYS PER LIGAND" refers to the minimum number of decoys you want for each ligand protomer;

"DECOYS PER LIGAND" refers to your preferred number of decoys for each ligand protomer;

"MAXIMUM TC BETWEEN DECOYS" refers to the maximum Tc allowed between decoys (the lower, the more dissimilar your decoys will be);

and "TANIMOTO" refers to whether the ligand-decoy full Tc matrix should be calculated - this must be done at least once and should not be set to "NO" unless you are re-running step 3.

These are arbitrary, and you can input your desired minimum and maximum values that decoys can differ by, relative to the ligands.

Once you have created this file, run the following command to create the decoy generation directory:

python /mnt/nfs/home/rstein/zzz.scripts/new_DUDE_SCRIPTS/0000_protonate_setup_dirs.py {SMILES_FILE} {NEW_DIR_NAME}

Provide a directory name that you want in place of {NEW_DIR_NAME}. This will create the directory with subdirectories named
"ligand_${number}" for each of the ligands in the SMILES file you input.

=== Step 2) Retrieving SMILES decoys from ZINC15 ===

If you have edited the "decoy_generation.in" file which is now located in {NEW_DIR_NAME} as you want, you can run the following command:

python /mnt/nfs/home/rstein/zzz.scripts/new_DUDE_SCRIPTS/0001_qsub_generate_decoys.py {NEW_DIR_NAME}

Jobs will run for 15 minutes to 1-2 hours depending on how many ligands you input.

***There was a new bug in Reed's original script that Brendan Hall was able to fix (July 30, 2024), run this script instead:

python /nfs/home/bwhall61/work/decoy_gen_improvement/0001_qsub_generate_decoys.py {NEW_DIR_NAME}

=== Step 3) Assigning decoys to ligands ===

To assign property matched decoys to your ligand protomers, run the following command:

python /mnt/nfs/home/rstein/zzz.scripts/new_DUDE_SCRIPTS/0002_qsub_filter_decoys.py {NEW_DIR_NAME}

*** Again Brendan Hall was able to fix the errors in the above script, run this instead:
cd {NEW_DIR_NAME}
source /nfs/home/bwhall61/.python_envs/pulp/bin/activate
python /nfs/home/bwhall61/work/decoy_gen_improvement/filter_decoys.py

this may take some time to run, so it is recommended to run in a screen.

This will run on the queue. As with "Querying ZINC for Protomers":

If you don't get enough decoys, the "decoy_generation.in" file can be modified by changing "MAXIMUM TC BETWEEN DECOYS", "MINIMUM DECOYS PER LIGAND", etc.
To not run the time-consuming Tanimoto calculation between all ligands and decoys again, simply add/change this in the "decoy_generation.in" file:

TANIMOTO NO

If you set Tanimoto to "NO", make sure that your {NEW_DIR_NAME} still has the original files:

"test_ligdecoy_smiles.smi"
"cluster_head.list"

Otherwise, this step will not run.

If these original files still remain, this will skip the Tanimoto calculation step, and filter property matched decoys based on the new parameters in the "decoy_generation.in" file.

If this has completed successfully, you should see files in your {NEW_DIR_NAME} with the format "{LIGAND_ID}_final_property_matched_decoys.txt". These files have the format "SMILES", "ZINC ID", "logP", "#Rotatable Bonds", "# Hydrogen Bond Donors", "# Hydrogen Bond Acceptors", "Charge", "Protomer ID", and "Tanimoto Coefficient to Ligand".

These files contain the ligands and their properties, as well as property-matched decoys that have been assigned to them. There should also be files with the format "{LIGAND_ID}_replacements.txt", which include extra property-matched decoys that were assigned to that ligand.

=== Step 4) Setting up ligand/decoy directories for building SMILES ===

If you have queried ZINC for SMILES, you need to build the decoys yourself. To write the SMILES file, run the following command:

python /mnt/nfs/home/rstein/zzz.scripts/new_DUDE_SCRIPTS/0003b_write_out_ligands_decoys.py {NEW_DIR_NAME} {COPY_TO_DIR}

This will create {COPY_TO_DIR} with two subdirectories, "ligands" and "decoys" as well as SMILES files for:

ligands.smi - this includes the input ligands for which there are X number property matched decoys (these are all ligands with "{LIGAND_ID}_final_property_matched_decoys.txt" files in {NEW_DIR_NAME})
decoys.smi - this includes the canonicalized property-matched decoy SMILES
decoy_protomers.smi - this includes the actual property-matched decoy protomer SMILES

SMILES for decoys can now be built.

For decoy building, use the following command:

setenv DOCKBASE /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk
source /nfs/soft/dock/versions/dock37/DOCK-3.7-trunk/env.csh
${DOCKBASE}/ligand/generate/build_database_ligand.sh -H $ph decoy_protomers.smi --pre-tautomerized --no-db

If not all decoys successfully build, more property matched decoys can be taken from the "{LIGAND_ID}_replacements.txt" files. Additionally, you can build decoys without the --pre-tautomerized flag:

${DOCKBASE}/ligand/generate/build_database_ligand.sh -H $ph decoys.smi --no-db

This will produce all protomers of each decoy, including the property-matched decoy protomer.

== Visualizing Decoy Properties ==
=== Visualizing property distributions ===

To visualize the distributions of molecular properties of matched decoys relative to the ligands, run the following command:
python /mnt/nfs/home/rstein/zzz.scripts/new_DUDE_SCRIPTS/0004_plot_properties.py {NEW_DIR_NAME}

There will be 6 images in {NEW_DIR_NAME} for molecular weight, logP, number of rotatable bonds, number of hydrogen bond donors, number of hydrogen bond acceptors, and net charge of ligands and decoys.

=== Visualizing decoy Tanimotos to ligands ===

To visualize how different the matched decoys are to the input ligands, run the following command:
python /mnt/nfs/home/rstein/zzz.scripts/new_DUDE_SCRIPTS/0005_plot_tanimoto_to_lig.py {NEW_DIR_NAME}

There will be a box and whisker plot image in {NEW_DIR_NAME} showing the Tanimotos calculated between each ligand and all decoys.

Generating decoys (Reed's way)

2024-07-30T18:58:30Z

Sevigneron:

Hit picking party

2024-06-06T17:10:28Z

Sevigneron: /* Before the party */

Before we buy compounds, we have a hit picking party.

{{TOCright}}

= Before the party =
The investigator who performed the virtual screen looks critically at the results, selecting perhaps a dozen or two interesting compounds from among the top 500. Several copies of the top 500 list are printed, sometimes with additional supporting documentation, and distributed to the participants. Check [[CSD]] data for compound pose information. The [[Pka|pKa]] of each molecule can be checked. Think about specific ligand-protein interactions (A Medicinal Chemist’s Guide to Molecular Interactions, J. Med. Chem. 2010, 53, 5061–5084 DOI: 10.1021/jm100112j, can be a nice resource for thinking of what to look for). Think about [[reactive groups]] for molecules. Think about PAINS http://pasilla.health.unm.edu/tomcat/biocomp/smartsfilter

= During the party =
We take a good look at the site, and any crystallographic ligands or known experimental ligands. Think about enthalpic and entropic contributions to the binding. Think about possible water structure. Think of receptor flexibility, especially His, Gln, Asn, Ser, Tyr, Thr. Look for charged residues. Look at the surface colored by charge to look for greasy patches.

Look at one ligand at a time, and rate each compound (e.g. 1, 2, or 3 checkmarks). People speak out, making favorable and unfavorable comments about each ligand. Broken molecules are rapidly ignored.

Watch out for very high or very low calculated LogP. Watch for weird conformations and weird protonation states and weird tautomeric forms. ZINC is imperfect, use your judgement. Watch out for super floppy molecules. Watch out for buried polarity or missed opportunities. hydroxide is the worst. If hydroxide ain't happy ain't nobody happy.

= After the party =
The investigator purchases compounds as discussed and has them tested experimentally. Ideally, a report of the fate of each compound can be shared with other members of the group within a few weeks. We try to test at as high concentration as the assay will allow. We like to order 10mg so that we can test for purity, identity, and also repeat the experiment.

= Epilogue =
Sometimes we go through several iterations. If no hits arise, we go back to the drawing board. We try to incorporate all available information, both in the modeling, and in the judgement of the ligands. Watch out for aggregators, and be sure to control for these during the experiments. [http://advisor.docking.org/]

[[Category:Tutorials]]
[[Category:Docking]]
[[Category:Narratives]]

Building The 3D Pipeline ZINC22

2024-06-04T22:46:59Z

Sevigneron: /* Script Arguments */

= Introduction =

The 3D pipeline is a collection of scripts and software packages that enable the massively parallel creation of dockable 3D molecules.

= EZ Setup =

== BKS Cluster ==

<nowiki>
source /nfs/soft/dock/versions/dock38/pipeline_3D_ligands/env.(sh|csh)</nowiki>

This environment will set up most of the required variables for you, as well as adds the submission scripts to your PATH, which means submission can be as simple as:

bash
<nowiki>
export INPUT_FILE=$HOME/myligands.smi
export OUTPUT_DEST=$HOME/myoutput
submit-all-jobs-slurm.bash</nowiki>

csh
<nowiki>
setenv INPUT_FILE $HOME/myligands.smi
setenv OUTPUT_DEST $HOME/myoutput
submit-all-jobs-slurm.bash</nowiki>

== Wynton Cluster ==

<nowiki>
source /wynton/group/bks/soft/pipeline_3D_ligands/env.(sh|csh)</nowiki>

Similar to the BKS example, this environment will set most of the required variables for you

bash
<nowiki>
export INPUT_FILE=$HOME/myligands.smi
export OUTPUT_DEST=$HOME/myoutput
submit-all-jobs-sge.bash</nowiki>

csh
<nowiki>
setenv INPUT_FILE $HOME/myligands.smi
setenv OUTPUT_DEST $HOME/myoutput
submit-all-jobs-sge.bash</nowiki>

= Repackaging Output For Docking =

The output of the 3D pipeline scripts will be a number of tar.gz files with roughly LINES_PER_JOB molecules contained per package.

It is standard practice to repackage these smaller packages into larger packages for docking, as 50 molecules do not take long to process with DOCK.

See this wiki page for how to do this: [[Repackaging_DB2_DOCK38]]

= Script Arguments =

Main submission scripts are named submit-all-jobs-slurm.bash and submit-all-jobs-sge.bash. These scripts use environment variables as arguments instead of usual command line ones.

E.g, on bash you would pass one of these arguments like so:

<nowiki>export INPUT_FILE=$PWD/example.smi</nowiki>

or on csh:

<nowiki>setenv INPUT_FILE $PWD/example.smi</nowiki>

Prior to running the script.

== Required Arguments ==

==== INPUT_FILE ====

The input .smi file to be built. This file should contain only two columns of data: (SMILES, NAME) with no header.

==== OUTPUT_DEST ====

The base directory for output to be stored. The script will create a sub-directory here named $INPUT_FILE.batch-3d.d

Within this output directory there are 3 sub-directories:
<ol>
<li> in </li>
<li> log </li>
<li> out </li>
</ol>

In contains the input file split into fragments and sub-fragments. By default the script first splits the input file into batches of 50000, then splits those batches into sub-batches of 50. Each individual job works on one of these sub-batches. Each array batch job works on one of the batches of 50000. All of the other directories alongside 'in' share the same directory structure.

Log contains log messages from the jobs. If you are re-submitting a file, be aware that log messages from previous runs on this file will be overwritten.

Out contains tar.gz output from each job. The tarballs should contain a number of 3d molecule formats for each molecule in the batch, including 1 or more db2.gz files.

==== SOFT_HOME ====

Where software tarballs for the pipeline are stored. Symbolic links should be maintained in this directory according to the rules described in the "software arguments" section of this page. If you're sourcing a premade environment, don't worry about setting this value.

==== LICENSE_HOME ====

Where software licenses are stored. Currently our licensed software includes jchem and openeye, licenses must be named .jchem-license.cxl and .oe-license.txt respectively. if you're sourcing a premade environment, don't worry about setting this value.

== Script Arguments ==

==== SHRTCACHE ====

The base working directory for the script. By default it is /scratch

==== LONGCACHE ====

The base directory for persistent files that are shared between jobs to go (i.e where software is installed). By default it is /scratch.

==== CORINA_MAX_CONFS ====

How many nitrogen flapping configurations of each protomer corina should generate. By default only one is generated.

==== pH_LEVEL ====

Sets what pH to generate the compound(s) at. Default is a pH of 7.4

Note: For addition of new variables, that variable needs also needs to be added to the 'optional_vars' line of submit-all.bash

== Omega Arguments ==

These parameters correspond to torsion driving parameters described in the omega manual: https://docs.eyesopen.com/applications/omega/omega/omega_opt_params.html#torsion-driving-parameters

If you'd like to know more about how these parameters function, cross reference with the manual page.

==== OMEGA_MAX_CONFS ====

Maximum configurations OMEGA will generate, default 600.

==== OMEGA_ENERGY_WINDOW ====

Torsion energy window, if set to zero OMEGA will use an alternative rotatable bond dependent window method instead. Default is 12

==== OMEGA_TORLIB ====

Torsion library- can choose between GubaV21 or Original, default is Original.

==== OMEGA_FF ====

https://docs.eyesopen.com/toolkits/cpp/oefftk/OEFFConstants/OEMMFFSheffieldFFType.html#OEFF::OEMMFFSheffieldFFType::MMFF94Smod

Default is MMFF94Smod.

==== OMEGA_RMSD ====

Sets rmsd for clustering and filtering conformations. If zero, omega will use an alternative rotatable-bond dependent method instead. Default is 0.5

== Job Submission Arguments ==

==== SUBMIT_MODE ====

Choose the job submission method, choose between SGE, SLURM, or TEST_LOCAL. This will be automatically set if you use the job controller's corresponding superscript, e.g submit-all-jobs-slurm.bash. TEST_LOCAL will bypass the job controller and run the first input chunk in your shell.

==== LINES_PER_BATCH ====

How many lines of the source .smi file should be processed per array batch job, default is 50000.

==== LINES_PER_JOB ====

How many lines of the batch .smi file should be processed per array task, default is 50.

==== MAX_BATCHES ====

Each batch job will contain LINES_PER_BATCH/LINES_PER_JOB jobs, and there will be a maximum of MAX_BATCHES batches submitted at any given time. By default this value is 25, which corresponds to 25,000 queued jobs at any given time if there are 1000 jobs per batch.

The submit-all script will block until less than MAX_BATCHES job arrays are in the queue. TODO: block until less than MAX_BATCHES total jobs are running or in the queue.

==== SBATCH_ARGS ====

Additional arguments for the sbatch command. It is recommended to set a --time limit, as build jobs will save progress & terminate if they are still running two minutes before the --time limit.

==== QSUB_ARGS ====

Additional arguments for the qsub command. Similar to slurm, it is recommended to set a time limit, but you will need to manually specify both s_rt & h_rt. In the example, we set s_rt to be a minute and thirty seconds before h_rt. s_rt is the point where jobs will save progress and terminate, h_rt is when they will be forcibly terminated, even if they've not finished saving.

== Software Options ==

All software variables will be set automatically if there exists a symbolic link in $SOFT_HOME matching the software variable's name, for example:
<nowiki>
dock-latest -> DOCK.3.8.4.3d.tar.gz
jchem-latest -> jchem-19.15_r1.tar.gz
pyenv-latest -> lig_build_py3-3.7.1.tar.gz</nowiki>

They may also bet set manually- value is expected to be a path to a tar.gz file.

We use the following software:

* DOCK_VERSION

* JCHEM_VERSION

* PYENV_VERSION

* CORINA_VERSION

* OPENBABEL_VERSION

* EXTRALIBS_VERSION
Note on EXTRALIBS- Run the pipeline with an empty EXTRALIBS package (but all other software accounted for) and see which shared libraries come up as missing in the error log. Locate all missing libraries and toss them in EXTRALIBS, they will be added to LD_LIBRARY_PATH

* JAVA_VERSION

= Examples =

Minimal Example
<nowiki>
export INPUT_FILE=$PWD/example.smi
export OUTPUT_DEST=$PWD
bash submit-all-jobs-slurm.bash</nowiki>

BKS Example - limit time to 2 hours, change batch size variables. Slurm tasks should automatically save progress when reaching their time limit.

<nowiki>
export INPUT_FILE=$PWD/example.smi
export OUTPUT_DEST=$PWD/ligand_building
export SBATCH_ARGS="--time=02:00:00"
export LINES_PER_BATCH=20000
export LINES_PER_JOB=25
export MAX_BATCHES=15
bash submit-all-jobs-slurm.bash</nowiki>

Wynton Example - limit time to 30 minutes, but set a soft limit 1:30 prior to the hard limit - the interrupt generated by the soft limit will signal the job to save progress for any resubmissions and exit.

<nowiki>
export INPUT_FILE=$PWD/example.smi
export OUTPUT_DEST=$PWD/ligand_building
export QSUB_ARGS="-l s_rt=00:28:30 -l h_rt=00:30:00 -r y"
bash submit-all-jobs-sge.bash</nowiki>

= Resubmission =

If your jobs for building have finished (or timed out), and you want to continue process whatever has not been processed yet, just run submit-all-jobs-slurm/sge again (with same env arguments). The submit-all script will detect which entries haven't finished and resubmit them.

== Repatriation ==

At BKS, we currently store the tarred output of the pipeline @ /nfs/exb/zinc22/tarballs. Currently, we use the following command to repatriate output from other clusters to our cluster:

<nowiki>
### migrate_output.bash

for output in $OUTPUT_DEST/*.batch-3d.d; do
echo "starting rsync on $output to $MIGRATE_USER@files2.docking.org"
sshpass -f $PW_FILE rsync -arv $output/out $MIGRATE_USER@files2.docking.org:/nfs/exb/zinc22/tarballs/$(basename $output).out
done</nowiki>

sshpass is optional here but preferable for convenience's sake. Since files2.docking.org is only visible within the UCSF network, any clusters outside will need to maintain a network tunnel when rsyncing.

= Errors =

Sometimes an output tarball will have few or no entries within. Certain molecule types will fail to be built, and often these molecules get bunched together (i.e if the input file is sorted by SMILES). Additionally, a small percentage of all molecules may fail to be processed by corina or amsol. If neither of these explain what is causing your missing entries, check that tarball's corresponding log entry for more info.

= Additional Notes =

It is safe to re-run the same file multiple times- the script takes care of making sure not to re-run any jobs that have already completed successfully prior. This is only the case if that file's corresponding batch-3d.d output directory has not been moved or deleted.

For example, if one of your nodes went down and caused a bunch of jobs to fail, it would be safe to re-run ./submit-all-jobs.bash to re-submit those jobs. (assuming there are no jobs for that file currently queued/running)

back to [[ZINC22:Building_3D]]

[[Category:ZINC22]]
[[Category:DOCK_3.8]]

Building The 3D Pipeline ZINC22

2024-06-04T22:36:50Z

Sevigneron: /* Script Arguments */

= Introduction =

The 3D pipeline is a collection of scripts and software packages that enable the massively parallel creation of dockable 3D molecules.

= EZ Setup =

== BKS Cluster ==

<nowiki>
source /nfs/soft/dock/versions/dock38/pipeline_3D_ligands/env.(sh|csh)</nowiki>

This environment will set up most of the required variables for you, as well as adds the submission scripts to your PATH, which means submission can be as simple as:

bash
<nowiki>
export INPUT_FILE=$HOME/myligands.smi
export OUTPUT_DEST=$HOME/myoutput
submit-all-jobs-slurm.bash</nowiki>

csh
<nowiki>
setenv INPUT_FILE $HOME/myligands.smi
setenv OUTPUT_DEST $HOME/myoutput
submit-all-jobs-slurm.bash</nowiki>

== Wynton Cluster ==

<nowiki>
source /wynton/group/bks/soft/pipeline_3D_ligands/env.(sh|csh)</nowiki>

Similar to the BKS example, this environment will set most of the required variables for you

bash
<nowiki>
export INPUT_FILE=$HOME/myligands.smi
export OUTPUT_DEST=$HOME/myoutput
submit-all-jobs-sge.bash</nowiki>

csh
<nowiki>
setenv INPUT_FILE $HOME/myligands.smi
setenv OUTPUT_DEST $HOME/myoutput
submit-all-jobs-sge.bash</nowiki>

= Repackaging Output For Docking =

The output of the 3D pipeline scripts will be a number of tar.gz files with roughly LINES_PER_JOB molecules contained per package.

It is standard practice to repackage these smaller packages into larger packages for docking, as 50 molecules do not take long to process with DOCK.

See this wiki page for how to do this: [[Repackaging_DB2_DOCK38]]

= Script Arguments =

Main submission scripts are named submit-all-jobs-slurm.bash and submit-all-jobs-sge.bash. These scripts use environment variables as arguments instead of usual command line ones.

E.g, on bash you would pass one of these arguments like so:

<nowiki>export INPUT_FILE=$PWD/example.smi</nowiki>

or on csh:

<nowiki>setenv INPUT_FILE $PWD/example.smi</nowiki>

Prior to running the script.

== Required Arguments ==

==== INPUT_FILE ====

The input .smi file to be built. This file should contain only two columns of data: (SMILES, NAME) with no header.

==== OUTPUT_DEST ====

The base directory for output to be stored. The script will create a sub-directory here named $INPUT_FILE.batch-3d.d

Within this output directory there are 3 sub-directories:
<ol>
<li> in </li>
<li> log </li>
<li> out </li>
</ol>

In contains the input file split into fragments and sub-fragments. By default the script first splits the input file into batches of 50000, then splits those batches into sub-batches of 50. Each individual job works on one of these sub-batches. Each array batch job works on one of the batches of 50000. All of the other directories alongside 'in' share the same directory structure.

Log contains log messages from the jobs. If you are re-submitting a file, be aware that log messages from previous runs on this file will be overwritten.

Out contains tar.gz output from each job. The tarballs should contain a number of 3d molecule formats for each molecule in the batch, including 1 or more db2.gz files.

==== SOFT_HOME ====

Where software tarballs for the pipeline are stored. Symbolic links should be maintained in this directory according to the rules described in the "software arguments" section of this page. If you're sourcing a premade environment, don't worry about setting this value.

==== LICENSE_HOME ====

Where software licenses are stored. Currently our licensed software includes jchem and openeye, licenses must be named .jchem-license.cxl and .oe-license.txt respectively. if you're sourcing a premade environment, don't worry about setting this value.

== Script Arguments ==

==== SHRTCACHE ====

The base working directory for the script. By default it is /scratch

==== LONGCACHE ====

The base directory for persistent files that are shared between jobs to go (i.e where software is installed). By default it is /scratch.

==== CORINA_MAX_CONFS ====

How many nitrogen flapping configurations of each protomer corina should generate. By default only one is generated.

==== pH_LEVEL ====

Sets what pH to generate the compound(s) at. Default is a pH of 7.4

== Omega Arguments ==

These parameters correspond to torsion driving parameters described in the omega manual: https://docs.eyesopen.com/applications/omega/omega/omega_opt_params.html#torsion-driving-parameters

If you'd like to know more about how these parameters function, cross reference with the manual page.

==== OMEGA_MAX_CONFS ====

Maximum configurations OMEGA will generate, default 600.

==== OMEGA_ENERGY_WINDOW ====

Torsion energy window, if set to zero OMEGA will use an alternative rotatable bond dependent window method instead. Default is 12

==== OMEGA_TORLIB ====

Torsion library- can choose between GubaV21 or Original, default is Original.

==== OMEGA_FF ====

https://docs.eyesopen.com/toolkits/cpp/oefftk/OEFFConstants/OEMMFFSheffieldFFType.html#OEFF::OEMMFFSheffieldFFType::MMFF94Smod

Default is MMFF94Smod.

==== OMEGA_RMSD ====

Sets rmsd for clustering and filtering conformations. If zero, omega will use an alternative rotatable-bond dependent method instead. Default is 0.5

== Job Submission Arguments ==

==== SUBMIT_MODE ====

Choose the job submission method, choose between SGE, SLURM, or TEST_LOCAL. This will be automatically set if you use the job controller's corresponding superscript, e.g submit-all-jobs-slurm.bash. TEST_LOCAL will bypass the job controller and run the first input chunk in your shell.

==== LINES_PER_BATCH ====

How many lines of the source .smi file should be processed per array batch job, default is 50000.

==== LINES_PER_JOB ====

How many lines of the batch .smi file should be processed per array task, default is 50.

==== MAX_BATCHES ====

Each batch job will contain LINES_PER_BATCH/LINES_PER_JOB jobs, and there will be a maximum of MAX_BATCHES batches submitted at any given time. By default this value is 25, which corresponds to 25,000 queued jobs at any given time if there are 1000 jobs per batch.

The submit-all script will block until less than MAX_BATCHES job arrays are in the queue. TODO: block until less than MAX_BATCHES total jobs are running or in the queue.

==== SBATCH_ARGS ====

Additional arguments for the sbatch command. It is recommended to set a --time limit, as build jobs will save progress & terminate if they are still running two minutes before the --time limit.

==== QSUB_ARGS ====

Additional arguments for the qsub command. Similar to slurm, it is recommended to set a time limit, but you will need to manually specify both s_rt & h_rt. In the example, we set s_rt to be a minute and thirty seconds before h_rt. s_rt is the point where jobs will save progress and terminate, h_rt is when they will be forcibly terminated, even if they've not finished saving.

== Software Options ==

All software variables will be set automatically if there exists a symbolic link in $SOFT_HOME matching the software variable's name, for example:
<nowiki>
dock-latest -> DOCK.3.8.4.3d.tar.gz
jchem-latest -> jchem-19.15_r1.tar.gz
pyenv-latest -> lig_build_py3-3.7.1.tar.gz</nowiki>

They may also bet set manually- value is expected to be a path to a tar.gz file.

We use the following software:

* DOCK_VERSION

* JCHEM_VERSION

* PYENV_VERSION

* CORINA_VERSION

* OPENBABEL_VERSION

* EXTRALIBS_VERSION
Note on EXTRALIBS- Run the pipeline with an empty EXTRALIBS package (but all other software accounted for) and see which shared libraries come up as missing in the error log. Locate all missing libraries and toss them in EXTRALIBS, they will be added to LD_LIBRARY_PATH

* JAVA_VERSION

= Examples =

Minimal Example
<nowiki>
export INPUT_FILE=$PWD/example.smi
export OUTPUT_DEST=$PWD
bash submit-all-jobs-slurm.bash</nowiki>

BKS Example - limit time to 2 hours, change batch size variables. Slurm tasks should automatically save progress when reaching their time limit.

<nowiki>
export INPUT_FILE=$PWD/example.smi
export OUTPUT_DEST=$PWD/ligand_building
export SBATCH_ARGS="--time=02:00:00"
export LINES_PER_BATCH=20000
export LINES_PER_JOB=25
export MAX_BATCHES=15
bash submit-all-jobs-slurm.bash</nowiki>

Wynton Example - limit time to 30 minutes, but set a soft limit 1:30 prior to the hard limit - the interrupt generated by the soft limit will signal the job to save progress for any resubmissions and exit.

<nowiki>
export INPUT_FILE=$PWD/example.smi
export OUTPUT_DEST=$PWD/ligand_building
export QSUB_ARGS="-l s_rt=00:28:30 -l h_rt=00:30:00 -r y"
bash submit-all-jobs-sge.bash</nowiki>

= Resubmission =

If your jobs for building have finished (or timed out), and you want to continue process whatever has not been processed yet, just run submit-all-jobs-slurm/sge again (with same env arguments). The submit-all script will detect which entries haven't finished and resubmit them.

== Repatriation ==

At BKS, we currently store the tarred output of the pipeline @ /nfs/exb/zinc22/tarballs. Currently, we use the following command to repatriate output from other clusters to our cluster:

<nowiki>
### migrate_output.bash

for output in $OUTPUT_DEST/*.batch-3d.d; do
echo "starting rsync on $output to $MIGRATE_USER@files2.docking.org"
sshpass -f $PW_FILE rsync -arv $output/out $MIGRATE_USER@files2.docking.org:/nfs/exb/zinc22/tarballs/$(basename $output).out
done</nowiki>

sshpass is optional here but preferable for convenience's sake. Since files2.docking.org is only visible within the UCSF network, any clusters outside will need to maintain a network tunnel when rsyncing.

= Errors =

Sometimes an output tarball will have few or no entries within. Certain molecule types will fail to be built, and often these molecules get bunched together (i.e if the input file is sorted by SMILES). Additionally, a small percentage of all molecules may fail to be processed by corina or amsol. If neither of these explain what is causing your missing entries, check that tarball's corresponding log entry for more info.

= Additional Notes =

It is safe to re-run the same file multiple times- the script takes care of making sure not to re-run any jobs that have already completed successfully prior. This is only the case if that file's corresponding batch-3d.d output directory has not been moved or deleted.

For example, if one of your nodes went down and caused a bunch of jobs to fail, it would be safe to re-run ./submit-all-jobs.bash to re-submit those jobs. (assuming there are no jobs for that file currently queued/running)

back to [[ZINC22:Building_3D]]

[[Category:ZINC22]]
[[Category:DOCK_3.8]]

Building The 3D Pipeline ZINC22

2024-06-04T22:36:10Z

Sevigneron: /* Script Arguments */

= Introduction =

The 3D pipeline is a collection of scripts and software packages that enable the massively parallel creation of dockable 3D molecules.

= EZ Setup =

== BKS Cluster ==

<nowiki>
source /nfs/soft/dock/versions/dock38/pipeline_3D_ligands/env.(sh|csh)</nowiki>

This environment will set up most of the required variables for you, as well as adds the submission scripts to your PATH, which means submission can be as simple as:

bash
<nowiki>
export INPUT_FILE=$HOME/myligands.smi
export OUTPUT_DEST=$HOME/myoutput
submit-all-jobs-slurm.bash</nowiki>

csh
<nowiki>
setenv INPUT_FILE $HOME/myligands.smi
setenv OUTPUT_DEST $HOME/myoutput
submit-all-jobs-slurm.bash</nowiki>

== Wynton Cluster ==

<nowiki>
source /wynton/group/bks/soft/pipeline_3D_ligands/env.(sh|csh)</nowiki>

Similar to the BKS example, this environment will set most of the required variables for you

bash
<nowiki>
export INPUT_FILE=$HOME/myligands.smi
export OUTPUT_DEST=$HOME/myoutput
submit-all-jobs-sge.bash</nowiki>

csh
<nowiki>
setenv INPUT_FILE $HOME/myligands.smi
setenv OUTPUT_DEST $HOME/myoutput
submit-all-jobs-sge.bash</nowiki>

= Repackaging Output For Docking =

The output of the 3D pipeline scripts will be a number of tar.gz files with roughly LINES_PER_JOB molecules contained per package.

It is standard practice to repackage these smaller packages into larger packages for docking, as 50 molecules do not take long to process with DOCK.

See this wiki page for how to do this: [[Repackaging_DB2_DOCK38]]

= Script Arguments =

Main submission scripts are named submit-all-jobs-slurm.bash and submit-all-jobs-sge.bash. These scripts use environment variables as arguments instead of usual command line ones.

E.g, on bash you would pass one of these arguments like so:

<nowiki>export INPUT_FILE=$PWD/example.smi</nowiki>

or on csh:

<nowiki>setenv INPUT_FILE $PWD/example.smi</nowiki>

Prior to running the script.

== Required Arguments ==

==== INPUT_FILE ====

The input .smi file to be built. This file should contain only two columns of data: (SMILES, NAME) with no header.

==== OUTPUT_DEST ====

The base directory for output to be stored. The script will create a sub-directory here named $INPUT_FILE.batch-3d.d

Within this output directory there are 3 sub-directories:
<ol>
<li> in </li>
<li> log </li>
<li> out </li>
</ol>

In contains the input file split into fragments and sub-fragments. By default the script first splits the input file into batches of 50000, then splits those batches into sub-batches of 50. Each individual job works on one of these sub-batches. Each array batch job works on one of the batches of 50000. All of the other directories alongside 'in' share the same directory structure.

Log contains log messages from the jobs. If you are re-submitting a file, be aware that log messages from previous runs on this file will be overwritten.

Out contains tar.gz output from each job. The tarballs should contain a number of 3d molecule formats for each molecule in the batch, including 1 or more db2.gz files.

==== SOFT_HOME ====

Where software tarballs for the pipeline are stored. Symbolic links should be maintained in this directory according to the rules described in the "software arguments" section of this page. If you're sourcing a premade environment, don't worry about setting this value.

==== LICENSE_HOME ====

Where software licenses are stored. Currently our licensed software includes jchem and openeye, licenses must be named .jchem-license.cxl and .oe-license.txt respectively. if you're sourcing a premade environment, don't worry about setting this value.

== Script Arguments ==

==== SHRTCACHE ====

The base working directory for the script. By default it is /scratch

==== LONGCACHE ====

The base directory for persistent files that are shared between jobs to go (i.e where software is installed). By default it is /scratch.

==== CORINA_MAX_CONFS ====

How many nitrogen flapping configurations of each protomer corina should generate. By default only one is generated.

==== pH_LEVEL ====

Sets what pH to generate the compound at. Default is 7.4

== Omega Arguments ==

These parameters correspond to torsion driving parameters described in the omega manual: https://docs.eyesopen.com/applications/omega/omega/omega_opt_params.html#torsion-driving-parameters

If you'd like to know more about how these parameters function, cross reference with the manual page.

==== OMEGA_MAX_CONFS ====

Maximum configurations OMEGA will generate, default 600.

==== OMEGA_ENERGY_WINDOW ====

Torsion energy window, if set to zero OMEGA will use an alternative rotatable bond dependent window method instead. Default is 12

==== OMEGA_TORLIB ====

Torsion library- can choose between GubaV21 or Original, default is Original.

==== OMEGA_FF ====

https://docs.eyesopen.com/toolkits/cpp/oefftk/OEFFConstants/OEMMFFSheffieldFFType.html#OEFF::OEMMFFSheffieldFFType::MMFF94Smod

Default is MMFF94Smod.

==== OMEGA_RMSD ====

Sets rmsd for clustering and filtering conformations. If zero, omega will use an alternative rotatable-bond dependent method instead. Default is 0.5

== Job Submission Arguments ==

==== SUBMIT_MODE ====

Choose the job submission method, choose between SGE, SLURM, or TEST_LOCAL. This will be automatically set if you use the job controller's corresponding superscript, e.g submit-all-jobs-slurm.bash. TEST_LOCAL will bypass the job controller and run the first input chunk in your shell.

==== LINES_PER_BATCH ====

How many lines of the source .smi file should be processed per array batch job, default is 50000.

==== LINES_PER_JOB ====

How many lines of the batch .smi file should be processed per array task, default is 50.

==== MAX_BATCHES ====

Each batch job will contain LINES_PER_BATCH/LINES_PER_JOB jobs, and there will be a maximum of MAX_BATCHES batches submitted at any given time. By default this value is 25, which corresponds to 25,000 queued jobs at any given time if there are 1000 jobs per batch.

The submit-all script will block until less than MAX_BATCHES job arrays are in the queue. TODO: block until less than MAX_BATCHES total jobs are running or in the queue.

==== SBATCH_ARGS ====

Additional arguments for the sbatch command. It is recommended to set a --time limit, as build jobs will save progress & terminate if they are still running two minutes before the --time limit.

==== QSUB_ARGS ====

Additional arguments for the qsub command. Similar to slurm, it is recommended to set a time limit, but you will need to manually specify both s_rt & h_rt. In the example, we set s_rt to be a minute and thirty seconds before h_rt. s_rt is the point where jobs will save progress and terminate, h_rt is when they will be forcibly terminated, even if they've not finished saving.

== Software Options ==

All software variables will be set automatically if there exists a symbolic link in $SOFT_HOME matching the software variable's name, for example:
<nowiki>
dock-latest -> DOCK.3.8.4.3d.tar.gz
jchem-latest -> jchem-19.15_r1.tar.gz
pyenv-latest -> lig_build_py3-3.7.1.tar.gz</nowiki>

They may also bet set manually- value is expected to be a path to a tar.gz file.

We use the following software:

* DOCK_VERSION

* JCHEM_VERSION

* PYENV_VERSION

* CORINA_VERSION

* OPENBABEL_VERSION

* EXTRALIBS_VERSION
Note on EXTRALIBS- Run the pipeline with an empty EXTRALIBS package (but all other software accounted for) and see which shared libraries come up as missing in the error log. Locate all missing libraries and toss them in EXTRALIBS, they will be added to LD_LIBRARY_PATH

* JAVA_VERSION

= Examples =

Minimal Example
<nowiki>
export INPUT_FILE=$PWD/example.smi
export OUTPUT_DEST=$PWD
bash submit-all-jobs-slurm.bash</nowiki>

BKS Example - limit time to 2 hours, change batch size variables. Slurm tasks should automatically save progress when reaching their time limit.

<nowiki>
export INPUT_FILE=$PWD/example.smi
export OUTPUT_DEST=$PWD/ligand_building
export SBATCH_ARGS="--time=02:00:00"
export LINES_PER_BATCH=20000
export LINES_PER_JOB=25
export MAX_BATCHES=15
bash submit-all-jobs-slurm.bash</nowiki>

Wynton Example - limit time to 30 minutes, but set a soft limit 1:30 prior to the hard limit - the interrupt generated by the soft limit will signal the job to save progress for any resubmissions and exit.

<nowiki>
export INPUT_FILE=$PWD/example.smi
export OUTPUT_DEST=$PWD/ligand_building
export QSUB_ARGS="-l s_rt=00:28:30 -l h_rt=00:30:00 -r y"
bash submit-all-jobs-sge.bash</nowiki>

= Resubmission =

If your jobs for building have finished (or timed out), and you want to continue process whatever has not been processed yet, just run submit-all-jobs-slurm/sge again (with same env arguments). The submit-all script will detect which entries haven't finished and resubmit them.

== Repatriation ==

At BKS, we currently store the tarred output of the pipeline @ /nfs/exb/zinc22/tarballs. Currently, we use the following command to repatriate output from other clusters to our cluster:

<nowiki>
### migrate_output.bash

for output in $OUTPUT_DEST/*.batch-3d.d; do
echo "starting rsync on $output to $MIGRATE_USER@files2.docking.org"
sshpass -f $PW_FILE rsync -arv $output/out $MIGRATE_USER@files2.docking.org:/nfs/exb/zinc22/tarballs/$(basename $output).out
done</nowiki>

sshpass is optional here but preferable for convenience's sake. Since files2.docking.org is only visible within the UCSF network, any clusters outside will need to maintain a network tunnel when rsyncing.

= Errors =

Sometimes an output tarball will have few or no entries within. Certain molecule types will fail to be built, and often these molecules get bunched together (i.e if the input file is sorted by SMILES). Additionally, a small percentage of all molecules may fail to be processed by corina or amsol. If neither of these explain what is causing your missing entries, check that tarball's corresponding log entry for more info.

= Additional Notes =

It is safe to re-run the same file multiple times- the script takes care of making sure not to re-run any jobs that have already completed successfully prior. This is only the case if that file's corresponding batch-3d.d output directory has not been moved or deleted.

For example, if one of your nodes went down and caused a bunch of jobs to fail, it would be safe to re-run ./submit-all-jobs.bash to re-submit those jobs. (assuming there are no jobs for that file currently queued/running)

back to [[ZINC22:Building_3D]]

[[Category:ZINC22]]
[[Category:DOCK_3.8]]

Building The 3D Pipeline ZINC22

2024-06-04T22:35:56Z

Sevigneron: /* Script Arguments */

= Introduction =

The 3D pipeline is a collection of scripts and software packages that enable the massively parallel creation of dockable 3D molecules.

= EZ Setup =

== BKS Cluster ==

<nowiki>
source /nfs/soft/dock/versions/dock38/pipeline_3D_ligands/env.(sh|csh)</nowiki>

This environment will set up most of the required variables for you, as well as adds the submission scripts to your PATH, which means submission can be as simple as:

bash
<nowiki>
export INPUT_FILE=$HOME/myligands.smi
export OUTPUT_DEST=$HOME/myoutput
submit-all-jobs-slurm.bash</nowiki>

csh
<nowiki>
setenv INPUT_FILE $HOME/myligands.smi
setenv OUTPUT_DEST $HOME/myoutput
submit-all-jobs-slurm.bash</nowiki>

== Wynton Cluster ==

<nowiki>
source /wynton/group/bks/soft/pipeline_3D_ligands/env.(sh|csh)</nowiki>

Similar to the BKS example, this environment will set most of the required variables for you

bash
<nowiki>
export INPUT_FILE=$HOME/myligands.smi
export OUTPUT_DEST=$HOME/myoutput
submit-all-jobs-sge.bash</nowiki>

csh
<nowiki>
setenv INPUT_FILE $HOME/myligands.smi
setenv OUTPUT_DEST $HOME/myoutput
submit-all-jobs-sge.bash</nowiki>

= Repackaging Output For Docking =

The output of the 3D pipeline scripts will be a number of tar.gz files with roughly LINES_PER_JOB molecules contained per package.

It is standard practice to repackage these smaller packages into larger packages for docking, as 50 molecules do not take long to process with DOCK.

See this wiki page for how to do this: [[Repackaging_DB2_DOCK38]]

= Script Arguments =

Main submission scripts are named submit-all-jobs-slurm.bash and submit-all-jobs-sge.bash. These scripts use environment variables as arguments instead of usual command line ones.

E.g, on bash you would pass one of these arguments like so:

<nowiki>export INPUT_FILE=$PWD/example.smi</nowiki>

or on csh:

<nowiki>setenv INPUT_FILE $PWD/example.smi</nowiki>

Prior to running the script.

== Required Arguments ==

==== INPUT_FILE ====

The input .smi file to be built. This file should contain only two columns of data: (SMILES, NAME) with no header.

==== OUTPUT_DEST ====

The base directory for output to be stored. The script will create a sub-directory here named $INPUT_FILE.batch-3d.d

Within this output directory there are 3 sub-directories:
<ol>
<li> in </li>
<li> log </li>
<li> out </li>
</ol>

In contains the input file split into fragments and sub-fragments. By default the script first splits the input file into batches of 50000, then splits those batches into sub-batches of 50. Each individual job works on one of these sub-batches. Each array batch job works on one of the batches of 50000. All of the other directories alongside 'in' share the same directory structure.

Log contains log messages from the jobs. If you are re-submitting a file, be aware that log messages from previous runs on this file will be overwritten.

Out contains tar.gz output from each job. The tarballs should contain a number of 3d molecule formats for each molecule in the batch, including 1 or more db2.gz files.

==== SOFT_HOME ====

Where software tarballs for the pipeline are stored. Symbolic links should be maintained in this directory according to the rules described in the "software arguments" section of this page. If you're sourcing a premade environment, don't worry about setting this value.

==== LICENSE_HOME ====

Where software licenses are stored. Currently our licensed software includes jchem and openeye, licenses must be named .jchem-license.cxl and .oe-license.txt respectively. if you're sourcing a premade environment, don't worry about setting this value.

== Script Arguments ==

==== SHRTCACHE ====

The base working directory for the script. By default it is /scratch

==== LONGCACHE ====

The base directory for persistent files that are shared between jobs to go (i.e where software is installed). By default it is /scratch.

==== CORINA_MAX_CONFS ====

How many nitrogen flapping configurations of each protomer corina should generate. By default only one is generated.

==== pH_LEVEL ====

Sets what pH to generate the compound at. Default is 7.4

== Omega Arguments ==

These parameters correspond to torsion driving parameters described in the omega manual: https://docs.eyesopen.com/applications/omega/omega/omega_opt_params.html#torsion-driving-parameters

If you'd like to know more about how these parameters function, cross reference with the manual page.

==== OMEGA_MAX_CONFS ====

Maximum configurations OMEGA will generate, default 600.

==== OMEGA_ENERGY_WINDOW ====

Torsion energy window, if set to zero OMEGA will use an alternative rotatable bond dependent window method instead. Default is 12

==== OMEGA_TORLIB ====

Torsion library- can choose between GubaV21 or Original, default is Original.

==== OMEGA_FF ====

https://docs.eyesopen.com/toolkits/cpp/oefftk/OEFFConstants/OEMMFFSheffieldFFType.html#OEFF::OEMMFFSheffieldFFType::MMFF94Smod

Default is MMFF94Smod.

==== OMEGA_RMSD ====

Sets rmsd for clustering and filtering conformations. If zero, omega will use an alternative rotatable-bond dependent method instead. Default is 0.5

== Job Submission Arguments ==

==== SUBMIT_MODE ====

Choose the job submission method, choose between SGE, SLURM, or TEST_LOCAL. This will be automatically set if you use the job controller's corresponding superscript, e.g submit-all-jobs-slurm.bash. TEST_LOCAL will bypass the job controller and run the first input chunk in your shell.

==== LINES_PER_BATCH ====

How many lines of the source .smi file should be processed per array batch job, default is 50000.

==== LINES_PER_JOB ====

How many lines of the batch .smi file should be processed per array task, default is 50.

==== MAX_BATCHES ====

Each batch job will contain LINES_PER_BATCH/LINES_PER_JOB jobs, and there will be a maximum of MAX_BATCHES batches submitted at any given time. By default this value is 25, which corresponds to 25,000 queued jobs at any given time if there are 1000 jobs per batch.

The submit-all script will block until less than MAX_BATCHES job arrays are in the queue. TODO: block until less than MAX_BATCHES total jobs are running or in the queue.

==== SBATCH_ARGS ====

Additional arguments for the sbatch command. It is recommended to set a --time limit, as build jobs will save progress & terminate if they are still running two minutes before the --time limit.

==== QSUB_ARGS ====

Additional arguments for the qsub command. Similar to slurm, it is recommended to set a time limit, but you will need to manually specify both s_rt & h_rt. In the example, we set s_rt to be a minute and thirty seconds before h_rt. s_rt is the point where jobs will save progress and terminate, h_rt is when they will be forcibly terminated, even if they've not finished saving.

== Software Options ==

All software variables will be set automatically if there exists a symbolic link in $SOFT_HOME matching the software variable's name, for example:
<nowiki>
dock-latest -> DOCK.3.8.4.3d.tar.gz
jchem-latest -> jchem-19.15_r1.tar.gz
pyenv-latest -> lig_build_py3-3.7.1.tar.gz</nowiki>

They may also bet set manually- value is expected to be a path to a tar.gz file.

We use the following software:

* DOCK_VERSION

* JCHEM_VERSION

* PYENV_VERSION

* CORINA_VERSION

* OPENBABEL_VERSION

* EXTRALIBS_VERSION
Note on EXTRALIBS- Run the pipeline with an empty EXTRALIBS package (but all other software accounted for) and see which shared libraries come up as missing in the error log. Locate all missing libraries and toss them in EXTRALIBS, they will be added to LD_LIBRARY_PATH

* JAVA_VERSION

= Examples =

Minimal Example
<nowiki>
export INPUT_FILE=$PWD/example.smi
export OUTPUT_DEST=$PWD
bash submit-all-jobs-slurm.bash</nowiki>

BKS Example - limit time to 2 hours, change batch size variables. Slurm tasks should automatically save progress when reaching their time limit.

<nowiki>
export INPUT_FILE=$PWD/example.smi
export OUTPUT_DEST=$PWD/ligand_building
export SBATCH_ARGS="--time=02:00:00"
export LINES_PER_BATCH=20000
export LINES_PER_JOB=25
export MAX_BATCHES=15
bash submit-all-jobs-slurm.bash</nowiki>

Wynton Example - limit time to 30 minutes, but set a soft limit 1:30 prior to the hard limit - the interrupt generated by the soft limit will signal the job to save progress for any resubmissions and exit.

<nowiki>
export INPUT_FILE=$PWD/example.smi
export OUTPUT_DEST=$PWD/ligand_building
export QSUB_ARGS="-l s_rt=00:28:30 -l h_rt=00:30:00 -r y"
bash submit-all-jobs-sge.bash</nowiki>

= Resubmission =

If your jobs for building have finished (or timed out), and you want to continue process whatever has not been processed yet, just run submit-all-jobs-slurm/sge again (with same env arguments). The submit-all script will detect which entries haven't finished and resubmit them.

== Repatriation ==

At BKS, we currently store the tarred output of the pipeline @ /nfs/exb/zinc22/tarballs. Currently, we use the following command to repatriate output from other clusters to our cluster:

<nowiki>
### migrate_output.bash

for output in $OUTPUT_DEST/*.batch-3d.d; do
echo "starting rsync on $output to $MIGRATE_USER@files2.docking.org"
sshpass -f $PW_FILE rsync -arv $output/out $MIGRATE_USER@files2.docking.org:/nfs/exb/zinc22/tarballs/$(basename $output).out
done</nowiki>

sshpass is optional here but preferable for convenience's sake. Since files2.docking.org is only visible within the UCSF network, any clusters outside will need to maintain a network tunnel when rsyncing.

= Errors =

Sometimes an output tarball will have few or no entries within. Certain molecule types will fail to be built, and often these molecules get bunched together (i.e if the input file is sorted by SMILES). Additionally, a small percentage of all molecules may fail to be processed by corina or amsol. If neither of these explain what is causing your missing entries, check that tarball's corresponding log entry for more info.

= Additional Notes =

It is safe to re-run the same file multiple times- the script takes care of making sure not to re-run any jobs that have already completed successfully prior. This is only the case if that file's corresponding batch-3d.d output directory has not been moved or deleted.

For example, if one of your nodes went down and caused a bunch of jobs to fail, it would be safe to re-run ./submit-all-jobs.bash to re-submit those jobs. (assuming there are no jobs for that file currently queued/running)

back to [[ZINC22:Building_3D]]

[[Category:ZINC22]]
[[Category:DOCK_3.8]]