DISI - User contributions [en]

ADMET Builder

2026-07-24T22:20:31Z

Zdingman: Updated env paths for all users

= ADMET Builder (OpenADMET) =

== Introduction ==

As of 7/6/26, '''ADMET Builder''' is an internal Irwin Lab web tool for submitting one or more compounds in SMILES format and receiving OpenADMET prediction results.

The current backend runs on C7 using the `openadmet-models` conda environment.

ADMET Builder returns:

* Molecular formula
* SMILES string
* RDKit-generated 2D structure image
* logD
* Caco-2 A to B permeability
* Caco-2 B to A permeability
* Mouse plasma protein binding
* Human plasma protein binding
* Downloadable CSV output

[[File:ADMET builder.png|thumb]]
[[File:ADMET builder results.png|thumb]]

Created by Benjamin "Ben" Romero (Undergrad Northwestern University '29), Summer 2026.<br>
Developed in the Irwin Lab at UCSF.

== How to Login ==

Log into C7:

<pre>
ssh USER@epsilon.compbio.ucsf.edu
</pre>

If you do not have access to C7, you can [[Cluster 7|find more information here]].

== How to Open Website With SwitchyOmega ==

If using SwitchyOmega, use:

<pre>
Protocol: SOCKS5
Host: localhost
Port: 1074
</pre>

Then open:

<pre>
http://169.230.26.173:8501
</pre>

If the SOCKS proxy is not running, start it from a terminal:

<pre>
ssh -f -N -D 1074 USER@epsilon.compbio.ucsf.edu
</pre>

When finished, switch SwitchyOmega back to Direct.

== Where The Website Lives ==

Project directory on C7:

<pre>
/nfs/home/bromero/openadmet/admet-builder
</pre>

Important files:

<pre>
app.py
static/index.html
static/styles.css
static/app.js
.gitignore
</pre>

== Repository ==

GitLab repository:

<pre>
https://gitlab.docking.org/USERNAME/admet-builder
</pre>

Clone using HTTPS:

<pre>
git clone https://gitlab.docking.org/USERNAME/admet-builder.git
cd admet-builder
</pre>

Clone using SSH:

<pre>
git clone git@gitlab.docking.org:USERNAME/admet-builder.git
cd admet-builder
</pre>

== Environment ==

The backend uses the OpenADMET conda environment:

<pre>
module load anaconda3/2025.06.0
source "$(conda info --base)/etc/profile.d/conda.sh"
conda activate /nfs/home/bromero/.conda/envs/openadmet-models/
</pre>

Python dependencies include:

* Flask
* pandas
* RDKit
* OpenADMET models package

The OpenADMET model directory is expected at:

<pre>
~/openadmet/permeability-logd-ppb-chemeleon-baseline/anvil_training
</pre>

== How to Start Backend ==

On C7:

<pre>
module load anaconda3/2025.06.0
source "$(conda info --base)/etc/profile.d/conda.sh"
conda activate /nfs/home/bromero/.conda/envs/openadmet-models/
cd /nfs/home/bromero/openadmet/admet-builder
python app.py
</pre>

The backend runs on:

<pre>
http://0.0.0.0:8501
</pre>

The browser URL is:

<pre>
http://169.230.26.173:8501
</pre>

Test from C7:

<pre>
curl http://127.0.0.1:8501
</pre>

== How to Keep Backend Running ==

Use screen:

<pre>
screen -S admet-builder
module load anaconda3/2025.06.0
source "$(conda info --base)/etc/profile.d/conda.sh"
conda activate /nfs/home/bromero/.conda/envs/openadmet-models/
cd /nfs/home/bromero/openadmet/admet-builder
python app.py
</pre>

Detach:

<pre>
Ctrl-A
D
</pre>

Reconnect:

<pre>
screen -r admet-builder
</pre>

List screens:

<pre>
screen -ls
</pre>

Stop backend:

<pre>
screen -r admet-builder
Ctrl-C
exit
</pre>

== Test Input ==

Example SMILES input:

<pre>
CCO
c1ccccc1
CC(=O)Oc1ccccc1C(=O)O
</pre>

Larger test molecule:

<pre>
CC(C)Cc1ccc(cc1)[C@@H](C)C(=O)O
</pre>

== Test API With curl ==

Test the API from C7:

<pre>
curl -X POST http://127.0.0.1:8501/predict-smiles \
-H "Content-Type: application/json" \
-d '{"smiles":["CCO","c1ccccc1","CC(=O)Oc1ccccc1C(=O)O"]}'
</pre>

== Current API Endpoint ==

Main endpoint:

<pre>
POST /predict-smiles
</pre>

Example JSON request:

<pre>
{
"smiles": [
"CCO",
"c1ccccc1",
"CC(=O)Oc1ccccc1C(=O)O"
]
}
</pre>

Example response contains:

<pre>
{
"csv": "...",
"rows": [],
"columns": []
}
</pre>

The frontend uses this response to display the results table and automatically download the CSV.

== How It Works ==

ADMET Builder has three major parts:

{| class="wikitable"
! Layer
! Technology
! Purpose
|-
| Frontend
| HTML, CSS, JavaScript
| User interface, SMILES input, results table, and structure image display
|-
| Backend API
| Flask
| Receives SMILES, runs predictions, and returns results
|-
| Prediction Engine
| OpenADMET
| Generates ADMET predictions from molecular input
|}

Backend flow:

# Receives SMILES strings from the website.
# Converts the SMILES into a temporary CSV file.
# Runs the OpenADMET command-line prediction tool.
# Reads the OpenADMET output CSV.
# Uses RDKit to calculate molecular formulas.
# Uses RDKit to generate 2D molecular structure images.
# Sends results back to the web interface as JSON.
# Provides the same results as a downloadable CSV file.

== Modularity ==

ADMET Builder is modular by design. The frontend, backend, and prediction engine are separated so each part can be reused or replaced.

This means:

* The frontend can be redesigned without changing the model logic.
* The Flask backend can support new endpoints.
* Additional OpenADMET models can be added later.
* Other chemistry tools can be wrapped using the same backend pattern.
* The results table can be adapted for different prediction outputs.
* The SMILES-to-structure display can be reused in other cheminformatics apps.
* The app can serve as a template for future scientific web interfaces.

Because of this structure, ADMET Builder can be used both as a working lab tool and as a starting point for other modular research software.

== Output ==

The output table includes:

{| class="wikitable"
! Column
! Meaning
|-
| Molecule
| Molecular formula calculated from the SMILES string
|-
| SMILES
| Input molecule string
|-
| Structure Image
| RDKit-generated 2D molecular structure
|-
| logD
| Predicted distribution coefficient
|-
| Caco-2 A to B
| Predicted apical-to-basolateral permeability
|-
| Caco-2 B to A
| Predicted basolateral-to-apical permeability
|-
| Mouse PPB
| Predicted mouse plasma protein binding
|-
| Human PPB
| Predicted human plasma protein binding
|}

The CSV download contains the same prediction values in a machine-readable format.

== Troubleshooting ==

=== Site Cannot Be Reached ===

Check that:

* Backend is running on C7.
* The app is running on port 8501.
* SwitchyOmega is set to SOCKS5 localhost port 1074.
* The browser is using:

<pre>
http://169.230.26.173:8501
</pre>

=== SwitchyOmega Proxy Error ===

If Chrome says the proxy failed, the SOCKS proxy may not be running.

Start it from a Mac terminal:

<pre>
ssh -f -N -D 1074 USER@epsilon.compbio.ucsf.edu
</pre>

SwitchyOmega should be:

<pre>
Protocol: SOCKS5
Host: localhost
Port: 1074
</pre>

=== Port Already In Use ===

Check port 8501:

<pre>
ss -ltnp | grep 8501
</pre>

Only kill processes owned by you:

<pre>
kill PID_NUMBER
</pre>

=== Missing pandas or Flask ===

This usually means the conda environment is not active.

Run:

<pre>
module load anaconda3/2025.06.0
source "$(conda info --base)/etc/profile.d/conda.sh"
conda activate openadmet-models
</pre>

=== RDKit Image Error ===

Make sure the app is running inside the `openadmet-models` environment, which includes RDKit.

Test RDKit:

<pre>
python -c "from rdkit import Chem; print('rdkit works')"
</pre>

== Notes ==

ADMET Builder is a research tool. Its predictions are intended for screening, comparison, prioritization, and exploration.

Results should not be treated as experimental measurements.

For office-wide use without SSH/SOCKS tunnels, ADMET Builder should eventually be placed behind an internal hostname or proxy.

== Attribution ==

Created by Benjamin "Ben" Romero (Undergrad Northwestern University '29), Summer 2026.<br>
Developed in the Irwin Lab at UCSF.<br>
Released as part of an open-source UCSF research tool; please credit the creator and lab when using or adapting this work.

Init partitioned search.py (aka Search zinc22.py)

2026-06-22T18:51:11Z

Zdingman: corrected get_vendors argument. Does vendor-search still exist??

=== Description ===
<nowiki>
usage: init_partitioned_search.py [-i] input_file [-o] results_out [--get-vendors] [-s]

search for smiles and vendor codes by zinc22 id

required arguments:
input_file file containing list of zinc ids or vendor codes to look up
results_out destination for output directory

optional arguments:
-h, --help show this help message and exit
--vendor-search look up molecules by vendor code instead of zinc id
--get_vendors get vendor supplier codes associated with zinc id
-s submit slurm jobs and start the search process, can be omitted if you are only looking to split a list of molecules into tranches
</nowiki>

init_partitioned_search.py is a script that replaces search_zinc22.py for looking up zinc ids on the zinc22 system in a more efficient manner. The operation is simple- provide a file containing a list of zincids and the script will split the input into tranches for search across all zinc22 databases.

The output format is as follows:
<nowiki>
SMILES ZINC_ID TRANCHE_NAME</nowiki>

With --get_vendors or --vendor-search the output format looks like this:

SMILES ZINC_ID VENDOR_ID TRANCHE_NAME CATALOG</nowiki>

Meaning the script will find all vendor information and smiles associated with the provided zinc ids or vendor codes.

=== Location ===

You can activate the environment, using
source /nfs/soft/zinc22/search_zinc/miniconda/bin/activate zinc22_search

'''Side note about slow queries'''

Depending on the molecules you happen to be looking up, your search may go by very quickly, or somewhat slowly. Smaller molecules tend to look up very quickly, while larger molecules take longer to find. We're working on it.

'''Tracking Progress'''

After submitting the search, you can look at the slurm queue to see the status of your job. Jobs will be labeled 'search_zinc22'

'''Using Output'''

When all jobs are done, the output folder will have input files, log files, and .results files. If results files are missing for a tranche, refer to the logs to learn more or contact the JJI team for troubleshooting. You can merge all results together in a file with a simple command
cat *.results > yourresultsfile.txt

=== Usage w/ Bash on BKS cluster ===
<nowiki>
source /nfs/soft/zinc22/search_zinc/miniconda/bin/activate /nfs/soft/zinc22/search_zinc/miniconda/envs/zinc22_search
python /nfs/soft/zinc22/search_zinc/init_partitioned_search.py -i /path/to/input/file -o /path/to/output/folder -s
python /nfs/soft/zinc22/search_zinc/init_partitioned_search.py -i /path/to/input/file -o /path/to/output/folder -s --get_vendors
</nowiki>
=== Usage w/ Csh on BKS cluster ===

This version of the tool is currently only compatible with Bash.

=== Dealing with NULL ===
Sometimes a ZINC ID will fail to look up. This could be because a server is down (the script will notify you if this is the case), or because the ID is missing from the system for some reason. In this case, it may be helpful to separate the molecules that didn't look up from the molecules that did. You may want to save them for later when the servers come back online, or to run a deeper search with comb_legacy_files.py (more on this below).

How to:

<nowiki>
[env]$ cat legitimate_ids.txt > input.txt
[env]$ echo ZINCzz00ZZZZZZZZ >> input.txt
[env]$ echo ZINCyy00AAAAAAAA >> input.txt
[env]$ echo ZINCxx00BBBBBBBB >> input.txt
[env]$ python search_zinc.py input.txt output.txt
Searching Zinc22: |XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX| 100.0% 0.00s 23/23 complete!
[env]$ grep "_null_" output.txt
_null_ ZINCzz00ZZZZZZZZ H33P270
_null_ ZINCyy00AAAAAAAA H34P280
_null_ ZINCxx00BBBBBBBB H35P290</nowiki>

search_zinc.py will not omit IDs that don't look up from the output, instead it will return the zinc id with "_null_" in every other field. Therefore we can use grep to filter our results.

<nowiki>
[env]$ grep "_null_" output.txt > missing.txt
[env]$ grep -v "_null_" output.txt > found.txt</nowiki>

It should be very infrequent that ZINC IDs don't look up, but if this happens you can use the following script:

=== comb_legacy_files.py ===
<nowiki>
python3 /mnt/nfs/home/xyz/btingle/bin/2dload.testing/utils-2d/tin/misc/comb_legacy_files.py [INPUT_ZINC_IDS_FILE]</nowiki>

You don't need to source any particular python 3 environment for this script, but the environment used for search_zinc22.py will work just fine here.

This script will comb through our deprecated files and attempt to locate your ZINC IDs there. This script will create a file called "result" in your current directory containing all the smiles found.

If you're looking from vendor information, you can look up the SMILES you get back in arthor/smallworld sets to find vendor codes. Functionality is planned in search_zinc22.py for looking up by SMILES, but not implemented yet.

If after this you're STILL unable to find your zinc ids, you can send them to our development team and we will find them for you.

Email ben@tingle.org, ccing khtang015@gmail.com and josecastanon4@gmail.com. Include your missing file as an attachment.

Filter xREAL by Heavy Atom Count

2026-05-12T22:29:18Z

Zdingman: updated path

=== Preface ===

I wrote this quick tool to grab molecules below HAC19 from the xREAL database on Wynton. Since the database is >1 trillion compounds, I wanted to avoid the entire RDKit molecule build process if at all possible. The code removes every non-alphabetical character, lowercase letters, and the letter 'H' from a SMILES string. This leaves a list of "atoms", which can be counted to give the HAC for a compound. Assuming the HAC count falls within the specified bounds, the entire smi line is written out.

I'm assuming that the location of xREAL remains the same on disk, and that we never ever add any more compounds to xREAL. These are solid assumptions, and absolutely never ever going to be incorrect.

=== Usage ===

1. Copy the script to your working directory, and make a log dir:
cd /wynton/group/bks/work/zdingman/GAT1/zwitterion_scratch/zwitterion_scripts/HAC_filter_xREAL.sh .
mkdir logs
2. Modify the header variables defined in the header: ''HACMIN'', ''HACMAX'', ''outpath''. (When xREAL inevitably changes locations, modify ''inpath'' as well.

3. Submit the job to run:
qsub HAC_filter_xREAL.sh

Run time dependent on the Wynton queue. Anecdotally seems to run to completion overnight with no issues. Each job seems to take roughly 40 minutes (aka, frustratingly too long for the short cpu queue)

How to Use SWAG

2026-05-06T19:44:21Z

Zdingman:

SWAG is a python interface for the SmallWorld chemical database search tool, enabling users to submit a list of SMILES formatted molecules for exhaustive analog searching.

SWAG produces two output files: output_ascores.csv and output_responses.csv. The first file contains each query molecule from the input, as well as the corresponding number of analogs found within the database. The second file lists every analog found for every query molecule, and includes the SMILES, name, and edit distance.

== Usage Instructions ==

SWAG must be run on one either epyc/epyc2, and will not run on gimel/gimel2/etc. To run SWAG in a working directory, use the following commands:

source /nfs/home/zdingman/environments/SWAG/bin/activate
cp /nfs/home/zdingman/scripts/SWAG_v1-3-1/* .
python SWAG.py [arguments]

SWAG has several requirements that must be specified at run time. These are:

* The input smiles file, given with -f [YOUR_INPUT.smi]
* The prefix for output files, given with -o [YOUR_OUTPUT]
* The database for querying, given with -d [DATABASE_NAME]
* The maximum distance, given with -p [NUMBER]

If we wanted to find all the compounds in ZINC22, within 4 edits of the compounds listed in our file ligands.smi, we would run the following command:
python SWAG.py -f ligands.smi -o ligands -d ZINC22 -p [4]

== Database Selection ==

The databases used by SmallWorld are (in most cases) pre-computed by the Irwin lab for our usage, and are hard-coded into the python environment we source. The following are available to query with SWAG:

* 480K: 480 thousand building blocks from ChemSpace
* Accessible-BB: 94 million building blocks from ChemSpace + Enamine FastMADE
* REAL: 70 billion compounds from 2024 Enamine REAL Space
* ZINC22: 96 billion commercial compounds listed in ZINC22
* local: Checks the current directory for a local mapfile called 'local.anon.map'

Unlike the first four databases that refer to existing databases, specifying 'local' will search the current working directory for a map file. This is useful if you have a specific list of molecules that you want to search within. For example, you can search your top million docking vActives to find close analogs of your selected hits, or search a list of compounds from Infinisee/xREAL to find analogs by edit distance. You can read more at [[Using SWAG to filter analogs by Graph Edit Distance]].

== Advanced Usage ==

By default, SWAG takes a single integer value for the distance parameter. However, you can also pass a list of 7 integers instead as [d,d,d,d,d,d,d] which corresponds to the maximum allowed number of edits for each of the following:

* Maximum distance
* Ring count additions
* Ring count removals
* Linker length additions
* Linker length removals
* Terminal group additions
* Terminal group removals

SmallWorld will return similar compounds, beginning with the fewest number of ring/linker/terminal modifications, up to the maximum distance specified for both specific changes and overall distance.

Additional runtime flags are as follows:

--skipInputStandardization

--includeStereo

--excludeStereo

--storeQueryTime

By default, input SMILES are converted to the uncharged parent version of their largest species. Recommended to use input standardization. Otherwise, similar molecules may not be found due to differing counterions or protonation state.

By default, results found from SWAG will count differing stereoisomers as different compounds. If you would like to treat all stereoisomers as a single compound and remove them from the output, use the excludeStereo flag.

By default, SWAG does not write the per-query search time. If you would like to output the max_dist and query_time to ascores.csv, use the storeQueryTime flag.

== Final Notes ==

SWAG is built using the Cython wrapper for the SmallWorld C++ API, which relies on the gnu c compiler (GCC) to run. If running SWAG.py returns "ImportError: CXXABI_1.3.9 not found" then you must use a machine with an updated GCC version. On the Shoichet cluster, these machines are epyc and epyc2.

For further details on SmallWorld, please see the official documentation:
https://www.nextmovesoftware.com/downloads/smallworld/documentation/

[[Category:Smallworld]] [[Category:SWAG]]

Category:SWAG

2026-05-06T19:44:05Z

Zdingman: Created blank page

Using SWAG to search smaller vendors

2026-05-06T19:43:04Z

Zdingman:

In addition to the included databases that SWAG can search by default ([[How to Use SWAG|see here]]), it is possible to search any other database with a SmallWorld mapping. Recent interest has been expressed by multiple lab members in searching the vendor OnePot's CORE database for potential compounds to purchase. The below instructions describe how to search OnePot specifically, but can be adapted for any other vendor whose catalog has been mapped with SmallWorld.

== Instructions ==

Connect to either epyc or epyc2 on the lab cluster. Make a working directory in the location of your choice, open that directory, and copy query.smi (your SMILES of interest) into the directory. Copy and paste the following code:

source /nfs/home/zdingman/environments/SWAG/bin/activate
cp /nfs/home/zdingman/scripts/SWAG_v1-3-1/SWAG.py .
ln -s /nfs/db5/newdb/Purchasable/maps/onDemandPlus/onepot-tst.smi.anon.map local.anon.map
ln -s /nfs/db5/newdb/Purchasable/maps/onDemandPlus/onepot-tst.smi.anon.map.blm local.anon.map.blm
python SWAG.py -f query.smi -o lookup -d local -p [0]

SWAG will begin to lookup your SMILES. The file lookup_responses.csv contains the compounds that were successfully found in the vendor's catalog. Don't be surprised if the coverage is subpar for larger compounds; at larger HAC tranches, the overlap between ZINC22 and OnePot is fairly minimal.

== Notes ==

- If you're willing to explore single-atom changes, you can instead specify -p [1] when running SWAG.

- The above instructions here make use of SWAG's local map feature. More detail and uses for local maps can be [[Using SWAG to filter analogs by Graph Edit Distance|found here]].

[[Category:Smallworld]] [[Category:SWAG]]

Using SWAG to filter analogs by Graph Edit Distance

2026-05-06T19:42:50Z

Zdingman:

SWAG is a python interface for the SmallWorld chemical database search tool, enabling users to exhaustively find all small-change analogs for a given compound within a larger set of compounds. SWAG can be used to help filter lists of analogs created through other means (ex: fingerprint-based searches, docking hit lists, generative chemistry, enumerated spaces). We can take these larger lists, and use SWAG to filter for compounds differing by a discrete number of graph edits.

Following the procedure described here will ultimately output a list of analogs and their individual graph edit distances, relative to the compound(s) you're querying.

For a general overview of using SWAG, see [[How to Use SWAG]].

== Part 1: Initial Setup ==

Connect to epyc/epyc2, and create a working directory. Copy your list of SMILES-formatted molecules -- `analog_space.smi` -- to the working directory. Run the following:

source /nfs/home/zdingman/environments/SWAG/bin/activate
cp /nfs/home/zdingman/scripts/SWAG_v1-3-1/* .
cp /nfs/home/zdingman/scripts/GED/make_localmap.sh .
sh make_localmap.sh analog_space.smi

Doing this will generate several files:

unmapped.txt -- A list of SMILES that were unable to be mapped. Very rarely are SMILES unable to be mapped.

local.anon.map -- The unique map of your chemical space onto the larger SmallWorld index.

local.anon.map.blm -- A bloom filter used by SmallWorld to accelerate searches.

== Part 2: Filtering with SWAG ==

Filtering is straight-forward after we generate the map file from our analog space. Create the file `queries.smi`, which contains the compound(s) you want to find near analogs of in your locally mapped list of analogs. Run the below command. Note that the value in brackets corresponds to the maximum number of graph edits before the search is terminated. Generally, a distance of 2-4 is recommended. Distances greater than 7 are not supported.

python SWAG.py -f queries.smi -o queries -d local -p [2]

The following output files are generated:

queries_ascores.csv -- The number of analogs that were found, for each queried molecule.

queries_responses.csv -- A list of all analogs that were found, for each queried molecule. Columns correspond to the original query name/smiles, and the analog name/smiles/graph edit distance.

You can create a SMILES-formatted list of your small-change analogs with the following command:

awk -F',' '{print $5 " " $4}' queries_responses.csv > near_analogs.smi

== Useful flags ==

The below flags may be of use when analoging, depending on whether you're at the point of considering stereochemistry in your project.

--includeStereo

--excludeStereo

By default, results found from SWAG will count differing stereoisomers as different compounds. If you would like to treat all stereoisomers as a single compound and remove them from the output, use the excludeStereo flag.

[[Category:Smallworld]] [[Category:SWAG]]

2026-03-25T20:34:38Z

Zdingman: Updating script paths.

This is my current method (written 2024-09, updating 2026-03) of filtering LSD results, and consolidates various processing scripts used by Shoichet lab members into a single tutorial for filtering LSD results. Procedures below are adapted from various other lab sources.

'''Step 0: Source the environment and copy over relevant files'''

Source JK's environment with the following commands, accessible on gimel:

source /mnt/nfs/home/jklyu/anaconda3/etc/profile.d/conda.sh
conda activate bioinfo-env2
if ! ( $?PYTHONPATH ) setenv PYTHONPATH ""
setenv PYTHONPATH $PYTHONPATH\:/mnt/nfs/ex5/work/jklyu/IFP/package/ifp/scripts\:

Subsequently, copy the needed scripts:

cp /nfs/home/zdingman/scripts/IFP/jk_scripts ./scripts

'''Step 1: Interaction fingerprinting'''

Copy your initially extracted poses to prep_poses. These poses will need to be split so we can run them in parallel, so run the requisite script(s). The fewer poses you have in your mol2 files, the faster they will complete. Each mol2 file must be named 'filtered-#.mol2', owing to how the underlying scripts were written by JK. Rename as follows:

mkdir output_renamed
sh rename_utility.sh

Anecdotally, it takes about 6 hours to fingerprint a file with 2000 molecules in it. [Tia and Andrii both have splitting scripts. Alternatively, just use the poses_extract files, but note that the larger # of mol2 per file will run slower.]

Once your mol2 files are split, relocate them to your working directory. This is necessary. Place your pdb file in the working directory.

Generate a dirlist of mol2 files, and run from the working directory:

ls *.mol2 > dirlist
csh ../scripts/submit.csh [rec name.pdb]

Now we need to combine the output directories back together:

ls -d [0-9]* > dirlist_combine
python2.7 ../scripts/combine_ifp.py dirlist_combine combined

At this point, our interactions have been identified. We can view them in combined.interactions.csv and use the awk command to fetch a list of compounds matching our desired interaction patterns. For example:

awk -F, '$4 <= 1 && ($8 == 1 || $9 == 1) {print $1}' combined.interactions.csv > filtered_compounds

Note: awk is a powerful command-line tool that uses its own scripting language (essentially regex with additional features added on top) to process multi-column text files on a line-by-line basis. The above command opens the file 'combined.interactions.csv' and separates each line into multiple columns separated by commas, as specified by the -F flag. Columns are 1-indexed and can be accessed using $1, analogous to how bash scripting arguments are accessed. The operation being executed is specified within single-quotes; for our purposes we usually want to use a combination of logic operators as above. OR statements (||) are especially useful, especially in cases where multiple interactions are possible (e.x. either a hydrophobic interaction or a pi-stacking interaction with a phenylalanine). For lines which match the pattern specified, awk will output the contents contained within curly braces. We then redirect that output to a new file titled 'filtered_compounds' for future use.

awk is worth learning in greater detail. While far from exhaustive, I found [https://www.geeksforgeeks.org/awk-command-unixlinux-examples/ this page] and [https://www.freecodecamp.org/news/the-linux-awk-command-linux-and-unix-usage-syntax-examples/ this page] to be useful starting points.

Unfortunately, our process of splitting mol2 files and recombining them has scrambled the rank ordering of our compounds. This will cause issues later. I will write a script to deal with this later.

For now, we have a list of compounds that pass the interaction fingerprinting step.

'''Step 2: Novelty filtering'''

I've been making use of Olivier's method for novelty, which can be [[Olivier's way of computing novelty|found here]].

It has been recommended to restrict the results on ChEMBL for a target protein to just those compounds annotated with binding (EC50, IC50, etc) data. Additionally, it can be worthwhile with targets we have previously screened internally to include a second novelty filter for previously identified/purchased compounds.

'''Step 3: Best first clustering'''

Note that the input SMILES need to be in rank-order.

[[Category:Zoey's Way]]

How to Use SWAG

2026-03-14T00:58:45Z

Zdingman: Added link to analog filtering page.

2024-09-19T19:57:38Z

Zdingman:

Zoey's way of filtering LSD

2024-09-19T19:06:58Z

Zdingman: Created page with "This is my current method (2024-09) of filtering LSD results, and the culmination of discussion with several other lab members to determine the "best" way to filter LSD results. Procedures below are adapted from various other lab sources. '''Step 0: Source the environment and copy over relevant files''' '''Step 1: Interaction fingerprinting''' '''Step 2: Novelty filtering''' I've been making use of Olivier's method for novelty, which can be Olivier's way of computin..."

User:Zdingman

2024-09-19T18:08:03Z

Zdingman:

Allegedly a girl working in the Shoichet laboratory.

=== Zoey's Way ===

This is largely a placeholder section, which I intend on fleshing out as I establish and standardize more of my procedures.

=== Papers I Either Like, or Like to Think About ===

''Can a biologist fix a radio?— Or, what I learned while studying apoptosis''

Yuri Lazebnik, in Cancer Cell 2002

https://doi.org/10.1016/S1535-6108(02)00133-2

--

''Modern biomedical research: an internally self-consistent universe with little contact with medical reality?''

David F. Horrobin, in Nature Reviews Drug Discovery 2003

https://doi.org/10.1038/nrd1012

User:Zdingman

2024-06-13T20:48:52Z

Zdingman: Beginning a section of conceptual papers that have shaped my attitude towards research. Will add more.

Allegedly a girl working in the Shoichet laboratory.

=== Papers I Either Like, or Like to Think About ===

''Can a biologist fix a radio?— Or, what I learned while studying apoptosis''

Yuri Lazebnik, in Cancer Cell 2002

https://doi.org/10.1016/S1535-6108(02)00133-2

--

''Modern biomedical research: an internally self-consistent universe with little contact with medical reality?''

David F. Horrobin, in Nature Reviews Drug Discovery 2003

https://doi.org/10.1038/nrd1012

Dockopt (pydock3 script)

2024-06-13T20:33:57Z

Zdingman: Corrected the flag --job_dir_name to --job_dir_path per the dockopt new help page.

''dockopt'' allows the generation of many different docking configurations which are then evaluated & analyzed in parallel using a specified job scheduler (e.g. Slurm). If you are a Shoichet Lab user, please see a special section for you, below.

To use DOCK 3.8, you must first license it and install it.
[[DOCK 3.8:How to install pydock3]]

== Note for UCSF Shoichet Lab members ==

''pydock3'' is already installed on the following clusters. You can source the provided Python environment scripts to expose the ''pydock3'' executable and declare default environmental variables:

=== Wynton ===

source /wynton/group/bks/soft/python_envs/env.sh

=== Gimel ===

Only nodes other than ''gimel'' itself are supported, e.g., ''gimel5''.

ssh gimel5
source /nfs/soft/ian/env.sh

== Subcommand: ''new'' ==

Prepare rec.pdb, xtal-lig.pdb as described in Bender, 2021. https://pubmed.ncbi.nlm.nih.gov/34561691/
Or download pre-preared sample files from dudez2022.docking.org.

Be sure that you are in the directory containing the required input files:
* ''rec.pdb'' or ''rec.crg.pdb''
* ''xtal-lig.pdb''
* ''actives.tgz''
* ''decoys.tgz''

Note the inclusion of ''actives.tgz'' and ''actives.tgz''. Each of these is a tarball of DB2 files, which may be optionally gzipped (with extension .db2.gz). Each tarball represents a binary class for the binary classification task that the docking model is trained on. The positive class (actives) contains the molecules that you want the docking model to preferentially assign ''favorable'' docking scores to. The negative class (decoys) contains the molecules that you want the docking model to preferentially assign ''unfavorable'' scores to. The most common strategy is to use a set of known ligands as the actives and a larger set of property-matched decoys for the decoys (a decoy-to-active ratio of 50:1 is standard), but other strategies are supported. For example, to create a docking model that preferentially assigns favorable scores to agonists over antagonists, a set of agonists can be used as the actives and a set of antagonists can be used for the decoys.

Therefore, you need to build the molecules yourself (see: https://tldr.docking.org/start/build3d38). Each tarball should contain only DB2 files. For example, if one has a directory ''actives/'' containing only DB2 files to use as the actives, then ''acitves.tgz'' should be created as follows:

cd actives/
tar -czf actives.tgz *.db2*

Similarly, for a directory ''decoys/'' containing only DB2 files to use as decoys, ''decoys.tgz'' should be created as follows:

cd decoys/
tar -czf decoys.tgz *.db2*

To create the file structure for your dockopt job, simply type

pydock3 dockopt - new

By default, the job directory is named ''dockopt_job''. To specify a different name, use the "--job_dir_path" flag. E.g.:

pydock3 dockopt - new --job_dir_path=dockopt_job_2

The job directory contains two sub-directories:
# ''working'': input files, intermediate blaster files, sub-directories for individual blastermaster subroutines
# ''retrodock_jobs'': individual retrodock jobs for each docking configuration

The key difference between the working directories of ''blastermaster'' and ''dockopt'' is that the working directory of ''dockopt'' may contain multiple variants of the blaster files (suffixed by a number, e.g. "box_1"). These variant files are used to create the different docking configurations specified by the multi-valued entries of ''dockopt_config.yaml''. They are created efficiently, such that the same variant used in multiple docking configurations is not created more than once.

If your current working directory contains any of the following files, then they will be automatically copied into the working directory within the created job directory. This feature is intended to simplify the process of configuring the dockopt job.

* ''rec.pdb''
* ''rec.crg.pdb''
* ''xtal-lig.pdb''
* ''reduce_wwPDB_het_dict.txt''
* ''filt.params''
* ''radii''
* ''amb.crg.oxt''
* ''vdw.siz''
* ''delphi.def''
* ''vdw.parms.amb.mindock''
* ''prot.table.ambcrg.ambH''

Only the following are required. Default versions / generated versions of the others will be used instead if they are not detected.
* ''rec.pdb'' or ''rec.crg.pdb''. Either is required, but not both. If both are present, only ''rec.crg.pdb'' is used.
* ''xtal-lig.pdb''

If you would like to use files not present in your current working directory, copy them into your job's working directory, e.g.:
cp <FILE_PATH> <JOB_DIR_NAME>/working/

Finally, configure the ''dockopt_config.yaml'' file in the job directory to your specifications. The parameters in this file govern the behavior of dockopt.

'''Note:''' The ''dockopt_config.yaml'' file differs from the ''blastermaster_config.yaml'' file in that every parameter of the former may accept either a single value or a ''list of comma-separated values'', which indicates a pool of values to attempt for that parameter. Multiple such multi-valued parameters may be provided, and all unique resultant docking configurations will be attempted.

Single-valued YAML line format:

distance_to_surface: 1.0

Multi-valued YAML line format:

distance_to_surface: [1.0, 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1.9]

== Environmental variables ==

=== TMPDIR ===

Designate where temporary job files should be placed. E.g.:

export TMPDIR=/scratch

==== Note for UCSF researchers ====

On the Wynton cluster, ''/scratch'' only exists on development nodes (not log nodes). Therefore, we recommend running on development nodes (see: https://wynton.ucsf.edu/hpc/get-started/development-prototyping.html). E.g.:

ssh dev1
export TMPDIR=/scratch

If a log node must be used, then ''/wynton/scratch'' may be used:

ssh log1
export TMPDIR=/wynton/scratch

=== job scheduler environmental variables ===

In order for ''dockopt'' to know which scheduler it should use, please configure the following environmental variables according to which one of the job schedulers you have.

==== Slurm ====

E.g., on the UCSF Shoichet Lab Gimel cluster (on any node other than 'gimel' itself, such as 'gimel5'):

export SBATCH_EXEC=/usr/bin/sbatch
export SQUEUE_EXEC=/usr/bin/squeue

==== SGE ====

On most clusters using SGE the following should be correct:

export QSTAT_EXEC=/opt/sge/bin/lx-amd64/qstat
export QSUB_EXEC=/opt/sge/bin/lx-amd64/qsub
export SGE_SETTINGS=/opt/sge/default/common/settings.sh

===== Note for UCSF researchers =====

The following is necessary on the UCSF Wynton cluster:

export QSTAT_EXEC=/opt/sge/bin/lx-amd64/qstat
export QSUB_EXEC=/opt/sge/bin/lx-amd64/qsub
export SGE_SETTINGS=/opt/sge/wynton/common/settings.sh

== Subcommand: ''run'' ==

Once your job has been configured to your liking, navigate to the the job directory and run ''dockopt'':
cd <JOB_DIR_NAME>
pydock3 dockopt - run <JOB_SCHEDULER_NAME> [--retrodock_job_timeout_minutes=None] [--retrodock_job_max_reattempts=0] [--extra_submission_cmd_params_str=None] [--export_decoys_mol2=False]

where <JOB_SCHEDULER_NAME> is one of:
* ''sge''
* ''slurm''

This will execute the many dockopt subroutines in sequence. Once this is done, the retrodock jobs for all created docking configurations are run in parallel via the scheduler. The state of the program will be printed to standard output as it runs.

Once the dockopt job is complete, the following files will be generated in the job directory:
* ''report.html'': contains (1) a histogram of the performance of all tested docking configurations compared against a distribution of the performance of a random classifier, so as to show whether the test docking configurations are significantly better than ones that can be produced by a random classifier. This is necessary due to the fact that many configurations are being tested. Hence, a Bonferroni correction is applied to the significance threshold, dividing p=0.01 by the number of tested configurations. (2) ROC, charge, and energy plots of the top docking configurations, comparing actives vs. decoys, (3) box plots of enrichment for every multi-valued config parameter, and (4) heatmaps of enrichment for every pair of multi-valued config parameters.
* ''results.csv'': parameter values, criterion values, and other information about each docking configuration.

In addition, some number of the best retrodock jobs will be copied to their own sub-directory ''best_retrodock_jobs/''.

Within each sub-directory of ''best_retrodock_jobs/'', there are:
* ''dockfiles/'': parameters files and ''INDOCK'' for given docking configuration
* ''output/'': contains:
** sub-directories ''actives/'' (containing ''OUTDOCK'' and ''test.mol2'' files) and ''decoys/'' (containing just ''OUTDOCK'')
* plot images (e.g., ''roc.png'')

'''Note:''' by default, a mol2 file is exported only for actives, not for decoys, in order to prevent disk space issues.