Init partitioned search.py (aka Search zinc22.py) - Revision history

Jocastanon: Jocastanon moved page Search zinc22.py to Init partitioned search.py (aka Search zinc22.py): Added wrapper script for splitting into tranches

2025-02-18T20:13:52Z

Jocastanon moved page Search zinc22.py to Init partitioned search.py (aka Search zinc22.py): Added wrapper script for splitting into tranches

← Older revision

Revision as of 20:13, 18 February 2025

Jocastanon at 21:25, 3 December 2024

2024-12-03T21:25:30Z

← Older revision		Revision as of 21:25, 3 December 2024
Line 1:		Line 1:
	=== Description ===		=== Description ===
	<nowiki>		<nowiki>
	usage: ~~search_zinc22~~.py [-h] [-~~-vendor-search~~] [--get-vendors] [-~~-configuration-server-url CONFIGURATION_SERVER_URL~~] ~~input_file results_out~~		usage: init_partitioned_search.py [-i] input_file [-o] results_out [--get-vendors] [-s]

	search for smiles by zinc22 id		search for smiles and vendor codes by zinc22 id

	~~positional~~ arguments:		required arguments:
	input_file file containing list of zinc ids or vendor codes to look up		input_file file containing list of zinc ids or vendor codes to look up
	results_out destination ~~file~~ for output		results_out destination for output directory

	optional arguments:		optional arguments:
Line 13:		Line 13:
	--vendor-search look up molecules by vendor code instead of zinc id		--vendor-search look up molecules by vendor code instead of zinc id
	--get-vendors get vendor supplier codes associated with zinc id		--get-vendors get vendor supplier codes associated with zinc id
	-~~-configuration-server-url CONFIGURATION_SERVER_URL~~		-s submit slurm jobs and start the search process, can be omitted if you are only looking to split a list of molecules into tranches
	~~database containing configuration for zinc22 system~~
	</nowiki>		</nowiki>

	~~search_zinc22~~.py is a script for looking up zinc ids on the zinc22 system. The operation is simple- provide a file containing a list of zincids and ~~a destination file to write to. The script will give you a progress bar as it searches the system. If a database is down,~~ the script will ~~let you know and continue gathering~~ the ~~results it can~~.		init_partitioned_search.py is a script that replaces search_zinc22.py for looking up zinc ids on the zinc22 system in a more efficient manner. The operation is simple- provide a file containing a list of zincids and the script will split the input into tranches for search across all zinc22 databases.

	The output format is as follows:		The output format is as follows:
	<nowiki>		<nowiki>
	SMILES ZINC_ID TRANCHE_NAME</nowiki>		SMILES ZINC_ID TRANCHE_NAME</nowiki>

	With --get-vendors or --vendor-search the output format looks like this:		With --get-vendors or --vendor-search the output format looks like this:
	~~<nowiki>~~
	SMILES ZINC_ID VENDOR_ID TRANCHE_NAME CATALOG</nowiki>		SMILES ZINC_ID VENDOR_ID TRANCHE_NAME CATALOG</nowiki>

Line 31:		Line 30:
	=== Location ===		=== Location ===

	You can ~~find~~ the ~~script and~~ environment @ /nfs/soft/zinc22/search_zinc ~~on the BKS cluster.~~		You can activate the environment, using
			source /nfs/soft/zinc22/search_zinc/miniconda/bin/activate zinc22_search

			'''Side note about slow queries'''

	If you ~~get an error along the lines of "ImportError: /lib64/libc~~.~~so.6: version `GLIBC_2~~.14' ~~not found" after sourcing the environment and attempting to run the script, you can try using an alternative environment @ ~xyz/btingle/bin/2dload~~.~~testing/py36_psycopg2~~		Depending on the molecules you happen to be looking up, your search may go by very quickly, or somewhat slowly. Smaller molecules tend to look up very quickly, while larger molecules take longer to find. We're working on it.

	~~This happens because the required software libraries are not installed on whichever machine you are logged in to.~~		'''Tracking Progress'''

	~~You~~ can ~~log into one of our development nodes (ssh user@n-1-17, ssh user@n-1-16, etc.) which are guaranteed~~ to ~~have~~ the ~~correct libraries and run it there~~.		After submitting the search, you can look at the slurm queue to see the status of your job. Jobs will be labeled 'search_zinc22'

	'''~~Side note about slow queries~~'''		'''Using Output'''

	~~Depending on~~ the ~~molecules you happen to be looking up~~, ~~your search may go by very quickly~~, ~~or somewhat slowly~~. ~~Smaller molecules tend~~ to ~~look up very quickly, while larger molecules take longer~~ to ~~find~~. ~~We're working on it~~.		When all jobs are done, the output folder will have input files, log files, and .results files. If results files are missing for a tranche, refer to the logs to learn more or contact the JJI team for troubleshooting. You can merge all results together in a file with a simple command
			cat *.results > yourresultsfile.txt

	=== Usage w/ Bash on BKS cluster ===		=== Usage w/ Bash on BKS cluster ===
	<nowiki>		<nowiki>
	source /nfs/soft/zinc22/search_zinc/miniconda/bin/activate /nfs/soft/zinc22/search_zinc/miniconda/envs/zinc22_search		source /nfs/soft/zinc22/search_zinc/miniconda/bin/activate /nfs/soft/zinc22/search_zinc/miniconda/envs/zinc22_search
	python /nfs/soft/zinc22/search_zinc/~~search_zinc22~~.py ~~input_zinc_ids.txt output_zinc_ids.txt~~		python /nfs/soft/zinc22/search_zinc/init_partitioned_search.py -i /path/to/input/file -o /path/to/output/folder -s
	python /nfs/soft/zinc22/search_zinc/~~search_zinc22~~.py --~~get~~-~~vendors input_zinc_ids.txt output_vendor_ids.txt~~</nowiki>		python /nfs/soft/zinc22/search_zinc/init_partitioned_search.py -i /path/to/input/file -o /path/to/output/folder -s --get_vendors
			</nowiki>
			=== Usage w/ Csh on BKS cluster ===

	~~=== Usage w/ Csh on BKS cluster ===~~		This version of the tool is currently only compatible with Bash.
	~~<nowiki>~~
	~~source /nfs/soft/zinc22/search_zinc/env/bin/activate~~.~~csh~~
	~~python /nfs/soft/zinc22/search_zinc/search_zinc22.py input_zinc_ids.txt output_zinc_ids.txt~~
	~~python /nfs/soft/zinc22/search_zinc/search_zinc22.py --get-vendors input_zinc_ids.txt output_vendor_ids.txt</nowiki>~~

	=== Dealing with NULL ===		=== Dealing with NULL ===
Line 70:		Line 71:
	_null_ ZINCzz00ZZZZZZZZ H33P270		_null_ ZINCzz00ZZZZZZZZ H33P270
	_null_ ZINCyy00AAAAAAAA H34P280		_null_ ZINCyy00AAAAAAAA H34P280
	_null_ ZINCxx00BBBBBBBB H35P290		_null_ ZINCxx00BBBBBBBB H35P290</nowiki>
	</nowiki>

	search_zinc.py will not omit IDs that don't look up from the output, instead it will return the zinc id with "_null_" in every other field. Therefore we can use grep to filter our results.		search_zinc.py will not omit IDs that don't look up from the output, instead it will return the zinc id with "_null_" in every other field. Therefore we can use grep to filter our results.

Khtang: /* Usage w/ Bash on BKS cluster */

2024-07-11T18:41:55Z

Usage w/ Bash on BKS cluster

← Older revision		Revision as of 18:41, 11 July 2024
Line 45:		Line 45:
	=== Usage w/ Bash on BKS cluster ===		=== Usage w/ Bash on BKS cluster ===
	<nowiki>		<nowiki>
	source /nfs/soft/zinc22/search_zinc/~~env~~/bin/activate		source /nfs/soft/zinc22/search_zinc/miniconda/bin/activate /nfs/soft/zinc22/search_zinc/miniconda/envs/zinc22_search
	python /nfs/soft/zinc22/search_zinc/search_zinc22.py input_zinc_ids.txt output_zinc_ids.txt		python /nfs/soft/zinc22/search_zinc/search_zinc22.py input_zinc_ids.txt output_zinc_ids.txt
	python /nfs/soft/zinc22/search_zinc/search_zinc22.py --get-vendors input_zinc_ids.txt output_vendor_ids.txt</nowiki>		python /nfs/soft/zinc22/search_zinc/search_zinc22.py --get-vendors input_zinc_ids.txt output_vendor_ids.txt</nowiki>

Jgutierrez6 at 21:15, 24 June 2022

2022-06-24T21:15:41Z

← Older revision		Revision as of 21:15, 24 June 2022
Line 23:		Line 23:
	SMILES ZINC_ID TRANCHE_NAME</nowiki>		SMILES ZINC_ID TRANCHE_NAME</nowiki>

	With --get-vendors the output format looks like this:		With --get-vendors or --vendor-search the output format looks like this:
	<nowiki>		<nowiki>
	SMILES ZINC_ID VENDOR_ID TRANCHE_NAME CATALOG</nowiki>		SMILES ZINC_ID VENDOR_ID TRANCHE_NAME CATALOG</nowiki>

	Meaning the script will find all vendor information and smiles associated with the provided zinc ids.		Meaning the script will find all vendor information and smiles associated with the provided zinc ids or vendor codes.

	=== Location ===		=== Location ===

Jgutierrez6 at 21:14, 24 June 2022

2022-06-24T21:14:32Z

← Older revision		Revision as of 21:14, 24 June 2022
Line 1:		Line 1:
	=== Description ===		=== Description ===
	<nowiki>		<nowiki>
	usage: search_zinc22.py [-h] [--get-vendors]		usage: search_zinc22.py [-h] [--vendor-search] [--get-vendors] [--configuration-server-url CONFIGURATION_SERVER_URL] input_file results_out
	[--configuration-server-url CONFIGURATION_SERVER_URL]
	~~zinc_id_in~~ results_out

	search for smiles by zinc22 id		search for smiles by zinc22 id

	positional arguments:		positional arguments:
	~~zinc_id_in~~ file containing list of zinc ids to look up		input_file file containing list of zinc ids or vendor codes to look up
	results_out destination file for output		results_out destination file for output

	optional arguments:		optional arguments:
	-h, --help show this help message and exit		-h, --help show this help message and exit
			--vendor-search look up molecules by vendor code instead of zinc id
	--get-vendors get vendor supplier codes associated with zinc id		--get-vendors get vendor supplier codes associated with zinc id
	--configuration-server-url CONFIGURATION_SERVER_URL		--configuration-server-url CONFIGURATION_SERVER_URL

Btingle: /* comb_legacy_files.py */

2022-06-13T23:58:03Z

comb_legacy_files.py

← Older revision		Revision as of 23:58, 13 June 2022
Line 90:		Line 90:
	This script will comb through our deprecated files and attempt to locate your ZINC IDs there. This script will create a file called "result" in your current directory containing all the smiles found.		This script will comb through our deprecated files and attempt to locate your ZINC IDs there. This script will create a file called "result" in your current directory containing all the smiles found.

	If you're looking from vendor information, you can look up the SMILES you get back in arthor/smallworld ~~enamine~~ sets to find vendor codes. Functionality is planned in search_zinc22.py for looking up by SMILES, but not implemented yet.		If you're looking from vendor information, you can look up the SMILES you get back in arthor/smallworld sets to find vendor codes. Functionality is planned in search_zinc22.py for looking up by SMILES, but not implemented yet.

	If after this you're STILL unable to find your zinc ids, you can send them to our development team and we will find them for you.		If after this you're STILL unable to find your zinc ids, you can send them to our development team and we will find them for you.

	Email ben@tingle.org, ccing khtang015@gmail.com and josecastanon4@gmail.com. Include your missing file as an attachment.		Email ben@tingle.org, ccing khtang015@gmail.com and josecastanon4@gmail.com. Include your missing file as an attachment.

Btingle: /* comb_legacy_files.py */

2022-06-13T23:53:44Z

comb_legacy_files.py

← Older revision		Revision as of 23:53, 13 June 2022
Line 89:		Line 89:

	This script will comb through our deprecated files and attempt to locate your ZINC IDs there. This script will create a file called "result" in your current directory containing all the smiles found.		This script will comb through our deprecated files and attempt to locate your ZINC IDs there. This script will create a file called "result" in your current directory containing all the smiles found.

			If you're looking from vendor information, you can look up the SMILES you get back in arthor/smallworld enamine sets to find vendor codes. Functionality is planned in search_zinc22.py for looking up by SMILES, but not implemented yet.

	If after this you're STILL unable to find your zinc ids, you can send them to our development team and we will find them for you.		If after this you're STILL unable to find your zinc ids, you can send them to our development team and we will find them for you.

	Email ben@tingle.org, ccing khtang015@gmail.com and josecastanon4@gmail.com. Include your missing file as an attachment.		Email ben@tingle.org, ccing khtang015@gmail.com and josecastanon4@gmail.com. Include your missing file as an attachment.

Btingle at 23:14, 10 June 2022

2022-06-10T23:14:30Z

← Older revision		Revision as of 23:14, 10 June 2022
Line 1:		Line 1:
	==== Description ====		=== Description ===
	<nowiki>		<nowiki>
	usage: search_zinc22.py [-h] [--get-vendors]		usage: search_zinc22.py [-h] [--get-vendors]
Line 30:		Line 30:
	Meaning the script will find all vendor information and smiles associated with the provided zinc ids.		Meaning the script will find all vendor information and smiles associated with the provided zinc ids.

	==== Location ====		=== Location ===

	You can find the script and environment @ /nfs/soft/zinc22/search_zinc on the BKS cluster.		You can find the script and environment @ /nfs/soft/zinc22/search_zinc on the BKS cluster.
Line 44:		Line 44:
	Depending on the molecules you happen to be looking up, your search may go by very quickly, or somewhat slowly. Smaller molecules tend to look up very quickly, while larger molecules take longer to find. We're working on it.		Depending on the molecules you happen to be looking up, your search may go by very quickly, or somewhat slowly. Smaller molecules tend to look up very quickly, while larger molecules take longer to find. We're working on it.

	==== Usage w/ Bash on BKS cluster ====		=== Usage w/ Bash on BKS cluster ===
	<nowiki>		<nowiki>
	source /nfs/soft/zinc22/search_zinc/env/bin/activate		source /nfs/soft/zinc22/search_zinc/env/bin/activate
Line 50:		Line 50:
	python /nfs/soft/zinc22/search_zinc/search_zinc22.py --get-vendors input_zinc_ids.txt output_vendor_ids.txt</nowiki>		python /nfs/soft/zinc22/search_zinc/search_zinc22.py --get-vendors input_zinc_ids.txt output_vendor_ids.txt</nowiki>

	==== Usage w/ Csh on BKS cluster ====		=== Usage w/ Csh on BKS cluster ===
	<nowiki>		<nowiki>
	source /nfs/soft/zinc22/search_zinc/env/bin/activate.csh		source /nfs/soft/zinc22/search_zinc/env/bin/activate.csh
Line 56:		Line 56:
	python /nfs/soft/zinc22/search_zinc/search_zinc22.py --get-vendors input_zinc_ids.txt output_vendor_ids.txt</nowiki>		python /nfs/soft/zinc22/search_zinc/search_zinc22.py --get-vendors input_zinc_ids.txt output_vendor_ids.txt</nowiki>

	==== Dealing with NULL ====		=== Dealing with NULL ===
	Sometimes a ZINC ID will fail to look up. This could be because a server is down (the script will notify you if this is the case), or because the ID is missing from the system for some reason. In this case, it may be helpful to separate the molecules that didn't look up from the molecules that did. You may want to save them for later when the servers come back online, or to run a deeper search with comb_legacy_files.py (more on this below).		Sometimes a ZINC ID will fail to look up. This could be because a server is down (the script will notify you if this is the case), or because the ID is missing from the system for some reason. In this case, it may be helpful to separate the molecules that didn't look up from the molecules that did. You may want to save them for later when the servers come back online, or to run a deeper search with comb_legacy_files.py (more on this below).

Line 82:		Line 82:
	It should be very infrequent that ZINC IDs don't look up, but if this happens you can use the following script:		It should be very infrequent that ZINC IDs don't look up, but if this happens you can use the following script:

	==== comb_legacy_files.py ====		=== comb_legacy_files.py ===
	<nowiki>		<nowiki>
	python3 /mnt/nfs/home/xyz/btingle/bin/2dload.testing/utils-2d/tin/misc/comb_legacy_files.py [INPUT_ZINC_IDS_FILE]</nowiki>		python3 /mnt/nfs/home/xyz/btingle/bin/2dload.testing/utils-2d/tin/misc/comb_legacy_files.py [INPUT_ZINC_IDS_FILE]</nowiki>

Btingle: /* Location */

2022-06-10T23:13:48Z

Location

← Older revision		Revision as of 23:13, 10 June 2022
Line 41:		Line 41:

	'''Side note about slow queries'''		'''Side note about slow queries'''

	Depending on the molecules you happen to be looking up, your search may go by very quickly, or somewhat slowly. Smaller molecules tend to look up very quickly, while larger molecules take longer to find. We're working on it.		Depending on the molecules you happen to be looking up, your search may go by very quickly, or somewhat slowly. Smaller molecules tend to look up very quickly, while larger molecules take longer to find. We're working on it.

Btingle: /* Location */

2022-06-10T23:13:40Z

Location

← Older revision		Revision as of 23:13, 10 June 2022
Line 40:		Line 40:
	You can log into one of our development nodes (ssh user@n-1-17, ssh user@n-1-16, etc.) which are guaranteed to have the correct libraries and run it there.		You can log into one of our development nodes (ssh user@n-1-17, ssh user@n-1-16, etc.) which are guaranteed to have the correct libraries and run it there.

	Side note about slow queries		'''Side note about slow queries'''
	Depending on the molecules you happen to be looking up, your search may go by very quickly, or somewhat slowly. Smaller molecules tend to look up very quickly, while larger molecules take longer to find. We're working on it.		Depending on the molecules you happen to be looking up, your search may go by very quickly, or somewhat slowly. Smaller molecules tend to look up very quickly, while larger molecules take longer to find. We're working on it.

	==== Usage w/ Bash on BKS cluster ====		==== Usage w/ Bash on BKS cluster ====