Repackaging DB2 DOCK38 - Revision history

Btingle at 22:02, 27 March 2023

2023-03-27T22:02:45Z

← Older revision		Revision as of 22:02, 27 March 2023
Line 63:		Line 63:
	ai		ai
	aj		aj
	Done! Results in /~~tmp~~/~~user/output~~</nowiki>		Done! Results in $PWD/tarballs_repacked/H17P200_H19P400</nowiki>

	It should be noted that this script will be effective for fairly small batches of molecules, e.g on the range of millions, rather than billions of molecules. Talk to me (ben@tingle.org) or John Irwin for more information on how to repack Very Large ligand libraries.		It should be noted that this script will be effective for fairly small batches of molecules, e.g on the range of millions, rather than billions of molecules. Talk to me (ben@tingle.org) or John Irwin for more information on how to repack Very Large ligand libraries.

Btingle at 21:59, 27 March 2023

2023-03-27T21:59:45Z

← Older revision		Revision as of 21:59, 27 March 2023
Line 43:		Line 43:
	done		done
	cd ..		cd ..
	echo Done! Results in $~~PWD/output~~</nowiki>		rm -r $WORKING_DIRECTORY
			echo Done! Results in $TARBALL_REPACK_DEST</nowiki>

	Now, an example usage:		Now, an example usage:

	<nowiki>		<nowiki>
	[user@gimel5 ~] bash make_tarballs.bash ~~/nfs/exb/zinc22/tarballs~~/H17P200_H19P400.smi.batch-3d.d/out/~~aaa.d~~		[user@gimel5 ~] bash make_tarballs.bash $PWD/H17P200_H19P400.smi.batch-3d.d/out $PWD/tarballs_repacked/H17P200_H19P400
	finding		finding
	splitting		splitting
Line 64:		Line 65:
	Done! Results in /tmp/user/output</nowiki>		Done! Results in /tmp/user/output</nowiki>

	For docking from ligands built using our pipeline with default options, running this script unmodified is sufficient for creating appropriately sized packages for docking. You may wish to edit WORKING_DIRECTORY if ~~your files total more than 50G in size-~~ /tmp is not ~~usually any larger than this!~~		It should be noted that this script will be effective for fairly small batches of molecules, e.g on the range of millions, rather than billions of molecules. Talk to me (ben@tingle.org) or John Irwin for more information on how to repack Very Large ligand libraries.

			For docking from ligands built using our pipeline with default options, running this script unmodified is sufficient for creating appropriately sized packages for docking. You may wish to edit WORKING_DIRECTORY to /scratch or some other larger directory if running out of space on /tmp is a concern. The /tmp directory typically only holds around 50G of data, which may not be enough for some workloads or environments.

	[[Category:DOCK_3.8]]		[[Category:DOCK_3.8]]

Btingle at 18:11, 16 March 2023

2023-03-16T18:11:17Z

← Older revision		Revision as of 18:11, 16 March 2023
Line 8:		Line 8:
	TARBALL_SOURCE=$1		TARBALL_SOURCE=$1
	TARBALL_REPACK_DEST=$2		TARBALL_REPACK_DEST=$2

			TARBALL_SOURCE=$(realpath $TARBALL_SOURCE)
			TARBALL_REPACK_DEST=$(realpath $TARBALL_REPACK_DEST)

	[ -z $TARBALL_SOURCE ] && echo "need to provide TARBALL_SOURCE as 1st arg!" && exit 1		[ -z $TARBALL_SOURCE ] && echo "need to provide TARBALL_SOURCE as 1st arg!" && exit 1
Line 30:		Line 33:
	for f in ../tarball_split_list/*; do		for f in ../tarball_split_list/*; do
	for tb in $(cat $f); do		for tb in $(cat $f); do
			! [ -z $VERBOSE ] && echo tar --transform='s/^.\///' -xf $tb '.'$PACKAGE_TYPE 2>/dev/null
	tar --transform='s/^.\///' -xf $tb '.'$PACKAGE_TYPE 2>/dev/null		tar --transform='s/^.\///' -xf $tb '.'$PACKAGE_TYPE 2>/dev/null
	done		done
			! [ -z $VERBOSE ] && echo tar -czf $(basename $f).$PACKAGE_TYPE.tar.gz '*.'$PACKAGE_TYPE
	tar -czf $(basename $f).$PACKAGE_TYPE_SHORT.tar.gz *.$PACKAGE_TYPE		tar -czf $(basename $f).$PACKAGE_TYPE_SHORT.tar.gz *.$PACKAGE_TYPE
	mv $(basename $f).$PACKAGE_TYPE_SHORT.tar.gz $TARBALL_REPACK_DEST		mv $(basename $f).$PACKAGE_TYPE_SHORT.tar.gz $TARBALL_REPACK_DEST

Btingle at 18:05, 16 March 2023

2023-03-16T18:05:16Z

← Older revision		Revision as of 18:05, 16 March 2023
Line 6:		Line 6:

	# required parameter		# required parameter
	~~RESULT_DIRECTORY~~=$1		TARBALL_SOURCE=$1
			TARBALL_REPACK_DEST=$2

			[ -z $TARBALL_SOURCE ] && echo "need to provide TARBALL_SOURCE as 1st arg!" && exit 1
			[ -z $TARBALL_REPACK_DEST ] && echo "need to provide TARBALL_REPACK_DEST as 2nd arg!" && exit 1

	# optional parameters		# optional parameters
Line 19:		Line 23:

	echo finding		echo finding
	find $~~RESULT_DIRECTORY~~ -name '*.tar.gz' > tarball_list.txt		find $TARBALL_SOURCE -name '*.tar.gz' > tarball_list.txt
	echo splitting		echo splitting
	split -l $PACKAGES_PER_PACKAGE tarball_list.txt tarball_split_list/		split -l $PACKAGES_PER_PACKAGE tarball_list.txt tarball_split_list/
Line 29:		Line 33:
	done		done
	tar -czf $(basename $f).$PACKAGE_TYPE_SHORT.tar.gz *.$PACKAGE_TYPE		tar -czf $(basename $f).$PACKAGE_TYPE_SHORT.tar.gz *.$PACKAGE_TYPE
	mv $(basename $f).$PACKAGE_TYPE_SHORT.tar.gz ~~../output~~		mv $(basename $f).$PACKAGE_TYPE_SHORT.tar.gz $TARBALL_REPACK_DEST
	rm *.$PACKAGE_TYPE		rm *.$PACKAGE_TYPE
	echo $(basename $f)		echo $(basename $f)

Btingle at 21:33, 2 March 2023

2023-03-02T21:33:52Z

← Older revision		Revision as of 21:33, 2 March 2023
Line 57:		Line 57:
	For docking from ligands built using our pipeline with default options, running this script unmodified is sufficient for creating appropriately sized packages for docking. You may wish to edit WORKING_DIRECTORY if your files total more than 50G in size- /tmp is not usually any larger than this!		For docking from ligands built using our pipeline with default options, running this script unmodified is sufficient for creating appropriately sized packages for docking. You may wish to edit WORKING_DIRECTORY if your files total more than 50G in size- /tmp is not usually any larger than this!

	[[Category:~~DOCK3~~.8]]		[[Category:DOCK_3.8]]

Btingle at 21:33, 2 March 2023

2023-03-02T21:33:39Z

← Older revision		Revision as of 21:33, 2 March 2023
Line 56:		Line 56:

	For docking from ligands built using our pipeline with default options, running this script unmodified is sufficient for creating appropriately sized packages for docking. You may wish to edit WORKING_DIRECTORY if your files total more than 50G in size- /tmp is not usually any larger than this!		For docking from ligands built using our pipeline with default options, running this script unmodified is sufficient for creating appropriately sized packages for docking. You may wish to edit WORKING_DIRECTORY if your files total more than 50G in size- /tmp is not usually any larger than this!

			[[Category:DOCK3.8]]

Btingle at 20:31, 2 March 2023

2023-03-02T20:31:43Z

← Older revision		Revision as of 20:31, 2 March 2023
Line 54:		Line 54:
	aj		aj
	Done! Results in /tmp/user/output</nowiki>		Done! Results in /tmp/user/output</nowiki>

			For docking from ligands built using our pipeline with default options, running this script unmodified is sufficient for creating appropriately sized packages for docking. You may wish to edit WORKING_DIRECTORY if your files total more than 50G in size- /tmp is not usually any larger than this!

Btingle: Created page with "The following is a script for repackaging 3D pipeline results. First, here is the script: #!/bin/bash # make_tarballs.bash # required parameter RESULT_DIRECTORY=$1 # optional parameters WORKING_DIRECTORY=${WORKING_DIRECTORY-/tmp/$(whoami)} PACKAGES_PER_PACKAGE=${PACKAGES_PER_PACKAGE-100} PACKAGE_TYPE=${PACKAGE_TYPE-db2.gz} PACKAGE_TYPE_SHORT=$(echo $PACKAGE_TYPE | cut -d'.' -f1) echo WORKING_DIRECTORY=$WORKING_DIRECTORY mkdir -p $WORKING_DIRECTORY && cd $WO..."

2023-03-02T20:29:01Z

Created page with "The following is a script for repackaging 3D pipeline results. First, here is the script: <nowiki> #!/bin/bash # make_tarballs.bash # required parameter RESULT_DIRECTORY=$1 # optional parameters WORKING_DIRECTORY=${WORKING_DIRECTORY-/tmp/$(whoami)} PACKAGES_PER_PACKAGE=${PACKAGES_PER_PACKAGE-100} PACKAGE_TYPE=${PACKAGE_TYPE-db2.gz} PACKAGE_TYPE_SHORT=$(echo $PACKAGE_TYPE | cut -d'.' -f1) echo WORKING_DIRECTORY=$WORKING_DIRECTORY mkdir -p $WORKING_DIRECTORY && cd $WO..."

New page

The following is a script for repackaging 3D pipeline results. First, here is the script:

<nowiki>
#!/bin/bash
# make_tarballs.bash

# required parameter
RESULT_DIRECTORY=$1

# optional parameters
WORKING_DIRECTORY=${WORKING_DIRECTORY-/tmp/$(whoami)}
PACKAGES_PER_PACKAGE=${PACKAGES_PER_PACKAGE-100}
PACKAGE_TYPE=${PACKAGE_TYPE-db2.gz}
PACKAGE_TYPE_SHORT=$(echo $PACKAGE_TYPE | cut -d'.' -f1)

echo WORKING_DIRECTORY=$WORKING_DIRECTORY
mkdir -p $WORKING_DIRECTORY && cd $WORKING_DIRECTORY
mkdir -p output working tarball_split_list

echo finding
find $RESULT_DIRECTORY -name '*.tar.gz' > tarball_list.txt
echo splitting
split -l $PACKAGES_PER_PACKAGE tarball_list.txt tarball_split_list/
echo working
cd working
for f in ../tarball_split_list/*; do
for tb in $(cat $f); do
tar --transform='s/^.*\///' -xf $tb '*.'$PACKAGE_TYPE 2>/dev/null
done
tar -czf $(basename $f).$PACKAGE_TYPE_SHORT.tar.gz *.$PACKAGE_TYPE
mv $(basename $f).$PACKAGE_TYPE_SHORT.tar.gz ../output
rm *.$PACKAGE_TYPE
echo $(basename $f)
done
cd ..
echo Done! Results in $PWD/output</nowiki>

Now, an example usage:

<nowiki>
[user@gimel5 ~] bash make_tarballs.bash /nfs/exb/zinc22/tarballs/H17P200_H19P400.smi.batch-3d.d/out/aaa.d
finding
splitting
working
aa
ab
ac
ad
ae
af
ag
ah
ai
aj
Done! Results in /tmp/user/output</nowiki>