GPU DOCK - Revision history

Saadamin2006: /* The Memory Management System */

2023-04-22T20:50:40Z

The Memory Management System

← Older revision		Revision as of 20:50, 22 April 2023
Line 114:		Line 114:
	* <code>vdw.h/cpp</code> contains the CUDA/C++ definition of the Van der Waals map.		* <code>vdw.h/cpp</code> contains the CUDA/C++ definition of the Van der Waals map.

	===The Memory Management System===		===The GPU Memory Management System===

			GPU DOCK handles a lot of memory. It needs to upload many ligands and matches to the GPU, and download a scoring buffer that is in the hundreds of megabytes. Certainly, we need an efficient memory management system to make sure just moving around memory does not become a bottleneck. As mentioned earlier, each call to <code>cudaMemcpy</code> itself has significant overhead, regardless of the size of data we're transferring. Hence, we batch the uploads and downloads to keep performance high. To do this, we generally stuff as much data into a single buffer when uploading or downloading.

			However, putting everything into one buffer leads to another issue: that buffer needs to be very large, so how are we efficiently going to allocate that memory? The most straightforward approach to this is to keep the same large buffer from call to call. If our buffer proves too small, we free it and allocate the necessary size. This does lead to issues at the start of the program, where the buffer is rapidly growing, so we set the initial size of the buffer to a large value. This single-buffer approach, despite how "inelegant" it is, works quite well. When we take multiple data sets from <code>H17P050-N-la</code> and the <code>DUDE-Z DRD4</code> receptor and run GPU DOCK sequentially*, we find that across all GPU DOCK instances, we only have to reallocate when processing 3% of the batches (which would be even lower if we batched all data sets together into one data set and ran only one instance of GPU DOCK).

			Generally, the structure of this stuff-everything-into-one-buffer approach is as follows
			# For each ligand and its matches
			# # Sum how much GPU memory we would need to upload the ligand and matches
			# # Add this sum to the "total memory required" sum
			# If the current buffer is smaller than the total memory required, resize the buffer
			# Copy all ligands and matches to a temporary buffer
			# Call <code>cudaMemcpy</code> to upload this buffer

Saadamin2006: /* How GPU DOCK Works */

2023-04-22T20:25:43Z

How GPU DOCK Works

← Older revision		Revision as of 20:25, 22 April 2023
Line 113:		Line 113:
	* <code>upload.h/cu</code> writes the ligand and match data to a temporary buffer before uploading it to the GPU.		* <code>upload.h/cu</code> writes the ligand and match data to a temporary buffer before uploading it to the GPU.
	* <code>vdw.h/cpp</code> contains the CUDA/C++ definition of the Van der Waals map.		* <code>vdw.h/cpp</code> contains the CUDA/C++ definition of the Van der Waals map.

			===The Memory Management System===

Saadamin2006: /* Code Structure */

2023-04-22T20:16:04Z

Code Structure

← Older revision		Revision as of 20:16, 22 April 2023
Line 89:		Line 89:

	When Fortran nears the end of the reading/matching loop, it transfers the ligands and matches to C++. Since a single temporary transfer buffer is used for arrays, this requires multiple function calls. First, metadata (e.g. sizes of various arrays, or anything else that isn't a large resizable array, including 2D and 3D vectors) is packed into the function arguments of one call. Then, for each large array in the object, it copies it to the temporary transfer buffer and calls a C++ function that copies the temporary array into a C++ side array. Before the start of the program, various things like grids are sent to C++ using the same process. Generally, all data structures that will be sent to C++ have a function that abstracts their transfer into one function call. For instance, <code>db2</code> has the helper function <code>send_db2lig</code> in <code>db2type.f</code> to abstract the transferring process away from the main search loop.		When Fortran nears the end of the reading/matching loop, it transfers the ligands and matches to C++. Since a single temporary transfer buffer is used for arrays, this requires multiple function calls. First, metadata (e.g. sizes of various arrays, or anything else that isn't a large resizable array, including 2D and 3D vectors) is packed into the function arguments of one call. Then, for each large array in the object, it copies it to the temporary transfer buffer and calls a C++ function that copies the temporary array into a C++ side array. Before the start of the program, various things like grids are sent to C++ using the same process. Generally, all data structures that will be sent to C++ have a function that abstracts their transfer into one function call. For instance, <code>db2</code> has the helper function <code>send_db2lig</code> in <code>db2type.f</code> to abstract the transferring process away from the main search loop.

			===Index of Files===
			Here is an index of important GPU DOCK files and what they do:
			* <code>common.h/cpp</code> contains commonly included headers and basic debugging utilities.
			* <code>cpu_score.h/cpp</code> contains a CPU-side implementation of scoring and the prefiltering code.
			* <code>db2type.h/cpp</code> contains a C++/CUDA-side implementation of the <code>db2</code> data structure.
			* <code>dev_alloc.hpp</code> is a GPU memory pool.
			* <code>dev_vector.hpp</code> is an implementation of <code>std::vector</code> on CUDA. It does not support resizing.
			* <code>gpu_common.h/cpp</code> contains important includes and <code>struct</code>s for GPU-side scoring that are used throughout the code.
			* <code>gpu_score.h/cu</code> contains the GPU-side scoring code and performance collection.
			* <code>hdd_interface.hpp/cu</code> (short for host-device data interface) contains utility functions that help in the upload process.
			* <code>initialize.cpp</code> contains important start-up and clean-up code.
			* <code>interface.h/cpp</code> contains variables, functions, and macros that are important for Fortran/C++ interoperability, data transfer, and debugging.
			* <code>matchtype.h/cpp</code> contains a CUDA/C++ side definition of the <code>matcht</code> data structure.
			* <code>options.h/cpp</code> contains a CUDA/C++ side definition of the <code>options</code> data structure
			* <code>phimap.h/cpp</code> contains a CUDA/C++ side data structure for the phimap (the electrostatics grid).
			* <code>scoring_common.h/cpp</code> contains <code>struct</code>s important to both CPU-side and GPU-side scoring.
			* <code>shared_mem.hpp</code> contains a memory pool-like data structure for managing the memory for the master score (since the master score is deprecated, this too is deprecated).
			* <code>solvmap.h/cpp</code> contains the CUDA/C++ side implementation of the solvation and asolvation grids.
			* <code>status.h/cpp</code> contains a copy of Fortran's status codes.
			* <code>timer.hpp</code> contains a utility timer class that runs on the wall clock (not the CPU clock).
			* <code>transfer.h/cpp</code> contains helper functions for receiving integer and float data from Fortran.
			* <code>upload.h/cu</code> writes the ligand and match data to a temporary buffer before uploading it to the GPU.
			* <code>vdw.h/cpp</code> contains the CUDA/C++ definition of the Van der Waals map.

Saadamin2006: /* GPU DOCK Structure */

2023-04-22T19:50:53Z

GPU DOCK Structure

← Older revision		Revision as of 19:50, 22 April 2023
Line 45:		Line 45:
	Due to the communication overhead, data transferring follows a curve approximated by <math>T=log_c(a+b^x)</math>. More information can be found in this [https://www.cs.virginia.edu/~skadron/Papers/cuda_tuning_bof_sc09_final.pdf Nvidia presentation].		Due to the communication overhead, data transferring follows a curve approximated by <math>T=log_c(a+b^x)</math>. More information can be found in this [https://www.cs.virginia.edu/~skadron/Papers/cuda_tuning_bof_sc09_final.pdf Nvidia presentation].

	==GPU ~~DOCK Structure~~==		==Partially GPU-ified Docking Pipeline==
	CPU DOCK processes ligands sequentially which GPU DOCK process them in batches of size <code>n</code>, typically 48. For each batch, the docking process looks like this		CPU DOCK processes ligands sequentially which GPU DOCK process them in batches of size <code>n</code>, typically 48. For each batch, the docking process looks like this
	# Read <code>n</code> ligands from the disk		# Read <code>n</code> ligands from the disk
Line 63:		Line 63:

	===No Caching===		===No Caching===
	Another optimization CPU DOCK uses is that caches the results of confs in the set score computation. The issue with caching is that a lot of memory would have be written to the cache, and this is slow on the GPU. On the other hand, reading and arithmetic are strong points of GPUs, so it makes sense to recompute these scores.		Another optimization CPU DOCK uses is that caches the results of confs in the set score computation. The issue with caching is that a lot of memory would have be written to the cache, and this is slow on the GPU. On the other hand, reading and arithmetic are strong points of GPUs, so it makes sense to recompute these scores.

	==Code Structure==		==Code Structure==

Saadamin2006: /* How GPU DOCK Works */

2023-04-22T19:49:59Z

How GPU DOCK Works

← Older revision		Revision as of 19:49, 22 April 2023
Line 23:		Line 23:

	=How GPU DOCK Works=		=How GPU DOCK Works=
	Here, I discuss how GPU DOCK works. I'll cover how the code is structured, and the algorithmic designs.		GPU DOCK is written in CUDA. The code is structured such that the GPU components are as independent from the original Fortran program. Here, I discuss how GPU DOCK works. I'll cover how the code is structured, and the algorithmic designs.
	==Terminology and Background==		==Terminology and Background==
	Before I get into the technical aspects of GPU DOCK, here is some brief terminology I will use in this page:		Before I get into the technical aspects of GPU DOCK, here is some brief terminology I will use in this page:

Saadamin2006: /* C++ Side Data Structures */

2023-04-22T19:48:12Z

C++ Side Data Structures

← Older revision		Revision as of 19:48, 22 April 2023
Line 85:		Line 85:
	* The Van der Waals, electrostatics, and solvation grids, which can be found respectively in <code>vdw.h/cpp</code>, <code>phimap.h/cpp</code>, <code>solvmap.h/cpp</code> respectively		* The Van der Waals, electrostatics, and solvation grids, which can be found respectively in <code>vdw.h/cpp</code>, <code>phimap.h/cpp</code>, <code>solvmap.h/cpp</code> respectively
	The C++ side data structures also need to be replicated on the GPU as well as on the CPU in the C++ side code. These data structures are located in the same files as their CPU counterparts, and have <code>dev_</code> appended to them to indicate they are on the device (CUDA's terminology for GPU). For instance, <code>db2t</code>'s GPU counterpart is named <code>dev_db2t</code>.		The C++ side data structures also need to be replicated on the GPU as well as on the CPU in the C++ side code. These data structures are located in the same files as their CPU counterparts, and have <code>dev_</code> appended to them to indicate they are on the device (CUDA's terminology for GPU). For instance, <code>db2t</code>'s GPU counterpart is named <code>dev_db2t</code>.

			An important thing to note is the difference between the 2D, 3D, and 4D vector types of GPU-side and CPU-side vector structures. The CPU code uses the <code>glm</code> library, as that is the most convenient and versatile library to use on the CPU for linear algebra. However, CUDA doesn't support native vectorization and other optimizations with <code>glm</code>'s types. Instead, CUDA has built-in vector data structures that should be used on the CUDA code instead. This means that when we are uploading data structures to the GPU, we cannot use the source arrays exactly. We have to use an intermediary vector that contains the converted data structures. This does result in an inefficiency and the CPU/GPU vector type mismatch will be removed in a future update.

			When Fortran nears the end of the reading/matching loop, it transfers the ligands and matches to C++. Since a single temporary transfer buffer is used for arrays, this requires multiple function calls. First, metadata (e.g. sizes of various arrays, or anything else that isn't a large resizable array, including 2D and 3D vectors) is packed into the function arguments of one call. Then, for each large array in the object, it copies it to the temporary transfer buffer and calls a C++ function that copies the temporary array into a C++ side array. Before the start of the program, various things like grids are sent to C++ using the same process. Generally, all data structures that will be sent to C++ have a function that abstracts their transfer into one function call. For instance, <code>db2</code> has the helper function <code>send_db2lig</code> in <code>db2type.f</code> to abstract the transferring process away from the main search loop.

Saadamin2006 at 00:37, 22 April 2023

2023-04-22T00:37:11Z

← Older revision		Revision as of 00:37, 22 April 2023
Line 1:		Line 1:
	GPU DOCK is a fork of DOCK 3.8 that utilizes GPU acceleration via CUDA. Currently (as of April 2023) GPU DOCK uses the GPU for parallel scoring and only supports Van der Waals, electrostatics, and solvation scores.		GPU DOCK is a fork of DOCK 3.8 that utilizes GPU acceleration via CUDA. Currently (as of April 2023) GPU DOCK uses the GPU for parallel scoring and only supports Van der Waals, electrostatics, and solvation scores.

			```THIS PAGE IS CURRENTLY WIP```

	=Running and Building GPU DOCK=		=Running and Building GPU DOCK=

Saadamin2006: /* C++ Side Data Structures */

2023-04-21T01:17:42Z

C++ Side Data Structures

← Older revision		Revision as of 01:17, 21 April 2023
Line 78:		Line 78:

	===C++ Side Data Structures===		===C++ Side Data Structures===
	Since the data structures need to replicated on the C++ side,		Since the data structures need to replicated on the C++ side, we need to define them on the C++ side. Here are the currently replicated data structures:
			* <code>matcht</code>, which can be found in <code>matchtype.h/cpp</code>
			* <code>db2</code>, which can be found in <code>db2type.h/cpp</code> (in the C++ version of the code it is called <code>db2t</code> instead of just <code>db2</code>
			* The Van der Waals, electrostatics, and solvation grids, which can be found respectively in <code>vdw.h/cpp</code>, <code>phimap.h/cpp</code>, <code>solvmap.h/cpp</code> respectively
			The C++ side data structures also need to be replicated on the GPU as well as on the CPU in the C++ side code. These data structures are located in the same files as their CPU counterparts, and have <code>dev_</code> appended to them to indicate they are on the device (CUDA's terminology for GPU). For instance, <code>db2t</code>'s GPU counterpart is named <code>dev_db2t</code>.

Saadamin2006: /* GPU DOCK Structure */

2023-04-21T01:08:59Z

GPU DOCK Structure

← Older revision		Revision as of 01:08, 21 April 2023
Line 58:		Line 58:

	===Prefiltering===		===Prefiltering===
			CPU DOCK looks at the rigid conf to determine whether it should bump a ligand. This optimization means that it looks at only one conf to determine whether the pose is already bad enough that it should avoid scoring it. Letting the GPU do this is inefficient, as the GPU would have the launch threads that die immediately if bumping is enabled or do useless work if bumping is disabled. Furthermore, the CPU adjusts the size of the matching buffer according to expected number of matches, meaning that a lot of GPU memory would have to be wasted for bumped ligands. Instead, the CPU checks ahead of time what ligands will be bumped. As a result, it only sends ligands whose rigid conf was not bumped to the GPU to do scoring.

	CPU DOCK ~~looks at~~ the ~~rigid conf to determine whether it should bump a ligand~~. ~~This optimization means~~ that ~~it looks at only one conf~~ to ~~determine whether~~ the ~~pose~~ is ~~already bad enough that it should avoid scoring~~ it. ~~Letting~~ the GPU ~~do this~~ is ~~inefficient~~, as the ~~GPU would~~ have the ~~launch threads that die immediately if bumping~~ is ~~enabled or~~ do ~~useless~~ work ~~if bumping~~ is ~~disabled~~. ~~Furthermore~~, the ~~CPU adjusts~~ the ~~size~~ of the ~~matching buffer according to expected number of matches~~, ~~meaning that a lot of~~ GPU memory ~~would have to be wasted for bumped ligands~~.		===No Caching===
			Another optimization CPU DOCK uses is that caches the results of confs in the set score computation. The issue with caching is that a lot of memory would have be written to the cache, and this is slow on the GPU. On the other hand, reading and arithmetic are strong points of GPUs, so it makes sense to recompute these scores.

			==Code Structure==
			All the original Fortran code are in <code>src</code> and code <code>src/libfgz</code>. The GPU code is in <code>src/gpu</code>, all of which is written in C++ and CUDA. There also are a few new Fortran in <code>src</code> that is mixed with the old Fortran files (some of which have been modified).

			===The Fortran/C++ Interop===
			To allow the GPU parts to be written in C++, we need to write an interface for Fortran and C++ to interact with each other and send data. The easiest way to do this is to call C++ code via functions: we can hand off execution momentarily for C++ so it can do work. The interface provided in the Fortran standard for a Fortran/C interop is very restrictive. For this reason, the C++ code is designed to be largely self-contained and shows a very high level interface to Fortran. For example, Fortran calls <code>score_all_ligands_gpu</code> to score the ligands on the C++/CUDA side (instead of calling C++ functions that are much more closely involved in the scoring process, such as GPU memory allocation). Furthermore, no part of C++ calls a Fortran function, but the Fortran code calls the C++ functions.

	~~Instead, the CPU checks ahead of time what ligands will be bumped~~. ~~As a result, it only sends ligands whose rigid conf was no bumped~~ to the ~~GPU~~ to ~~do scoring~~. ~~Since memory accesses are expensive~~ on the ~~GPU~~, it ~~is more efficient~~ to ~~recompute~~ the ~~conf score~~ and ~~not reuse~~ the ~~CPU result~~.		Fortran by default passes everything by a pointer reference. It also appends an underscore (<code>_</code>) to all function names. Suppose the Fortran compiler saw the code:
			<nowiki>call myFunc(myIntegerValue)</nowiki>
			We would have to have the function signature <code>void myFunc_(int* myIntegerValue)</code> defined on the C side. For C++, on the other hand, the C++ compiler appends garbage to the symbol name. For example, the following function:
			<nowiki>void fooBar(int* xyz)</nowiki>
			Will be compiled into the symbol <code>__fooBar@INSERT_GARBAGE_CHARACTERS</code> (or some other variation) and thus the linker will not be able to find it. To get around this issue, we have to append <code>extern "C"</code> to compile without the name mangling. For brevity and clarity, I define the macro <code>FORTRAN_EXPORT</code> to mean <code>extern "C"</code>.

	~~===No Caching===~~		As said before, the interface is restrictive. For instance, it is near impossible to pass a struct or array legally and safely into a function. It is also very difficult to make Fortran and C++ share the same memory resources. For this reason, all data (ligands, matches, grids, scores, etc) unfortunately needs to be replicated on the C++ side. A lot of DOCK's data structures contain large arrays (such as ligand atom and conf data). Transferring these directly to C++ is a problem, since we cannot put a struct's array directly into a C++ function for reasons. As a result, I copy these large arrays to a secondary array that both Fortran and C++ have access to. Fortran writes to this array, and C++ (via a function call from Fortran) creates a copy of it on its side (so that the array can be reused for further transfers). This array is located in <code>transfer.f</code> and each of the major data structure files (e.g. <code>db2type.f</code> and <code>matchtype.f</code>) have functions that utilize this array and the other utility functions defined in <code>transfer.f</code>.

	~~Another optimization CPU DOCK uses is that caches~~ the ~~results of confs in the set score computation. As previously mentioned, reading memory is expensive~~ on the ~~GPU~~, ~~so instead GPU DOCK recalculates set scores.~~		===C++ Side Data Structures===
			Since the data structures need to replicated on the C++ side,

Saadamin2006: /* GPU DOCK Structure */

2023-04-21T00:33:55Z

GPU DOCK Structure

← Older revision		Revision as of 00:33, 21 April 2023
Line 45:		Line 45:
	==GPU DOCK Structure==		==GPU DOCK Structure==
	CPU DOCK processes ligands sequentially which GPU DOCK process them in batches of size <code>n</code>, typically 48. For each batch, the docking process looks like this		CPU DOCK processes ligands sequentially which GPU DOCK process them in batches of size <code>n</code>, typically 48. For each batch, the docking process looks like this
	# 1. Read <code>n</code> ligands from the disk		# Read <code>n</code> ligands from the disk
	# 2. Match <code>n</code> ligands sequentially		# Match <code>n</code> ligands sequentially
	# 3. Upload matching results and ligands to GPU memory and score		# Upload matching results and ligands to GPU memory and score
	# 4. Download scoring results		# Download scoring results
	# 5. Determine the <code>nsave</code> best poses and write them to the disk		# Determine the <code>nsave</code> best poses and write them to the disk

	Any parts of the scoring code (steps 3 and 4) are written in C++ and CUDA. The rational behind batching the ligands instead of processing them on the GPU one-by-one is due to the aforementioned communication bottleneck between the CPU and GPU. If we were communicating with the GPU per each ligand to upload, score, and download, we would be wasting a lot of time per each communication operation. However, we can distribute the communication overhead across multiple ligands and save performance this way.		Any parts of the scoring code (steps 3 and 4) are written in C++ and CUDA. The rational behind batching the ligands instead of processing them on the GPU one-by-one is due to the aforementioned communication bottleneck between the CPU and GPU. If we were communicating with the GPU per each ligand to upload, score, and download, we would be wasting a lot of time per each communication operation. However, we can distribute the communication overhead across multiple ligands and save performance this way.