Cluster Theory

You require a cluster to run molecular docking at scale.

Non-Scalable Cluster

You can just take a bunch of computers, new or used, and an ethernet switch and put them together to make a cluster. That's what we used to do. As the cluster scales, this approach becomes unwieldy and certainly labor intensive to maintain. If you plan to scale and you can afford it, we recommend starting with a scalable design, below.

Minimalist Scalable Cluster - One computer - $20K

You can begin a scalable cluster with as few as one computer, albeit a biggish one. We recommend using a hypervisor so you can run multiple virtual machines on a single computer. You can get away without a rack, just put the machine in a sound-proof room. We would buy:

• a) 1 x 128 core / 256 GB / 16 TB disk @ $20K
• b) a workstation, but you could even use a laptop

Plug it in to the network and the power supply. That's it. That's all you need to get started.

Starter Scalable Cluster - $40K

If you can afford it, starting with a few more computers will be easier to scale up. This cluster would be an excellent resource and a good foundation for managed growth for a new group getting started. We would buy:

• a) 1 x 96 TB NAS server, $18K
• b) 1 x 128 core / 512 GB @ $20K
• c) switch, cables, economy workstation: $2K

You do not need a rack just yet, but you need a sound-proof room with normal room air conditioning. If you add any more computers to this configuration, you'll need a rack.

• 2 x 15A circuits.

Small Scaleable Cluster - $92K

If you had around $100K to spend, we would buy:

• a) 1 x 96 TB NAS server, $18K
• b) 3 x 128 core / 512 GB @ $20K
• c) barebones rack, switch, cables: $1K
• d) 4 workstations @ $2K = $8K
• e) spare parts (disks, switch) $4K
• f) ssh portal machine / firewall $1K

Sound proof room, 1 T of cooling = 12,000 Btu/h = 3.516 KW

• 30 A @ 208V (e.g. 1 x L6-30)

This is a natural progression from the starter cluster above, and can be grown towards the larger cluster below.

Bigger Scaleable Cluster - $250K

If you had around $250K to spend, we would buy:

• a) 2 x 96 TB NAS server @ $18K = $36K
• b) 10 x 128 core / 512 GB @ $20K = $200K
• c) barebones rack, switch, cables: $1K
• d) 4 workstations @ $2K = $8K
• e) spare parts (disks, switch) $4K
• f) ssh portal machine / firewall $1K

Sound proof room, 5 T of cooling

• 90 A @ 208V (e.g. 3 L6-30s)

Currently...

We are currently buying CPU from Silicon Mechanics and Dell and disk from Silicon Mechanics and HP. We are currently buying enclosures holding 12 SAS disks of 4TB each for 48 TB raw for around $8000 or about 6 raw GB per dollar. Formatted RAID6 this works out to 36TB or 4.5 formatted GB per dollar. We like the HP P822 high performance RAID controller. Compare this to what we were paying just a year ago in spring 2013: 25 TB for $10,000 or 2.5 TB per dollar unformatted. And it is SAS instead of SATA so we only need RAID6 and not RAID10 formatting. An amazing development in the last 12 months.

For CPU, we like the C6145 from Dell. For around $20,000 you get 2 machines in a 2U form each with 64 cores and 256 GB memory and a pair of RAID1 formatted disks each. A single 42U rack could hold 2560 cores and still have room for a switch. Of course, this would cost you $400,000. Running flat out it would pull 28 kW and need 10 T of cooling to be safe. Amazing density at commodity prices.

Back to So you want to set up a lab.