Cluster Theory - Revision history

Frodo: /* Starter Scalable Cluster - $40K */

2014-03-18T21:09:07Z

Starter Scalable Cluster - $40K

← Older revision		Revision as of 21:09, 18 March 2014
Line 19:		Line 19:
	*b) 1 x 128 core / 512 GB @ $20K		*b) 1 x 128 core / 512 GB @ $20K
	*c) switch, cables, economy workstation: $2K		*c) switch, cables, economy workstation: $2K
	You do not need a rack, but you need a sound-proof room with normal room air conditioning.		You do not need a rack just yet, but you need a sound-proof room with normal room air conditioning. If you add any more computers to this configuration, you'll need a rack.
	* 2 x 15A circuits.		* 2 x 15A circuits.

2014-03-18T16:20:42Z

← Older revision		Revision as of 16:20, 18 March 2014
Line 45:		Line 45:
	* 90 A @ 208V (e.g. 3 L6-30s)		* 90 A @ 208V (e.g. 3 L6-30s)

			= Currently... =
			We are currently buying CPU from Silicon Mechanics and Dell and disk from Silicon Mechanics and HP.
			We are currently buying enclosures holding 12 SAS disks of 4TB each for 48 TB raw for around $8000 or about 6 raw GB per dollar. Formatted RAID6 this works out to 36TB or 4.5 formatted GB per dollar. We like the HP P822 high performance RAID controller. Compare this to what we were paying just a year ago in spring 2013: 25 TB for $10,000 or 2.5 TB per dollar unformatted. And it is SAS instead of SATA so we only need RAID6 and not RAID10 formatting. An amazing development in the last 12 months.

			For CPU, we like the C6145 from Dell. For around $20,000 you get 2 machines in a 2U form each with 64 cores and 256 GB memory and a pair of RAID1 formatted disks each. A single 42U rack could hold 2560 cores and still have room for a switch. Of course, this would cost you $400,000. Running flat out it would pull 28 kW and need 10 T of cooling to be safe. Amazing density at commodity prices.

	Back to [[So you want to set up a lab]].		Back to [[So you want to set up a lab]].

2014-03-18T16:16:35Z

Small Scaleable Cluster - $92K

← Older revision		Revision as of 16:16, 18 March 2014
Line 32:		Line 32:
	Sound proof room, 1 T of cooling = 12,000 Btu/h = 3.516 KW		Sound proof room, 1 T of cooling = 12,000 Btu/h = 3.516 KW
	* 30 A @ 208V (e.g. 1 x L6-30)		* 30 A @ 208V (e.g. 1 x L6-30)
			This is a natural progression from the starter cluster above, and can be grown towards the larger cluster below.

	= Bigger Scaleable Cluster - $250K =		= Bigger Scaleable Cluster - $250K =

2014-03-18T16:15:07Z

Starter Scalable Cluster - $40K

← Older revision		Revision as of 16:15, 18 March 2014
Line 15:		Line 15:

	= Starter Scalable Cluster - $40K =		= Starter Scalable Cluster - $40K =
	If you can afford it, starting with a few more computers will be easier to scale up. This cluster would be an excellent resource and a good foundation for managed growth for a new group getting started.		If you can afford it, starting with a few more computers will be easier to scale up. This cluster would be an excellent resource and a good foundation for managed growth for a new group getting started. We would buy:

	We would buy:
	*a) 1 x 96 TB NAS server, $18K		*a) 1 x 96 TB NAS server, $18K
	*b) 1 x 128 core / 512 GB @ $20K		*b) 1 x 128 core / 512 GB @ $20K
	*c) switch, cables, economy workstation: $2K		*c) switch, cables, economy workstation: $2K
	You do not need a rack, but you need a sound proof room with normal room air conditioning.		You do not need a rack, but you need a sound-proof room with normal room air conditioning.
	* 2 x 15A circuits.		* 2 x 15A circuits.

2014-03-18T16:14:51Z

Minimalist Scalable Cluster - One computer - $20K

← Older revision		Revision as of 16:14, 18 March 2014
Line 9:		Line 9:
	You can begin a scalable cluster with as few as one computer, albeit a biggish one.		You can begin a scalable cluster with as few as one computer, albeit a biggish one.
	We recommend using a hypervisor so you can run multiple virtual machines on a single computer.		We recommend using a hypervisor so you can run multiple virtual machines on a single computer.
	You can get away without a rack, just put the machine in a sound-proof room.		You can get away without a rack, just put the machine in a sound-proof room. We would buy:

	We would buy:
	* a) 1 x 128 core / 256 GB / 16 TB disk @ $20K		* a) 1 x 128 core / 256 GB / 16 TB disk @ $20K
	* b) a workstation, but you could even use a laptop		* b) a workstation, but you could even use a laptop

2014-03-18T16:14:35Z

Starter Scalable Cluster - $40K

← Older revision		Revision as of 16:14, 18 March 2014
Line 17:		Line 17:

	= Starter Scalable Cluster - $40K =		= Starter Scalable Cluster - $40K =
	If you can afford it, starting with a few more computers will be easier to scale up.		If you can afford it, starting with a few more computers will be easier to scale up. This cluster would be an excellent resource and a good foundation for managed growth for a new group getting started.

	We would buy:		We would buy:

2014-03-18T16:12:49Z

Minimalist Scalable Cluster - One computer - $20K

← Older revision		Revision as of 16:12, 18 March 2014
Line 13:		Line 13:
	We would buy:		We would buy:
	* a) 1 x 128 core / 256 GB / 16 TB disk @ $20K		* a) 1 x 128 core / 256 GB / 16 TB disk @ $20K
			* b) a workstation, but you could even use a laptop
			Plug it in to the network and the power supply. That's it. That's all you need to get started.

	= Starter Scalable Cluster - $40K =		= Starter Scalable Cluster - $40K =

2014-03-18T16:11:28Z

Non-Scalable Cluster

← Older revision		Revision as of 16:11, 18 March 2014
Line 2:		Line 2:

	= Non-Scalable Cluster =		= Non-Scalable Cluster =
	You can just take a bunch of computers, new or used, and put them together to make a cluster. That's ~~mostly~~ what we~~'ve done for~~ the ~~last 15 years. It works ok~~, ~~but as you scale it~~ becomes unwieldy. If you can afford it, we recommend starting with a scalable design, below.		You can just take a bunch of computers, new or used, and an ethernet switch and put them together to make a cluster. That's what we used to do. As the cluster scales, this approach becomes unwieldy and certainly labor intensive to maintain. If you plan to scale and you can afford it, we recommend starting with a scalable design, below.

	{{TOCright}}		{{TOCright}}

2014-03-18T16:04:33Z

← Older revision		Revision as of 16:04, 18 March 2014
Line 3:		Line 3:
	= Non-Scalable Cluster =		= Non-Scalable Cluster =
	You can just take a bunch of computers, new or used, and put them together to make a cluster. That's mostly what we've done for the last 15 years. It works ok, but as you scale it becomes unwieldy. If you can afford it, we recommend starting with a scalable design, below.		You can just take a bunch of computers, new or used, and put them together to make a cluster. That's mostly what we've done for the last 15 years. It works ok, but as you scale it becomes unwieldy. If you can afford it, we recommend starting with a scalable design, below.

			{{TOCright}}

	= Minimalist Scalable Cluster - One computer - $20K =		= Minimalist Scalable Cluster - One computer - $20K =

2014-03-14T00:38:46Z

Bigger Scaleable Cluster - $250K

← Older revision		Revision as of 00:38, 14 March 2014
Line 41:		Line 41:
	*e) spare parts (disks, switch) $4K		*e) spare parts (disks, switch) $4K
	*f) ssh portal machine / firewall $1K		*f) ssh portal machine / firewall $1K
	* Sound proof room, 5 T of cooling		Sound proof room, 5 T of cooling
	* 90 A @ 208V (e.g. 3 L6-30s)		* 90 A @ 208V (e.g. 3 L6-30s)