How to Create Cluster 7 (Ubuntu 20.04)

Introduction

Here are the instructions on how to build Cluster 7 using Ubuntu 20.04.

The build is not complete as of October 19, 2021.

Hardware Requirements (so far)

• 2 x 1 gigabit unmanaged switches

• one switch for private IP
• one switch for public IP

• 4 x unused ProLiant DL160 G5 Machines

• one for NAT machine
• one for Foreman Server
• one for LDAP Server
• one extra for any testing

• 1 x USB for Operating System (OS)

Create a Network Address Translation(NAT) machine

A NAT machine will act as a local VPN to the cluster. This will be the first machine you SSH into.

Current machine:

• Hostname: segundo.dot
• Private IP Address: 10.70.1.2
• Public IP Address: 169.230.26.58

Instructions

1. Install Ubuntu 20.04 using USB. You can set everything to default but make sure to disable network interfaces if DHCP Server is not set up yet.
2. After OS installation, grab MAC Address of network interface enp4s0. You'll need it to make a public IP.
   • ip a
3. Create a DNS entry for this machine in the website below to give you a public IP address
   • https://www.cgl.ucsf.edu/cgi-bin/secure/dns_dhcp_registration.py?action=view_zone&name=compbio.ucsf.edu
4. File a ticket to UCSF IT to open port 22 on the public interface
   • https://ucsf.service-now.com/ucsfit?id=ucsf_sc_cat_item&sys_id=10ea7459e0cef000e2dc0adb442e954c&sysparm_category=6654dac6877050009b5428676d434dec
5. Configure private and public network interfaces
   • vim /etc/netplan/00-installer-config.yaml
   • Configuration should look like this:

   # This is the network config written by 'subiquity'

   network:

   ethernets:

   enp3s0:

   addresses:

   - 10.70.1.2/16

   gateway4: 127.0.0.1

   nameservers:

   addresses:

   - 10.70.1.1

   search:

   - dot

   enp4s0:

   addresses:

   - 169.230.26.58/16

   gateway4: 169.230.26.1

   nameservers:

   addresses:

   - 8.8.8.8

   - 8.8.4.4

   - 128.218.254.10

   - 128.218.254.40

   search:

   - dot

   - compbio.ucsf.edu

   version: 2

6. Test if configuration works by pinging google
   • ping google.com
7. Enable and allow ssh in UFW firewall
   • ufw enable
   • ufw allow ssh
   • ufw status
8. Enable NAT in UFW
   • vim /etc/default/ufw
     • Change parameter of "DEFAULT_FORWARD_POLICY" to "ACCEPT"
   • vim /etc/ufw/sysctl.conf
     • Uncomment "net.ipv4.ip_forward=1"
   • vim /etc/ufw/before.rules
     • Add these lines at the beginning of the file:

   # NAT table rules

   *nat

   :POSTROUTING ACCEPT [0:0]

   -A POSTROUTING -s 169.230.26.58/16 -o enp4s0 -j MASQUERADE

   COMMIT

   • ufw reload
9. Test by setting another machine's gateway to 10.70.1.2 and then pinging google.com

Install Foreman

Foreman is an all in one cluster management tool with a graphical user interface.

Current machine:

• Hostname: primero.dot
• IP Address: 10.70.1.1
• Public IP Address: 169.230.26.69

Instructions

1. Install Ubuntu 20.04 using USB. You can set everything to default but make sure to disable network interfaces if DHCP Server is not set up yet.
2. After OS installation, grab MAC Address of network interface enp4s0. You'll need it to make a public IP.
   • ip a
3. Create a DNS entry for this machine in the website below to give you a public IP address
   • https://www.cgl.ucsf.edu/cgi-bin/secure/dns_dhcp_registration.py?action=view_zone&name=compbio.ucsf.edu
4. Configure private and public network interfaces
   • vim /etc/netplan/00-installer-config.yaml
   • Configuration should look like this:

   # This is the network config written by 'subiquity'

   network:

   ethernets:

   enp3s0:

   addresses:

   - 10.70.1.1/16

   gateway4: 10.70.1.2

   nameservers:

   addresses:

   - 10.70.1.1

   search:

   - dot

   enp4s0:

   addresses:

   - 169.230.26.69/16

   gateway4: 169.230.26.1

   nameservers:

   addresses:

   - 8.8.8.8

   - 8.8.4.4

   - 128.218.254.10

   - 128.218.254.40

   search:

   - dot

   - compbio.ucsf.edu

   version: 2

5. Test if configuration works by pinging google
   • ping google.com
6. Enable listed ports for Foreman and additional services
   • ufw allow 53/tcp
   • ufw allow 53/udp
   • ufw allow 67:69/udp
   • ufw allow 80/tcp
   • ufw allow 443/tcp
   • ufw allow 3000/tcp
   • ufw allow 5910:5930/tcp
   • ufw allow 5432/tcp
   • ufw allow 8140/tcp
   • ufw allow 8443/tcp
7. Prepare Puppet for Foreman (Foreman Guide)
   • sudo apt-get -y install ca-certificates
   • cd /tmp && wget https://apt.puppet.com/puppet6-release-focal.deb
   • sudo dpkg -i /tmp/puppet6-release-focal.deb
   • echo "deb http://deb.theforeman.org/ focal 3.0" | sudo tee /etc/apt/sources.list.d/foreman.list
   • echo "deb http://deb.theforeman.org/ plugins 3.0" | sudo tee -a /etc/apt/sources.list.d/foreman.list
   • sudo apt-get -y install ca-certificates
   • wget -q https://deb.theforeman.org/pubkey.gpg -O- | sudo apt-key add -
   • sudo apt-get update && sudo apt-get -y install foreman-installer
8. Add IPs and hostname to hosts file
   • vim /etc/hosts
     • 10.70.1.1 primero.dot primero
     • 169.230.26.69 primero.compbio.ucsf.edu
9. Generate “en_US.etf-8” for postgreSQL
   • locale-gen en_US.UTF-8
   • locale -a //to check for en_US.UTF-8
   • systemctl restart postgresql
10. Run foreman-installer

    foreman-installer \

    --foreman-proxy-tftp=true \

    --foreman-proxy-bmc=true \

    --foreman-proxy-dns=true \

    --foreman-proxy-dns-interface=enp3s0 \

    --foreman-proxy-dns-zone=dot \

    --foreman-proxy-dns-reverse=0.70.10.in-addr.arpa \

    --foreman-proxy-dns-forwarders=8.8.8.8 \

    --foreman-proxy-dns-forwarders=8.8.4.4 \

    --foreman-proxy-dns-forwarders=128.218.254.10 \

    --foreman-proxy-dns-forwarders=128.218.254.40 \

    --foreman-proxy-dhcp=true \

    --foreman-proxy-dhcp-interface=enp3s0 \

    --foreman-proxy-dhcp-gateway=10.70.1.1 \

    --foreman-proxy-dhcp-range="10.70.0.0 10.70.99.99" \

    --foreman-proxy-dhcp-nameservers="10.70.1.1"

    --foreman-proxy-foreman-base-url

    --enable-foreman-plugin-discovery

    --foreman-proxy-plugin-discovery-install-images

    --enable-foreman-compute-libvirt

    --enable-foreman-plugin-bootdisk

11. Here's a tutorial on Foreman
12. Here's the solutions for the labs