Difference between revisions of "AWS Auto Scaling"

From DISI
Jump to: navigation, search
m
m
Line 26: Line 26:
 
   pcluster configure
 
   pcluster configure
  
Allowed values for AWS Region ID:
+
  Allowed values for AWS Region ID:
1. ap-northeast-1
+
  1. ap-northeast-1
2. ap-northeast-2
+
  2. ap-northeast-2
3. ap-south-1
+
  3. ap-south-1
4. ap-southeast-1
+
  4. ap-southeast-1
5. ap-southeast-2
+
  5. ap-southeast-2
6. ca-central-1
+
  6. ca-central-1
7. eu-central-1
+
  7. eu-central-1
8. eu-north-1
+
  8. eu-north-1
9. eu-west-1
+
  9. eu-west-1
10. eu-west-2
+
  10. eu-west-2
11. eu-west-3
+
  11. eu-west-3
12. sa-east-1
+
  12. sa-east-1
13. us-east-1
+
  13. us-east-1
14. us-east-2
+
  14. us-east-2
15. us-west-1
+
  15. us-west-1
16. us-west-2
+
  16. us-west-2
AWS Region ID [us-east-1]:
+
  AWS Region ID [us-east-1]:
  
 
Network & Security -> Key Pairs -> Create New Pair
 
Network & Security -> Key Pairs -> Create New Pair
Allowed values for EC2 Key Pair Name:
+
 
1. EC2_v1
+
  Allowed values for EC2 Key Pair Name:
EC2 Key Pair Name [EC2_v1]:
+
  1. EC2_v1
Allowed values for Scheduler:
+
  EC2 Key Pair Name [EC2_v1]:
1. sge
+
  Allowed values for Scheduler:
2. torque
+
  1. sge
3. slurm
+
  2. torque
4. awsbatch
+
  3. slurm
Scheduler [slurm]:
+
  4. awsbatch
Allowed values for Scheduler:
+
  Scheduler [slurm]:
1. sge
+
  Allowed values for Scheduler:
2. torque
+
  1. sge
3. slurm
+
  2. torque
4. awsbatch
+
  3. slurm
Scheduler [slurm]:
+
  4. awsbatch
Minimum cluster size (instances) [0]:  
+
  Scheduler [slurm]:
Maximum cluster size (instances) [10]:  
+
  Minimum cluster size (instances) [0]:  
Master instance type [t2.micro]:  
+
  Maximum cluster size (instances) [10]:  
Compute instance type [t2.micro]:
+
  Master instance type [t2.micro]:  
Automate VPC creation? (y/n) [n]:  
+
  Compute instance type [t2.micro]:
Allowed values for VPC ID:
+
  Automate VPC creation? (y/n) [n]:  
1. vpc-579d8e2d | 0 subnets inside
+
  Allowed values for VPC ID:
VPC ID [vpc-579d8e2d]:  
+
  1. vpc-579d8e2d | 0 subnets inside
Allowed values for Network Configuration:
+
  VPC ID [vpc-579d8e2d]:  
1. Master in a public subnet and compute fleet in a private subnet
+
  Allowed values for Network Configuration:
2. Master and compute fleet in the same public subnet
+
  1. Master in a public subnet and compute fleet in a private subnet
Network Configuration [Master in a public subnet and compute fleet in a private subnet]: 1
+
  2. Master and compute fleet in the same public subnet
 +
  Network Configuration [Master in a public subnet and compute fleet in a private subnet]: 1
  
  

Revision as of 06:46, 2 June 2020

A step-by-step instruction how to create a slurm cluster on AWS with auto-scaling possibility


a) upgrade your pip: pip install --upgrade pip

b) install aws client: pip install awscli

c) pip install aws-parallelcluster


d) prepare your aws_access_key_id and aws_secret_access_key.
Those can be found in "My Security Credentials -> Access keys (access key ID and secret access key)" section.
If you haven't got it yet, press "Create New Access Key" and follow the instructions.

 aws configure
 Access Key ID [None]: _YOUR_ACCESS_KEY_ID_
 AWS Secret Access Key [None]: _YOUR_SECRET_ACCESS_KEY_
 this will be stored in .aws/credentials

e) parallel cluster configuration NB: We use AWS Region us-east-1, which corresponds to N.Virginia.
You are welcome to re-consider this choice.

 pcluster configure
 Allowed values for AWS Region ID:
 1. ap-northeast-1
 2. ap-northeast-2
 3. ap-south-1
 4. ap-southeast-1
 5. ap-southeast-2
 6. ca-central-1
 7. eu-central-1
 8. eu-north-1
 9. eu-west-1
 10. eu-west-2
 11. eu-west-3
 12. sa-east-1
 13. us-east-1
 14. us-east-2
 15. us-west-1
 16. us-west-2
 AWS Region ID [us-east-1]:

Network & Security -> Key Pairs -> Create New Pair

 Allowed values for EC2 Key Pair Name:
 1. EC2_v1
 EC2 Key Pair Name [EC2_v1]:
 Allowed values for Scheduler:
 1. sge
 2. torque
 3. slurm
 4. awsbatch
 Scheduler [slurm]:
 Allowed values for Scheduler:
 1. sge
 2. torque
 3. slurm
 4. awsbatch
 Scheduler [slurm]:
 Minimum cluster size (instances) [0]: 
 Maximum cluster size (instances) [10]: 
 Master instance type [t2.micro]: 
 Compute instance type [t2.micro]:
 Automate VPC creation? (y/n) [n]: 
 Allowed values for VPC ID:
 1. vpc-579d8e2d | 0 subnets inside
 VPC ID [vpc-579d8e2d]: 
 Allowed values for Network Configuration:
 1. Master in a public subnet and compute fleet in a private subnet
 2. Master and compute fleet in the same public subnet
 Network Configuration [Master in a public subnet and compute fleet in a private subnet]: 1


Useful links: