Difference between revisions of "AWS Auto Scaling"

From DISI
Jump to: navigation, search
m
m
Line 11: Line 11:
  
  
d) prepare your aws_access_key_id and aws_secret_access_key. Those can be found in "My Security Credentials -> Access keys (access key ID and secret access key)" section.
+
d) prepare your aws_access_key_id and aws_secret_access_key.<br>
 +
Those can be found in "My Security Credentials -> Access keys (access key ID and secret access key)" section.<br>
 
If you haven't got it yet, press "Create New Access Key" and follow the instructions.
 
If you haven't got it yet, press "Create New Access Key" and follow the instructions.
  
Line 17: Line 18:
 
   Access Key ID [None]: _YOUR_ACCESS_KEY_ID_
 
   Access Key ID [None]: _YOUR_ACCESS_KEY_ID_
 
   AWS Secret Access Key [None]: _YOUR_SECRET_ACCESS_KEY_
 
   AWS Secret Access Key [None]: _YOUR_SECRET_ACCESS_KEY_
 +
  this will be stored in .aws/credentials
  
 +
e) parallel cluster configuration
 +
NB: We use AWS Region us-east-1, which corresponds to N.Virginia.<br>
 +
You are welcome to re-consider this choice.
 +
 +
  pcluster configure
 +
 +
Allowed values for AWS Region ID:
 +
1. ap-northeast-1
 +
2. ap-northeast-2
 +
3. ap-south-1
 +
4. ap-southeast-1
 +
5. ap-southeast-2
 +
6. ca-central-1
 +
7. eu-central-1
 +
8. eu-north-1
 +
9. eu-west-1
 +
10. eu-west-2
 +
11. eu-west-3
 +
12. sa-east-1
 +
13. us-east-1
 +
14. us-east-2
 +
15. us-west-1
 +
16. us-west-2
 +
AWS Region ID [us-east-1]:
 +
 +
Network & Security -> Key Pairs -> Create New Pair
 +
Allowed values for EC2 Key Pair Name:
 +
1. EC2_v1
 +
EC2 Key Pair Name [EC2_v1]:
 +
Allowed values for Scheduler:
 +
1. sge
 +
2. torque
 +
3. slurm
 +
4. awsbatch
 +
Scheduler [slurm]:
 +
Allowed values for Scheduler:
 +
1. sge
 +
2. torque
 +
3. slurm
 +
4. awsbatch
 +
Scheduler [slurm]:
 +
Minimum cluster size (instances) [0]:
 +
Maximum cluster size (instances) [10]:
 +
Master instance type [t2.micro]:
 +
Compute instance type [t2.micro]:
 +
Automate VPC creation? (y/n) [n]:
 +
Allowed values for VPC ID:
 +
1. vpc-579d8e2d | 0 subnets inside
 +
VPC ID [vpc-579d8e2d]:
 +
Allowed values for Network Configuration:
 +
1. Master in a public subnet and compute fleet in a private subnet
 +
2. Master and compute fleet in the same public subnet
 +
Network Configuration [Master in a public subnet and compute fleet in a private subnet]: 1
  
  

Revision as of 05:42, 2 June 2020

A step-by-step instruction how to create a slurm cluster on AWS with auto-scaling possibility


a) upgrade your pip: pip install --upgrade pip

b) install aws client: pip install awscli

c) pip install aws-parallelcluster


d) prepare your aws_access_key_id and aws_secret_access_key.
Those can be found in "My Security Credentials -> Access keys (access key ID and secret access key)" section.
If you haven't got it yet, press "Create New Access Key" and follow the instructions.

 aws configure
 Access Key ID [None]: _YOUR_ACCESS_KEY_ID_
 AWS Secret Access Key [None]: _YOUR_SECRET_ACCESS_KEY_
 this will be stored in .aws/credentials

e) parallel cluster configuration NB: We use AWS Region us-east-1, which corresponds to N.Virginia.
You are welcome to re-consider this choice.

 pcluster configure

Allowed values for AWS Region ID: 1. ap-northeast-1 2. ap-northeast-2 3. ap-south-1 4. ap-southeast-1 5. ap-southeast-2 6. ca-central-1 7. eu-central-1 8. eu-north-1 9. eu-west-1 10. eu-west-2 11. eu-west-3 12. sa-east-1 13. us-east-1 14. us-east-2 15. us-west-1 16. us-west-2 AWS Region ID [us-east-1]:

Network & Security -> Key Pairs -> Create New Pair Allowed values for EC2 Key Pair Name: 1. EC2_v1 EC2 Key Pair Name [EC2_v1]: Allowed values for Scheduler: 1. sge 2. torque 3. slurm 4. awsbatch Scheduler [slurm]: Allowed values for Scheduler: 1. sge 2. torque 3. slurm 4. awsbatch Scheduler [slurm]: Minimum cluster size (instances) [0]: Maximum cluster size (instances) [10]: Master instance type [t2.micro]: Compute instance type [t2.micro]: Automate VPC creation? (y/n) [n]: Allowed values for VPC ID: 1. vpc-579d8e2d | 0 subnets inside VPC ID [vpc-579d8e2d]: Allowed values for Network Configuration: 1. Master in a public subnet and compute fleet in a private subnet 2. Master and compute fleet in the same public subnet Network Configuration [Master in a public subnet and compute fleet in a private subnet]: 1


Useful links: