AWS Auto Scaling: Difference between revisions

From DISI
Jump to navigation Jump to search
mNo edit summary
mNo edit summary
Line 11: Line 11:




d) prepare your aws_access_key_id and aws_secret_access_key. Those can be found in "My Security Credentials -> Access keys (access key ID and secret access key)" section.
d) prepare your aws_access_key_id and aws_secret_access_key.<br>
Those can be found in "My Security Credentials -> Access keys (access key ID and secret access key)" section.<br>
If you haven't got it yet, press "Create New Access Key" and follow the instructions.
If you haven't got it yet, press "Create New Access Key" and follow the instructions.


Line 17: Line 18:
   Access Key ID [None]: _YOUR_ACCESS_KEY_ID_
   Access Key ID [None]: _YOUR_ACCESS_KEY_ID_
   AWS Secret Access Key [None]: _YOUR_SECRET_ACCESS_KEY_
   AWS Secret Access Key [None]: _YOUR_SECRET_ACCESS_KEY_
  this will be stored in .aws/credentials


e) parallel cluster configuration
NB: We use AWS Region us-east-1, which corresponds to N.Virginia.<br>
You are welcome to re-consider this choice.
  pcluster configure
Allowed values for AWS Region ID:
1. ap-northeast-1
2. ap-northeast-2
3. ap-south-1
4. ap-southeast-1
5. ap-southeast-2
6. ca-central-1
7. eu-central-1
8. eu-north-1
9. eu-west-1
10. eu-west-2
11. eu-west-3
12. sa-east-1
13. us-east-1
14. us-east-2
15. us-west-1
16. us-west-2
AWS Region ID [us-east-1]:
Network & Security -> Key Pairs -> Create New Pair
Allowed values for EC2 Key Pair Name:
1. EC2_v1
EC2 Key Pair Name [EC2_v1]:
Allowed values for Scheduler:
1. sge
2. torque
3. slurm
4. awsbatch
Scheduler [slurm]:
Allowed values for Scheduler:
1. sge
2. torque
3. slurm
4. awsbatch
Scheduler [slurm]:
Minimum cluster size (instances) [0]:
Maximum cluster size (instances) [10]:
Master instance type [t2.micro]:
Compute instance type [t2.micro]:
Automate VPC creation? (y/n) [n]:
Allowed values for VPC ID:
1. vpc-579d8e2d | 0 subnets inside
VPC ID [vpc-579d8e2d]:
Allowed values for Network Configuration:
1. Master in a public subnet and compute fleet in a private subnet
2. Master and compute fleet in the same public subnet
Network Configuration [Master in a public subnet and compute fleet in a private subnet]: 1




Revision as of 13:42, 2 June 2020

A step-by-step instruction how to create a slurm cluster on AWS with auto-scaling possibility


a) upgrade your pip: pip install --upgrade pip

b) install aws client: pip install awscli

c) pip install aws-parallelcluster


d) prepare your aws_access_key_id and aws_secret_access_key.
Those can be found in "My Security Credentials -> Access keys (access key ID and secret access key)" section.
If you haven't got it yet, press "Create New Access Key" and follow the instructions. 

 aws configure
 Access Key ID [None]: _YOUR_ACCESS_KEY_ID_
 AWS Secret Access Key [None]: _YOUR_SECRET_ACCESS_KEY_
 this will be stored in .aws/credentials

e) parallel cluster configuration NB: We use AWS Region us-east-1, which corresponds to N.Virginia.
You are welcome to re-consider this choice. 

 pcluster configure

Allowed values for AWS Region ID: 1. ap-northeast-1 2. ap-northeast-2 3. ap-south-1 4. ap-southeast-1 5. ap-southeast-2 6. ca-central-1 7. eu-central-1 8. eu-north-1 9. eu-west-1 10. eu-west-2 11. eu-west-3 12. sa-east-1 13. us-east-1 14. us-east-2 15. us-west-1 16. us-west-2 AWS Region ID [us-east-1]:

Network & Security -> Key Pairs -> Create New Pair Allowed values for EC2 Key Pair Name: 1. EC2_v1 EC2 Key Pair Name [EC2_v1]: Allowed values for Scheduler: 1. sge 2. torque 3. slurm 4. awsbatch Scheduler [slurm]: Allowed values for Scheduler: 1. sge 2. torque 3. slurm 4. awsbatch Scheduler [slurm]: Minimum cluster size (instances) [0]: Maximum cluster size (instances) [10]: Master instance type [t2.micro]: Compute instance type [t2.micro]: Automate VPC creation? (y/n) [n]: Allowed values for VPC ID: 1. vpc-579d8e2d | 0 subnets inside VPC ID [vpc-579d8e2d]: Allowed values for Network Configuration: 1. Master in a public subnet and compute fleet in a private subnet 2. Master and compute fleet in the same public subnet Network Configuration [Master in a public subnet and compute fleet in a private subnet]: 1


Useful links:

Retrieved from "http://wiki.docking.org/index.php?title=AWS_Auto_Scaling&oldid=12657"