Multiple EC2 instances with boto and Salt

Published | Permalink |
By

python, devops

Borrowed from the Salt docs:
"Salt delivers a dynamic communication bus for infrastructures that can be used for orchestration, remote execution, configuration management and much more."

This HOWTO is intended to get you up and running a master with one or more salt minions that you can remote-execute commands on. I'm not covering actual usage here, just providing a condensed guide on how to actually get up and running.

Requirements

You'll need:
  • A GNU/Linux host (I use a VPS, but you could use a VM/local Vagrant box with port-forwarding if you're on OS X and you don't want to use brew)
  • An AWS account and some familiarity with EC2 (free tier users won't want more than one micro instance running; expect to incur the usual AWS charges)
Extra notes:
  • Pick your AWS region appropriately based on your location
  • Any values in < > should be replaced with appropriate IPs or hostnames

Salt Master

Let's go ahead and set up the salt master which will manage our minion(s):

1
2
master$ sudo apt-get install salt-master
master$ sudo service salt-master start

Ensure that ports 4505 and 4506 are open on salt master if you're running on a VPS, or port-forward appropriately if it's your current machine.

Whether you're using boto or the AWS Console you'll want to create or use a Security Group that allows port 22 from either 0.0.0.0/0 or your hostname/static IP.

(Optional) Boto

This guide assumes you want to do initial setup via the boto library. You can of course manage your EC2 instances via the AWS Console instead.

First up:

On the salt master:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
master$ python
>>> import boto.ec2
>>> conn = boto.ec2.connect_to_region('ap-southeast-2')
>>> conn.run_instances('ami-b80f9c82',
...     key_name='<your_key>',
...     instance_type='t1.micro',
...     security_groups=['basic-ssh-only'])
Reservation:r-00000000
>>> reservations = conn.get_all_instances()
>>> reservations
[...]
# Find your reservation by its id hash
>>> res[0].instances[0].public_dns_name
u'ec2-<host-ip>.<region>.compute.amazonaws.com'

Minion Setup

Log in to the ec2 instance:

1
2
master$ ssh-add ~/.ssh/<your_aws_keyfile>.pem
master$ ssh ubuntu@ec2-<host-ip>.<region>compute.amazonaws.com

Now we want to install Salt and configure this machine as a minion.

1
2
3
ubuntu@minion1:~$ wget -O - http://bootstrap.saltstack.org | sudo sh
...[ snip ]...
*  INFO: Salt installed!

Set the "master" value to your Salt master's IP or hostname (or use vim):

1
2
ubuntu@minion1:~$ sudo sed -i 's/#master: salt/master: <master-ip>/g' /etc/salt/minion
ubuntu@minion1:~$ sudo salt-minion -d

Back to the salt master host:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
master$ sudo salt-key -L
Accepted Keys:
Unaccepted Keys:
ip-<host-ip>.<region>.compute.internal
Rejected Keys:
(If you trust this host):
master$ sudo salt-key -a ip-<host-ip>.<region>.compute.internal
The following keys are going to be accepted:
Unaccepted Keys:
ip-<host-ip>.<region>.compute.internal
Proceed? [n/Y]
Key for minion ip-<host-ip>.<region>.compute.internal accepted.
master$ sudo salt '*' test.ping
ip-<host-ip>.<region>.compute.internal:
    True
master$

You can add more hosts via boto and go through the above setups to add more minions and run any commands you want against multiple hosts. For example:

1
2
3
4
5
master$ sudo salt '*' cmd.run "echo hello world"
ip-<minion1>.<region>.compute.internal:
hello world
ip-<minion2>.<region>.compute.internal:
hello world

Cloudtastic.

References: