## Deployment at TrackMaven
## About Me
Matt Camilli
@mlcamilli
matt@trackmaven.com
http://mattcamilli.com
## Where we were
## Where we ended up
## Fabric
Fabric is a python command-line tool for streamlining the use
of SSH for application deployment or systems administration tasks.
We use it to provide an easy interface for our developers to interact
with Ansible and Boto.
## fab prod shell
**fab** - Invokes a fabric command from the fabfile
**prod** - Sets fabric's environment dictionary to production
**shell** - Custom fabric method that will cd into the project directory on the host specified by the environment dictionary, and run the project shell
```python
def shell():
'''
Calls the shell_plus of the TrackMaven project
'''
with env.cd(env.project_path):
env.run('foreman run python trackmaven/manage.py shell_plus')
```
fab prod shell
## Boto
## Tagging
* Env - Prod, Puppystream, Puppycrate
* Type - Web, Celery, Manager
* Name - EnvType# (prodcelery1)
## AWSManager
```python
from boto import ec2
from boto.ec2 import elb
class AWSManager():
REGION = 'us-east-1'
def __init__(self, env):
if not env:
raise Exception('No environment set')
self.env = env
self.ec2 = ec2.connect_to_region(self.REGION,
profile_name="trackmaven")
self.elb = elb.connect_to_region(self.REGION,
profile_name="trackmaven")
def scale_web(self, number):
'''
Scales the web workers to the number inputted
'''
web_instances = self._get_instances('web')
if number > len(web_instances):
self._scale_up('web', number - len(web_instances))
elif number < len(web_instances) and number >= 2:
self._scale_down('web', len(web_instances) - number)
else:
raise Exception('Cannot scale below 2 web instances or scale the same number')
def _scale_up(self, instance_type, number):
'''
Scales the specified instances up
'''
# Fetch the main instance to base the new instances off of
instances = self._get_instances(instance_type)
main = next(instance for instance in instances
if '1' in instance.tags.get('Name'))
image = self.ec2.get_all_images(owners='self', filters={
'tag:Name': main.tags.get('Name')}).pop()
for i in range(0, number):
reservation = self.ec2.run_instances(
image.id,
key_name=main.key_name,
security_groups=[group.name for group in main.groups],
instance_type=main.instance_type,
placement=main.placement
)
instance = reservation.instances[0]
# Add the proper tags
instance.add_tag('env', self.env)
instance.add_tag('type', instance_type)
instance.add_tag('Name', '{}{}{}'.format(
self.env, instance_type, len(instances) + 1 + (i * 1)))
# If it is a web worker, register it with the elb
if instance_type == 'web':
self.elb.register_instances(self.env, [instance.id])
```
## Ansible
Ansible is an open source orchestration engine that manages nodes over SSH. (Built on Python)
## Modules
## Playbook Example
```
---
- hosts: webservers
vars:
http_port: 80
max_clients: 200
remote_user: root
tasks:
- name: ensure apache is at the latest version
yum: pkg=httpd state=latest
- name: write the apache config file
template: src=/srv/httpd.j2 dest=/etc/httpd.conf
notify:
- restart apache
- name: ensure apache is running
service: name=httpd state=started
handlers:
- name: restart apache
service: name=httpd state=restarted
```
## Inventory
A file that describes Hosts and Groups in Ansible (INI or JSON)
```
mail.example.com
[webservers]
foo.example.com
bar.example.com
[dbservers]
one.example.com
two.example.com
three.example.com
[webservers:vars]
some_var=somevalue
```
## Deployment
fab prod deploy
ENVRIONMENT=prod ansible-playbook deployment/playbooks/deploy.yml
-i deployment/hosts -e "branch=master"
-i deployment/hosts
```python
from boto import ec2
import os, json
def main():
instances = {}
instances['_meta'] = { 'hostvars': {}}
env = os.environ.get('ENVIRONMENT')
conn = ec2.connect_to_region('us-east-1', profile_name='trackmaven')
prod_reservations = ec2.get_all_instances(filters={'tag:env': env})
for reservation in prod_reservations:
for instance in reservation.instances:
instance_type = instance.tags.get('type')
# Group instances by type
instances.setdefault(instance_type,
[]).append(instance.public_dns_name)
# Group instances by name
instances.setdefault(instance.tags.get('Name'),
[]).append(instance.public_dns_name)
# Set instance specific variables
instances['_meta']['hostvars'][instance.public_dns_name] = {
'type': instance_type,
'env': env,
'id': instance.id,
'region': 'us-east-1'
}
print json.dumps(instances)
main()
```
output
```
{
"prodweb1":[
"made-up-web1.compute-1.amazonaws.com"
],
"web":[
"made-up-web1.compute-1.amazonaws.com",
"made-up-web2.compute-1.amazonaws.com"
],
"all":[
"made-up-web1.compute-1.amazonaws.com",
"made-up-web2.compute-1.amazonaws.com",
"made-up-celery1.compute-1.amazonaws.com",
"made-up-manager1.compute-1.amazonaws.com"
],
"prodweb2":[
"made-up-web2.compute-1.amazonaws.com"
],
"prodcelery1":[
"made-up-celery1.compute-1.amazonaws.com"
],
"prodmanager1"[
"made-up-manager1.compute-1.amazonaws.com"
],
"celery":[
"made-up-celery1.compute-1.amazonaws.com"
],
"manager":[
"made-up-manager1.compute-1.amazonaws.com"
],
"_meta":{
"hostvars":{
"made-up-web1.compute-1.amazonaws.com":{
"type":"web",
"env":"prod",
"id":"someid",
"region":"us-east-1"
}
}
}
}
```
playbooks/deploy.yml
```
---
- hosts : web
vars:
branch : "{{branch}}"
roles:
- deploy
serial: 1
- hosts : celery:manager
vars:
branch : "{{branch}}"
roles:
- deploy
```
roles/deploy/tasks/main.yml
```
---
- name: Instance De-register
local_action: ec2_elb
args:
instance_id: "{{ hostvars[inventory_hostname].id}}"
state: 'absent'
profile: 'trackmaven'
ec2_elbs: "{{ hostvars[inventory_hostname].env}}"
wait: 'yes'
wait_timeout: 10
region: "{{ hostvars[inventory_hostname].region }}"
when: '"{{ hostvars[inventory_hostname].type }}" == "web"'
- include: deploy.yml
- name: Instance Register
local_action: ec2_elb
args:
instance_id: "{{ hostvars[inventory_hostname].id}}"
state: 'present'
profile: 'trackmaven'
ec2_elbs: "{{ hostvars[inventory_hostname].env}}"
wait: 'yes'
wait_timeout: 20
region: "{{ hostvars[inventory_hostname].region }}"
when: '"{{ hostvars[inventory_hostname].type }}" == "web"'
```
roles/deploy/tasks/deploy.yml
```
---
- name: turn on maintenance mode
sudo: yes
command: chdir=/home/web/www/ touch maintenance
when: '"{{ hostvars[inventory_hostname].type }}" == "web"'
- name: shut down supervisor
sudo: yes
service: name=supervisor state=stopped
- name: pull the latest code
git: repo=git@github.com:TrackMaven/TrackMaven.git dest=/home/web/www/TrackMaven force=yes update=yes version={{branch}}
- name: install lastest requirements
pip: requirements=/home/web/www/TrackMaven/requirements.txt virtualenv=/home/web/www/.virtualenvs/trackmaven
- name: recreate supervisor conf files
sudo: yes
command: chdir=/home/web/www/TrackMaven/ /home/web/www/.virtualenvs/trackmaven/bin/python deployment/export.py -p {{ hostvars[inventory_hostname].procfile }}
- name: remove pycs
command: chdir=/home/web/www/TrackMaven/ find . -name '*.pyc' -delete
- name: start supervisor
sudo: yes
service: name=supervisor state=started
- name: Django collectstatic
command: chdir=/home/web/www/TrackMaven/ foreman run python trackmaven/manage.py collectstatic --noinput
- name: turn off maintenance mode
sudo: yes
command: chdir=/home/web/www/ rm maintenance
when: '"{{ hostvars[inventory_hostname].type }}" == "web"'
```
## Questions?
