PyCon talk: Deploy Python apps in 5 min with a PaaS

DEPLOY PYTHON APPS
IN 5 MINUTES
WITH A PAAS
Nate Aune (@natea)
PyCon Santa Clara
March 15, 2013

Sunday, March 17, 13 1

JAZKARTA (2004-NOW)


2010-2011

• Nothing equivalent to Heroku for Django developers

• Built my own PaaS (“How hard can it be?”)

• Shut down last year. Increased competition from big corps
meant a race to the bottom


APPSEMBLER (2012-CURRENT)


AGENDA

• Why should I care?
• What is a PaaS? What are the advantages?
• Which PaaS should I use?
• When might I not want to use a PaaS?


WHERE ARE YOU
DEPLOYING/HOSTING
TODAY?
• Shared hosting (i.e. Webfaction)
• Running your own servers (co-located)?
• Using an IaaS provider (i.e. AWS or Rackspace)
• Already using a platform-as-a-service (PaaS)


MEET DAVE


CONFIGURING A SERVER
• deciding what size to get (memory, disk)
• getting all the dependencies installed on it
• SSHing into the machine to deploy stuff
(feels dirty, but he’s under time pressure)

1 day 1 day


MAINTAINING THE
SERVER

1 day 1 day 1 day


SECURITY

1 day 1 day 1 day 1 day


SCALING

1 day 1 day 1 day 1 day 1 day


OH SH*T
MOMENT.


Sysadmin $80k
te
sed

DBA $90k
Train new guy

6
8 days

WHY TOUCH SERVERS IF YOU
DON’T HAVE TO?


PaaS
Is it the promised land?


WHAT IS A PAAS?

Platform-as-a-Service
enables developers to create
innovative applications without
operational overhead around
conﬁguration, deployment and
management.


LAYERS OF
INFRASTRUCTURE

Source: EngineYard “PaaS - State of the Market Survey” May 2012 - http://venturebeat.com/2012/07/05/engine-yard-paas-infographic/

Source: AppFog “Evolution of the Cloud: Toward a NoOps World” Jan 2012
http://gigaom.com/cloud/why-2013-is-the-year-of-noops-for-programmers-infographic/

EFFICIENT, ELASTIC, SECURE

• Lots of applications co-located on a few servers

• Drastically reduces resources

• Add/remove capacity depending on load

• All secured using SELinux or LXC


LET THEM DO
THE BORING STUFF

• Patches and updates

• Migrating applications

• Backups / snapshots

• Conﬁguring everything (web servers, load balancers,
modules, databases)


FOLLOW ALONG!

$ git clone http://github.com/appsembler/paasbakeoff

$ git branch
dotcloud
elasticbeanstalk
gae
heroku
* master
mezzpycon
openshift
stackato


DEFINE OS DEPENDENCIES

www:
type: python
systempackages:
- openoffice.org
- mysql-client-5.1

5 min


BACKUPS

5m 5m

https://devcenter.heroku.com/articles/heroku-postgres-fork

SSL

5m 5m 5m


SCALING

5m 5m 5m 5m


CACHING

5m 5m 5m 5m 5m


IAAS VS. PAAS

IaaS - days

PaaS - minutes


WHICH PAAS?

• Hosted

• DIY

• Hybrid


WHO ARE THE PLAYERS?

• CloudFoundry
(open source by VMWare) • Heroku
(now Salesforce)
• OpenShift
(Redhat’s open source PaaS)

• AppFog • Dotcloud • App Engine
(Google)
• Stackato
(ActiveState)
• Gondor
(Python only) • Elastic Beanstalk
(Amazon)

• Azure
(Microsoft)


WHO ARE THE PLAYERS?
We’ll look at these ones today.

• CloudFoundry
(open source by VMWare) • Heroku • OpenShift
(now Salesforce) (Redhat’s open source PaaS)

• AppFog • Dotcloud • App Engine
(Google)
• Stackato
(ActiveState)
• Gondor • Elastic Beanstalk
(Python only)
(Amazon)

• Azure (Microsoft)


HEROKU
by Salesforce

One of the ﬁrst PaaS, extensive add-ons catalog


ANATOMY OF A
DJANGO APP
ON HEROKU

customized settings.py for Heroku
if not using S3, must configure static serve

Procfile to configure process

requirements.txt to define dependencies

INSTALL THE HEROKU
TOOLBELT
Download the Heroku Toolbelt
http://toolbelt.herokuapp.com/osx/download

$ heroku login
Enter your Heroku credentials.
Email: someone@example.com
Password: ******
Could not find an existing public key.
Would you like to generate one? [Yn]
Generating new SSH public key.
Uploading ssh public key /Users/someone/.ssh/id_rsa.pub


CREATE APP AND DB
$ heroku create paasbakeoff

Creating paasbakeoff... done, stack is cedar

http://paasbakeoff.herokuapp.com/ | git@heroku.com:paasbakeoff.git

Git remote heroku added

$ heroku addons:add heroku-postgresql:dev

Adding heroku-postgresql:dev on paasbakeoff... done, v3 (free)

Attached as HEROKU_POSTGRESQL_GREEN_URL

Database has been created and is available

Use `heroku addons:docs heroku-postgresql:dev` to view documentation.

$ heroku pg:info

=== HEROKU_POSTGRESQL_GREEN_URL

Plan: Dev

Status: available


SET DB ENV VARIABLES

$ heroku config

=== paasbakeoff Config Vars

HEROKU_POSTGRESQL_GREEN_URL: postgres://x:y@z.com:5432/d2b3c9ichbauv0

$ heroku pg:promote HEROKU_POSTGRESQL_GREEN

Promoting HEROKU_POSTGRESQL_GREEN_URL to DATABASE_URL... done

$ heroku config

=== paasbakeoff Config Vars

DATABASE_URL: postgres://x:y@z.com:5432/d2b3c9ichbauv0


ADD TO SETTINGS.PY

Set the RACK_ENV environment variable to production

$ heroku config:set RACK_ENV=production


USE S3 FOR SERVING STATIC
AND MEDIA FILES

And Sendgrid for sending emails

PROCFILE
web: gunicorn_django -b 0.0.0.0:$PORT -w 9 -k gevent --max-requests 250
--preload mywebsite/settings.py

Conﬁgure AWS settings
$ heroku config:set AWS_ACCESS_KEY_ID=xxxxxxxxxxxxxxxxx
$ heroku config:set AWS_SECRET_ACCESS_KEY=yyyyyyyyyyyyyyyyyyyyyyyyy
$ heroku config:set AWS_STORAGE_BUCKET_NAME=zzzzzzzzzzzzz

Sync, migrate, collectstatic
$ heroku run python mywebsite/manage.py createdb

$ heroku run python mywebsite/manage.py collectstatic


$ git push heroku master

Counting objects: 8, done.

DEPLOY WITH GIT PUSH
Delta compression using up to 2 threads.

Compressing objects: 100% (6/6), done.

Writing objects: 100% (6/6), 616 bytes, done.

Total 6 (delta 3), reused 0 (delta 0)

-----> Heroku receiving push

-----> Python/Django app detected

-----> Preparing Python interpreter (2.7.2)

-----> Creating Virtualenv version 1.7.2

New python executable in .heroku/venv/bin/python2.7

...

Running virtualenv with interpreter /usr/local/bin/python2.7

-----> Activating virtualenv

-----> Installing dependencies using pip version 1.1

Downloading/unpacking Django==1.4.2 (from -r requirements.txt (line 1))

...

Cleaning up...

-----> Collecting static files


ANATOMY OF A
DJANGO APP
ON HEROKU

customized settings.py for Heroku

Procfile to configure process

requirements.txt to define dependencies

MORE HEROKU RESOURCES

• Getting Started with Django on Heroku
https://devcenter.heroku.com/articles/django

• Heroku Hackers Guide ($10 eBook)
http://theherokuhackersguide.com

• Developers Guide to Running Django Applications on Heroku
http://kencochrane.net/blog/2011/11/developers-guide-for-running-django-apps-on-heroku/

• django-skel
http://django-skel.readthedocs.org


DOTCLOUD
polyglot from the start, very ﬂexible, most Python centric


ANATOMY OF A
DJANGO APP
ON
DOTCLOUD
customized settings.py for Dotcloud

createdb.py to create the database
dotcloud.yml to store conﬁg info
mkadmin.py to make the admin user
nginx.conf to conﬁg URL rewriting
postinstall to run syncdb, collectstatic

wsgi.py to serve using uWSGI

INSTALL THE DOTCLOUD
CLIENT
$ sudo pip install dotcloud

$ dotcloud setup

dotCloud username or email: natea@jazkarta.com

Password:

==> dotCloud authentication is complete!

You are recommended to run `dotcloud check` now.


CREATE AND PUSH AN
APP
$ dotcloud create mezz

==> Creating a sandbox application named "mezz"

==> Application "mezz" created.

Connect the current directory to "mezz"? [Y/n]:

==> Connecting with the application "mezz"

==> Connected with default push options: --rsync

$ dotcloud push

==> Pushing code with rsync from "./" to application mezz

building file list ... done

14:17:51: [www.0] Migrating stateful data located in ~/data

14:18:05: [www.0] Launching...

14:18:07: [www.0] Re-routing traffic to the new build...

14:18:08: [www.0] Successfully installed build revision rsync instance #0

ANATOMY OF A
DJANGO APP
ON
DOTCLOUD
customized settings.py for Dotcloud

createdb.py to create the database
dotcloud.yml to store conﬁg info
mkadmin.py to make the admin user
nginx.conf to conﬁg URL rewriting
postinstall to run syncdb, collectstatic


DOTCLOUD.YML


DATABASE IN
SETTINGS.PY


STATIC_ROOT AND
MEDIA_ROOT IN


NGINX.CONF


POSTINSTALL


CREATEDB.PY AND
MKADMIN.PY
createdb.py
https://github.com/dotcloud/django-on-dotcloud/blob/master/createdb.py

mkadmin.py


MORE RESOURCES

• Dotcloud Python docs
http://docs.dotcloud.com/0.9/services/python/

• Dotcloud Django docs
http://docs.dotcloud.com/0.9/tutorials/python/django/

• django-on-dotcloud
https://github.com/dotcloud/django-on-dotcloud/

• python-on-dotcloud
https://github.com/kencochrane/python-on-dotcloud


STACKATO
by ActiveState

Python 3, Run anywhere, New Relic integration

http://appsembler.com/blog/django-deployment-using-stackato/

ANATOMY OF
DJANGO APP
ON STACKATO

customized settings ﬁle

stackato.yml to deﬁne services

STACKATO


Secure using Linux Containers (LXC)


NEW RELIC


Consistent deployment at all stages of the lifecycle


STACKATO FOR DJANGO
1.Download the Stackato client
http://www.activestate.com/stackato/download_client

2.Create a wsgi.py file

3.Create a requirements.txt file
(if you don’t already have one)

4.Edit the DATABASES and MEDIA_ROOT settings in
settings.py file

5.Create a stackato.yml file to persist the

WSGI.PY FILE


REQUIREMENTS.TXT
Django==1.4.2

psycopg2==2.4.5

-e git+git@github.com:yourname/django-awesome.git#egg=django-awesome

Or you can just reference another
requirements ﬁle in your repo

-r requirements/project.txt


DATABASE OVERRIDES


MEDIA_ROOT


TARGET AND LOGIN

$ stackato target api.appsembler.net

$ stackato login --email user@domain.com

Attempting to login to [https://api.appsembler.net]

Password: ********

Successfully logged into [https://api.appsembler.net]


INITIAL PUSH

$ stackato push

Would you like to deploy from the current directory ? [Yn]:

Would you like to use 'paasbakeoff' as application name ? [Yn]:

Detected a Python Application, is this correct ? [Yn]:

Framework: python

Runtime: <framework-specific default>

Application Deployed URL [paasbakeoff.appsembler.net]:

Application Url: paasbakeoff.appsembler.net

Enter Memory Reservation [128M]:


BIND SERVICES
What kind of service ?

1. filesystem

2. memcached

3. mongodb

4. mysql

5. postgresql

6. rabbitmq

7. redis

Choose: 5

Specify the name of the service [postgresql-cf691]:

Creating Service: OK

Binding Service: OK

Create another ? [yN]:

Would you like to save this configuration? [yN]: y

Uploading Application [paasbakeoff]:

AUTO-GENERATED
STACKATO.YML FILE
name: paasbakeoff

instances: 1

framework:

type: python

mem: 128

Let’s add some other requirements to be installed
requirements: requirements:
pypm: pypm:
- pillow -OR- - pillow
- psycopg2 pip:
- psycopg2


RUNNING SYNCDB
$ stackato update -n

$ stackato start

$ stackato logs

$ stackato run python mywebsite/manage.py syncdb --noinput

Creating tables ...

...

Creating table django_comments

Creating table django_comment_flags

Creating default account (username: admin / password: default) ...

Creating default Site 127.0.0.1:8000 ...


HANDLING STATIC
ASSETS
$ stackato run python mywebsite/manage.py collectstatic --noinput
...

Copying '/app/python/lib/python2.7/site-packages/django/contrib/admin/static/admin/css/rtl.css'

Copying '/app/python/lib/python2.7/site-packages/django/contrib/admin/static/admin/css/ie.css'

Copying '/app/python/lib/python2.7/site-packages/django/contrib/admin/static/admin/css/forms.css'

Make sure they were copied to the right place
framework:
type: python
home-dir: app

processes:
web: $STACKATO_UWSGI --static-map /static=$HOME/mywebsite/static


RUN MANAGEMENT
COMMANDS
hooks:

post-staging:

- python mywebsite/manage.py syncdb --noinput

- python mywebsite/manage.py collectstatic --noinput

- python mywebsite/manage.py migrate --noinput

Make sure you add South to the requirements
requirements:
pypm:
- pillow
- psycopg2
- south


PERSISTED FILESYSTEM FOR
FILE UPLOADS
services:

postgresql-cf691: postgresql

filesystem-paasbakeoff: filesystem

Remember to set the MEDIA_ROOT in settings.py:

MEDIA_ROOT = os.environ['STACKATO_FILESYSTEM']


STACKATO.YML FILE


DIRECTORY LAYOUT


OPENSHIFT
by Redhat

Open source, Auto-scaling, Jenkins builds

http://appsembler.com/blog/django-deployment-using-openshift/

Action hooks for running commands
during build, deploy, post-deploy, etc.

/data/ dir to store uploaded media ﬁles

ANATOMY OF AN
OPENSHIFT REPO
.htaccess to serve up static ﬁles
application inside of wsgi dir

setup.py instead of requirements.txt

INSTALL THE RHC CLIENT
$ sudo gem install rhc

$ rhc setup

...

$ rhc domain status
7 tests, 12 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications

100% passed

You may need to add the SSH key and start the SSH agent

$ ssh-add ~/.ssh/id_rsa
$ ssh-agent


QUICK ‘N DIRTY
INSTRUCTIONS
$ rhc app create -a mezzanineopenshift -t python 2.6

$ rhc cartridge add -c mysql-5.1 -a mezzanineopenshift

$ cd mezzanineopenshift

$ git remote add paasbakeoff git://github.com/appsembler/paasbakeoff.git

$ git fetch paasbakeoff

$ git merge paasbakeoff/openshift


Action hooks for running commands
during build, deploy, post-deploy, etc.

/data/ dir to store uploaded media ﬁles

ANATOMY OF AN
OPENSHIFT REPO
.htaccess to serve up static ﬁles
application inside of wsgi dir

setup.py instead of requirements.txt

/WSGI/APPLICATION


SETUP.PY


STATIC_ROOT AND MEDIA_ROOT


/WSGI/STATIC/.HTACCESS


.OPENSHIFT/
ACTION_HOOKS/DEPLOY


.OPENSHIFT/
ACTION_HOOKS/BUILD


CREATE AND BIND THE
DATABASE
$ rhc cartridge add -c mysql-5.1 -a mezz

Password: ******

Adding 'mysql-5.1' to application 'mezz'

Success

mysql-5.1

=========

Properties

==========

Similar for PostgreSQL
$ rhc cartridge add -c postgresql-8.4 -a mezz


$ git push

GIT PUSH TO DEPLOY
Counting objects: 5, done.

Delta compression using up to 2 threads.

Compressing objects: 100% (3/3), done.

Writing objects: 100% (3/3), 498 bytes, done.

Total 3 (delta 1), reused 0 (delta 0)

remote: restart_on_add=false

remote: Waiting for stop to finish

remote: Done

remote: restart_on_add=false

remote: ~/git/mezz.git ~/git/mezz.git

remote: ~/git/mezz.git

remote: Running .openshift/action_hooks/pre_build

remote: setup.py found. Setting up virtualenv

remote: New python executable in /var/lib/openshift/x/python-2.6/virtenv/bin/python

remote: Installing setuptools............done.

remote: Installing pip...............done.

...

MORE RESOURCES

• Getting started with Django on OpenShift
https://openshift.redhat.com/community/get-started/django

• openshift-django-example
https://bitbucket.org/mdoglio/openshift-django-sample

• Rapid Python and Django app deployment
https://openshift.redhat.com/community/blogs/rapid-python-and-django-app-
deployment-to-the-cloud-with-a-paas


ELASTIC BEANSTALK
by Amazon Web Services

Leverages EC2 and S3, Auto-scaling


.elasticbeanstalk directory to store
config files

config file with database settings
for RDS

wsgi.py file

APP ENGINE
by Google

Task queues, Memcache, Blobstore

Inbound/outbound email, sockets via Channel API


ANATOMY OF GAE

lib directory needed since GAE can’t
pip install -r requirements.txt

modified settings file: add lib dir to
python path and DB settings
config file which defines libraries
and static files mappings

SETTINGS.PY


GAE

Dotcloud

OpenShift


App Engine Stackato
app.yaml stackato.yml


Stackato postinstall

OpenShift .openshift/action_hooks/deploy


Stackato OpenShift Dotcloud Heroku
Python 2.7, 3.2 2.6 (2.7) 2.6.5, 2.7.2, 2.7.2
stackato runtimes
3.1.2, 3.2.2
PostgreSQL 9.1 8.4 9.0 9.1.6
MySQL 5.5 5.1 5.1 (Yes, via RDS)

Persisted FS Yes Yes Yes (Yes, via S3)

Redis Yes, 2.4 No Yes, 2.4.11 (Yes, via addon)

MongoDB Yes, 2.0 Yes, 2.2 Yes, 2.2.1 (Yes, via addon)

Memcached Yes, 1.4 No Yes (Yes, via addon)

RabbitMQ Yes, 2.4 No Yes, 2.8.5 (Yes, via addon)

Solr No No Yes, 3.4.0 (Yes, via Websolr)

Cron Yes Yes Yes Yes
Extensible Yes, apt-get install Yes, DIY cartridge Yes, custom service Yes, buildpacks

WebSockets Yes Yes Yes Yes, via Pusher add-on


Google App Engine Elastic Beanstalk
Python 2.7 2.6

PostgreSQL No No
MySQL 5.5 via Cloud SQL Yes, via RDS
Persisted FS Yes, via Blob storage Yes, via S3
Redis No No
MongoDB No No
Memcached Yes Yes, ElasticCache
RabbitMQ No (SQS)
Solr No (CloudSearch)
Cron Yes, via Task Queues No
Extensible No Yes, via AMI
WebSockets Yes, via Channel API No

WHY NOT PAAS?

• Already invested in your own infrastructure.

• Need to run on servers outside U.S.

• Special requirements not met by PaaS services


OPENSHIFT PRICING


DOTCLOUD PRICING


HEROKU PRICING


GOOGLE CLOUD SQL


THANKS!

Questions?


MORE INFO
• Wrap-up from PaaS bake-off
http://appsembler.com/blog/wrap-up-from-paas-bake-off/

• Django deployment using PaaS
http://appsembler.com/blog/django-deployment-using-paas/

• django-deployer
https://github.com/natea/django-deployer

• paasbakeoff - code examples
https://github.com/appsembler/paasbakeoff/


PyCon talk: Deploy Python apps in 5 min with a PaaS

Recomendados

Recomendados

Mais conteúdo relacionado

Destaque

Destaque (20)

Semelhante a PyCon talk: Deploy Python apps in 5 min with a PaaS

Semelhante a PyCon talk: Deploy Python apps in 5 min with a PaaS (20)

Último

Último (20)

PyCon talk: Deploy Python apps in 5 min with a PaaS