12
May
11

Flying with Tornado on AppEngine

Some time ago I bumped into the posts from Francisco Souza on running appengine with several frameworks (tipfy, django, flask and web2py). My choice of micro framework for appengine is tornado, so when reading those posts I thought that one day I would do a remix with the tornado flavor. And here it is, my tornado remix of Francisco series.

First we need to install the needed libs. Assuming the appengine SDK is already installed, we will need tornado and also WTForms (tipfy and flask already include WTForms, with tornado this lib has to be installed separately).

$mkdir gaeseries-tornado
$wget http://github.com/downloads/facebook/tornado/tornado-1.2.1.tar.gz
$tar zxvf tornado-1.2.1.tar.gz
$mv tornado-1.2.1/tornado gaeseries-tornado/
$wget http://pypi.python.org/packages/source/W/WTForms/WTForms-0.6.2.zip
$unzip WTForms-0.6.2.zip
$mv WTForms-0.6.2/wtforms gaeseries-tornado/

So gaeseries-tornado will be our root project directory and by installing tornado and WTForms in the root they will be available in the python path.

So first thing the app.yaml file, just a minimal setup that sends all urls to main.py since we don't have static files.

application: gaeseries
version: 5
runtime: python
api_version: 1

handlers:
- url: /.*
  script: main.py

A first minimal tornado application is set in main.py. When creating the tornado application we give a mapping of URLs and request handlers. The URLs can have regular expressions that will be sent as parameters to the request handler get or post method. A dictionary of settings is also given for the application creation, in this case we need to give the path for the application to find the templates (the "templates" folder will be under the project root folder).

#main.py
import os
import tornado.web
import tornado.wsgi
import wsgiref.handlers
import handlers

settings = {
    'template_path': os.path.join(os.path.dirname(__file__), 'templates'),
    'debug': os.environ.get('SERVER_SOFTWARE', '').startswith('Development/'),
}

application = tornado.wsgi.WSGIApplication([
  (r'/', handlers.PostListingHandler),
], **settings)

def main():
    wsgiref.handlers.CGIHandler().run(application)

if __name__ == '__main__':
    main()

So next we create models.py to have the application models. This file looks exactly like the example for tipfy or flask since it is pure appengine datastore code.

#models.py
from google.appengine.ext import db

class Post(db.Model):
    title = db.StringProperty(required = True)
    content = db.TextProperty(required = True)
    when = db.DateTimeProperty(auto_now_add = True)
    author = db.UserProperty(required = True)

Having defined the model, and we can create the application request handlers. These are classes that process the HTTP requests into responses and are similar to tipfy handlers or flask views.

#handlers.py
import tornado.web
from models import Post

class PostListingHandler(tornado.web.RequestHandler):
    def get(self):
        posts = Post.all()
        self.render('list_posts.html', posts=posts)

The tornado templates are quite similar to django or jinja2 templates, the concept of blocks, loops and if tags is the same. Like jinja2 (and unlike django) the tornado templates allow arbitrary python code. The example base.html follows bellow.

<html>
    <head>
      <meta http-equiv="Content-type" content="text/html; charset=utf-8"/>
      <title>{% block title %}{% end %}</title>
    </head>
    <body>
        {% block content %}
        {% end %}
    </body>
</html>

The list_posts.html template extends the base template.

{% extends "base.html" %}

{% block title %}Posts list{% end %}

{% block content %}
Listing all posts:

<ul>
{% for post in posts %}
	<li>{{ post.title }} (written by {{ post.author.nickname() }})<br />
	{{ post.content }}</li>
{% end %}
</ul>
{% end %}

So now everything is ready to run, although no post will be displayed since no content was added to the datastore.

The next step is to create the pages to post content to the site. This area will need authentication and we will use WTForms to simplify the creation and validation of form data. We will keep the form in a new file named forms.py.

#forms.py
import wtforms as forms
from wtforms import validators
from django.utils.datastructures import MultiValueDict

class Form(forms.Form):
    def __init__(self, handler=None, obj=None, prefix='', formdata=None, **kwargs):
        if handler:
            formdata = MultiValueDict()
            for name in handler.request.arguments.keys():
                formdata.setlist(name, handler.get_arguments(name))            
        forms.Form.__init__(self, formdata, obj=obj, prefix=prefix, **kwargs)

class PostForm(Form):
    title = forms.TextField('Title', validators=[validators.Required()])
    content = forms.TextAreaField('Content', 
				 		validators=[validators.Required()])

In the forms code we had to introduce an extra class to adapt the tornado request handler to a multidict that WTForms needs. The form itself is the same as in tipfy or flask (both also use WTForms). The template for displaying the form, file new_post.html follows.

{% extends "base.html" %}

{% block title %}New post{% end %}

{% block content %}
<form action="{{ request.uri }}" method="post" accept-charset="utf-8">
<p><label for="title">{{ form.title.label() }}</label>
	{{ form.title() }}

    {% if form.title.errors %}
    <ul class="errors">
    	{% for error in form.title.errors %}
        <li>{{ error }}</li>
        {% end %}
    </ul>
    {% end %}
</p>
<p><label for="content">{{ form.content.label() }}</label>
	{{ form.content() }}

    {% if form.content.errors %}
    <ul class="errors">
    	{% for error in form.content.errors %}
        <li>{{ error }}</li>
        {% end %}
    </ul>
    {% end %}
</p>
<p><input type="submit" value="Save post"/></p>
</form>
{% end %}

So finally the use of the form in the request handler. This request handler needs to be protected with only access for administrators. In Tornado one has to subclass the RequestHandler and override the get_current_user method to return the logged user. In this example we use the users appengine api for authentication. The revised handlers.py file has a base class that does authentication and has NewPostHandler that manages the creation of post entries.

#handlers.py
import tornado.web
from google.appengine.api import users
from models import Post
import forms

class BaseHandler(tornado.web.RequestHandler):
    def get_current_user(self):
        user = users.get_current_user()
        if user: 
            user.is_admin = users.is_current_user_admin()
        return user

class PostListingHandler(tornado.web.RequestHandler):
    def get(self):
        posts = Post.all()
        self.render('list_posts.html', posts=posts)

class NewPostHandler(BaseHandler):
    def get(self):
        if not (self.current_user and self.current_user.is_admin):
            return self.redirect(users.create_login_url(self.request.uri))
        form = forms.PostForm()
        self.render('new_post.html', form=form)
    
    def post(self):
        if not (self.current_user and self.current_user.is_admin):
            return self.redirect(users.create_login_url(self.request.uri))
        form = forms.PostForm(self)
        if form.validate():
            post = Post(title=form.title.data,
                        content=form.content.data,
                        author=self.current_user)
            post.put()
            return self.redirect('/')
        self.render('new_posts.html', form=form)

Last piece, modify main.py to add the /new url to the application.

application = tornado.wsgi.WSGIApplication([
  (r'/', handlers.PostListingHandler),
  (r'/new', handlers.NewPostHandler),
], **settings)

So now start the dev_server, go to /new to create some posts. Hope this simple example was enough to give a taste of how simple it is to run a tornado application on appengine.

$python2.5 /usr/local/google_appengine/dev_appserver.py .

This example code is available in this repository.




8 Responses to “Flying with Tornado on AppEngine”


  1. 1 May 12, 2011 at 12:42 pm by Francisco Souza

    That's awesome!

    Congratz :)

  2. 2 September 11, 2011 at 07:27 am by Pkbyron

    Pedro,

    thanks for the brief tutorial, I like you work.

    Are you still using tornado on GAE?

  3. 3 September 11, 2011 at 10:21 am by Pedro

    Hi Pkbyron, tornado is still my first option for GAE. But currently there are some changes going on GAE (the new pricing model and threads support) so I am waiting to see what comes out of that. It's possible that other frameworks become more cost effective.

  4. 4 September 16, 2011 at 12:36 am by Nullset

    Hi Pedro,

    Great article.

    Do you have any issues with your Tornado expressions rendering properly in your templates using WTForms? (ex. {{ post.author.nickname() }})

    In my web app, when I try to render {{ form.email.label() }} I get the following when I view the source:

    <div><label for="email">Email Address</label>: <input id="email" name="email" type="text" value="" /></div>

    instead of:

    <div><label for=

  5. 5 September 16, 2011 at 12:37 am by Nullset

    Hi Pedro,
    Do you have any issues with your Tornado expressions rendering properly in your templates using WTForms? In my web app, when I try to render {{ form.email.label() }} I get the following when I view the source:

    <div><label for="email">Email Address</label>: <input id="email" name="email" type="text" value="" /></div>

    instead of:

    <div><label for=

  6. 6 September 16, 2011 at 08:50 am by Pedro

    Hi Nullset, the way it works for me

    <tr><td width="80" align="right">{{ form.email.label() }}</td>
    <td width="200" align="left">{{ form.email(class_="sl") }}</td></tr>

    it renders

    <tr><td width="80" align="right"><label for="email">Email</label></td>
    <td width="200" align="left"><input class="sl" id="email" name="email" type="text" value="" /></td></tr>

    Can you post again your problem? (the issue with this comments form is fixed)

  7. 7 August 31, 2014 at 08:58 am by sashko

    Hi Pedro,

    Could you tell me how to make python see google.appengine?

    It tells me:

    Traceback (most recent call last):
    File "tutorial/main.py", line 8, in <module>
    import handlers
    File "/home/sashko/WebProgramming/tutorial/handlers.py", line 3, in <module>
    from google.appengine.api import users
    ImportError: No module named appengine.api

Comments closed for this old post