Entries with tag python


Two-step authentication with Django

A quick post on django-twostepauth, a package I helped develop. This is a package that enables Django to use token devices for user authentication, like for example the free Google Authenticator available for many mobile phones.

I've been using the two-step token authentication on my gmail account and that reduced my fear of having my email account cracked. I hope other websites, yes I'm thinking of you home banking, start to allow this kind of protection.

If you do not know what two-step authentication is, the video below gives a quick tour of the authentication using the example application shipped with this django package.

The package should be easy to integrate with any website. You can download it from the usual place, or clone it from the repository. As always, feedback is very welcome.


Flying with Tornado on AppEngine

Some time ago I bumped into the posts from Francisco Souza on running appengine with several frameworks (tipfy, django, flask and web2py). My choice of micro framework for appengine is tornado, so when reading those posts I thought that one day I would do a remix with the tornado flavor. And here it is, my tornado remix of Francisco series.

First we need to install the needed libs. Assuming the appengine SDK is already installed, we will need tornado and also WTForms (tipfy and flask already include WTForms, with tornado this lib has to be installed separately).

$mkdir gaeseries-tornado
$wget http://github.com/downloads/facebook/tornado/tornado-1.2.1.tar.gz
$tar zxvf tornado-1.2.1.tar.gz
$mv tornado-1.2.1/tornado gaeseries-tornado/
$wget http://pypi.python.org/packages/source/W/WTForms/WTForms-0.6.2.zip
$unzip WTForms-0.6.2.zip
$mv WTForms-0.6.2/wtforms gaeseries-tornado/

So gaeseries-tornado will be our root project directory and by installing tornado and WTForms in the root they will be available in the python path.

So first thing the app.yaml file, just a minimal setup that sends all urls to main.py since we don't have static files.

application: gaeseries
version: 5
runtime: python
api_version: 1

- url: /.*
  script: main.py

A first minimal tornado application is set in main.py. When creating the tornado application we give a mapping of URLs and request handlers. The URLs can have regular expressions that will be sent as parameters to the request handler get or post method. A dictionary of settings is also given for the application creation, in this case we need to give the path for the application to find the templates (the "templates" folder will be under the project root folder).

import os
import tornado.web
import tornado.wsgi
import wsgiref.handlers
import handlers

settings = {
    'template_path': os.path.join(os.path.dirname(__file__), 'templates'),
    'debug': os.environ.get('SERVER_SOFTWARE', '').startswith('Development/'),

application = tornado.wsgi.WSGIApplication([
  (r'/', handlers.PostListingHandler),
], **settings)

def main():

if __name__ == '__main__':

So next we create models.py to have the application models. This file looks exactly like the example for tipfy or flask since it is pure appengine datastore code.

from google.appengine.ext import db

class Post(db.Model):
    title = db.StringProperty(required = True)
    content = db.TextProperty(required = True)
    when = db.DateTimeProperty(auto_now_add = True)
    author = db.UserProperty(required = True)

Having defined the model, and we can create the application request handlers. These are classes that process the HTTP requests into responses and are similar to tipfy handlers or flask views.

import tornado.web
from models import Post

class PostListingHandler(tornado.web.RequestHandler):
    def get(self):
        posts = Post.all()
        self.render('list_posts.html', posts=posts)

The tornado templates are quite similar to django or jinja2 templates, the concept of blocks, loops and if tags is the same. Like jinja2 (and unlike django) the tornado templates allow arbitrary python code. The example base.html follows bellow.

      <meta http-equiv="Content-type" content="text/html; charset=utf-8"/>
      <title>{% block title %}{% end %}</title>
        {% block content %}
        {% end %}

The list_posts.html template extends the base template.

{% extends "base.html" %}

{% block title %}Posts list{% end %}

{% block content %}
Listing all posts:

{% for post in posts %}
	<li>{{ post.title }} (written by {{ post.author.nickname() }})<br />
	{{ post.content }}</li>
{% end %}
{% end %}

So now everything is ready to run, although no post will be displayed since no content was added to the datastore.

The next step is to create the pages to post content to the site. This area will need authentication and we will use WTForms to simplify the creation and validation of form data. We will keep the form in a new file named forms.py.

import wtforms as forms
from wtforms import validators
from django.utils.datastructures import MultiValueDict

class Form(forms.Form):
    def __init__(self, handler=None, obj=None, prefix='', formdata=None, **kwargs):
        if handler:
            formdata = MultiValueDict()
            for name in handler.request.arguments.keys():
                formdata.setlist(name, handler.get_arguments(name))            
        forms.Form.__init__(self, formdata, obj=obj, prefix=prefix, **kwargs)

class PostForm(Form):
    title = forms.TextField('Title', validators=[validators.Required()])
    content = forms.TextAreaField('Content', 

In the forms code we had to introduce an extra class to adapt the tornado request handler to a multidict that WTForms needs. The form itself is the same as in tipfy or flask (both also use WTForms). The template for displaying the form, file new_post.html follows.

{% extends "base.html" %}

{% block title %}New post{% end %}

{% block content %}
<form action="{{ request.uri }}" method="post" accept-charset="utf-8">
<p><label for="title">{{ form.title.label() }}</label>
	{{ form.title() }}

    {% if form.title.errors %}
    <ul class="errors">
    	{% for error in form.title.errors %}
        <li>{{ error }}</li>
        {% end %}
    {% end %}
<p><label for="content">{{ form.content.label() }}</label>
	{{ form.content() }}

    {% if form.content.errors %}
    <ul class="errors">
    	{% for error in form.content.errors %}
        <li>{{ error }}</li>
        {% end %}
    {% end %}
<p><input type="submit" value="Save post"/></p>
{% end %}

So finally the use of the form in the request handler. This request handler needs to be protected with only access for administrators. In Tornado one has to subclass the RequestHandler and override the get_current_user method to return the logged user. In this example we use the users appengine api for authentication. The revised handlers.py file has a base class that does authentication and has NewPostHandler that manages the creation of post entries.

import tornado.web
from google.appengine.api import users
from models import Post
import forms

class BaseHandler(tornado.web.RequestHandler):
    def get_current_user(self):
        user = users.get_current_user()
        if user: 
            user.is_admin = users.is_current_user_admin()
        return user

class PostListingHandler(tornado.web.RequestHandler):
    def get(self):
        posts = Post.all()
        self.render('list_posts.html', posts=posts)

class NewPostHandler(BaseHandler):
    def get(self):
        if not (self.current_user and self.current_user.is_admin):
            return self.redirect(users.create_login_url(self.request.uri))
        form = forms.PostForm()
        self.render('new_post.html', form=form)
    def post(self):
        if not (self.current_user and self.current_user.is_admin):
            return self.redirect(users.create_login_url(self.request.uri))
        form = forms.PostForm(self)
        if form.validate():
            post = Post(title=form.title.data,
            return self.redirect('/')
        self.render('new_posts.html', form=form)

Last piece, modify main.py to add the /new url to the application.

application = tornado.wsgi.WSGIApplication([
  (r'/', handlers.PostListingHandler),
  (r'/new', handlers.NewPostHandler),
], **settings)

So now start the dev_server, go to /new to create some posts. Hope this simple example was enough to give a taste of how simple it is to run a tornado application on appengine.

$python2.5 /usr/local/google_appengine/dev_appserver.py .

This example code is available in this repository.


Deploying Compacted Javascript with Django

Here is a small extension to the manage command to make deployment of compacted javascript easier (hopefully).

I think this is better explained with a usage example. I have the templates referring both the standard javascript files for easier debugging and compacted ones for deployment (the debug variable is the standard one and allows the split between development/deployment).

{% if debug %}
  <script src="{{ MEDIA_URL }}js/jquery-1.2.2b.js"
  <script src="{{ MEDIA_URL }}js/jquery.cookie.js"
  <script src="{{ MEDIA_URL }}js/jquery.dimensions.js"
  <script src="{{ MEDIA_URL }}js/jquery.hoverIntent.js"
  <script src="{{ MEDIA_URL }}js/jquery.cluetip.js"
{% else %}
  <script src="{{ MEDIA_URL }}js/jqbase-min.js" 
{% endif %}

In the settings file I maintain some variables for the JSC command.

JSC_PATH = '/path_to_media/static/js'

    ('jqbase-min.js',('jquery-1.2.2b.js', 'jquery.cookie.js',
           'jquery.dimensions.js', 'jquery.hoverIntent.js',
           'ui.draggable.ext.js', 'ui.droppable.js',

# either jsmin or jspacker, defaults to jsmin
JSC_METHOD = 'jsmin'

The first one is the path to the javascript files, the second is a list of compacted filenames and list of files to be included in the compacted one. The third setting is the method to compact the javascript, with options being the jsmin or the jspacker.

Then in the command line I run

./management.py jsc 

to build the compacted files before deployment. Some command line parameters are also available, for example:

./management.py jsc -m jspacker -f jqbase-min.js

The jsc command script (jsmin and jspacker included) must be installed according to these instructions.

I would love to ear about other approaches.


Including variables as templates

To manage the website content in a flexible way it's practical to have some portions of some pages generated from variables created as FlatPages. For example, in a frontpage I have a "featured item" section which renders a flatpage variable.

def frontpage(request):
    fp = FlatPage.objects.get(url='featured')
    return render_to_response('frontpage.html', {'fp':fp })
then in the template
...<div id="featured">{{ fp.content }}</div> ...

But sometimes it would also be nice to put some template tags inside those flatpages. Django does not seem to provide a tag for such task (most probably because it can be a bad idea from the security point of view), but the ssi tag does exactly the same thing for files in the filesystem. Based on ssi, the code bellow defines a new tag that includes the content of a variable as a template.

def do_templatevar(parser, token):
    bits = token.contents.split()
    if len(bits) != 2:
        raise TemplateSyntaxError, "%s tag takes one argument" % bits[0]
    return TemplateVarNode(parser.compile_filter(bits[1]))

register.tag('templatevar', do_templatevar)

class TemplateVarNode(Node):
    def __init__(self, content):
        self.content = content

    def render(self, context):
        content = self.content.resolve(context)
            t = Template(content)
            return t.render(context)
        except TemplateSyntaxError, e:
            if settings.DEBUG:
                return "[Included template had syntax error: %s]" % e
                return '' # Fail silently

Now I can put the following code in my flat page

<p>This is my media_url: {{ MEDIA_URL }}</p>

And it will work if I define my template as

...<div id="featured">{% templatevar fp.content %}</div> ...

A last remark to mention databasetemplateloader that allows to load template data from the database. I never tried it, but seems to be a more generic solution (and also a way to have a Zope 1 experience in Django).


Boxes as template tags

With CSS it's very simple to do the markup of a square box. But to make those popular boxes with round corners most approaches use nested divs. For a flexible size box the following markup is typical:

<div class="box">
  <div class="box-outer">
    <div class="box-inner">

It's a bad idea to repeat this code all over the templates. So a template tag can be a good option. The kind of template that would be interesting to have is one that outputs the boilerplate divs given the headline and the content. Something like:

{% menubox "Box title here" %}
    <p>Content here</p>
{% endmenubox %}

Django makes the writing of this tag very easy and the documentation is very clear. Bellow is the result for such tag.

from django.template import Library, Node, TemplateSyntaxError

register = Library()

def do_menubox(parser, token):
    nodelist = parser.parse(('endmenubox',))
        tag_name, title = token.split_contents()
    except ValueError:
        raise TemplateSyntaxError, "%r tag requires exactly two arguments" % \
    return MenuboxNode(nodelist, parser.compile_filter(title))

class MenuboxNode(Node):
    def __init__(self, nodelist, title):
        self.nodelist = nodelist
        self.title = title
    def render(self, context):
        title = self.title.resolve(context)
        output = self.nodelist.render(context)
        return '''<div class="box"><div class="box-outer"><div 
class="box-inner"><h2>%s</h2>%s</div></div></div>''' % (title, output)

register.tag('menubox', do_menubox)

Update: The tag now also works with variable names in the title. The following would output the value of titlevar as the title of the box

{% menubox titlevar %}
    <p>Content here</p>
{% endmenubox %}