Now in UTC

from datetime import datetime

Convert from datetime.datetime to time.time

>>> t =
>>> t
datetime.datetime(2011, 11, 5, 11, 26, 15, 37496)

>>> time.mktime(t.timetuple()) + t.microsecond / 1E6

Touch reloading Django sites running with uwsgi in emperor mode

To gracefully reload a site on a new deployment of code, you can simply touch the vassal file, e.g. touch /etc/uwsgi/vassals/example.ini

How size of dict changes on deletes

To avoid thrashing when a mix of additions and deletions are made when the table is near a resize threshold, Python doesn't actually check the # of virgins [never used slots] after a delete (in effect, it assumes you'll soon be replacing the turds [slots that are empty but were used] with citizens [currently used slots] again). So, curiously enough, deleting a key never shrinks the table. A long sequence of deletes followed by an add may shrink it, though. A way to force possible shrinkage without adding a key is:

    dict = dict.copy()

Redirecting unicode output from console directly to file

You could redefine sys.stdout so all output is encoded in utf-8:

import sys
import codecs

Or specify encoding while printing (this seems less appropriate when many prints are used).

print (unicode_obj.encode('utf-8'))

Change working directory to file directory during script execution


Multiple files in with statement

with open('a', 'w') as a, open('b', 'w') as b:

Source code file encoding

# -*- coding: utf-8 -*-

Attempt to write a readonly database error in Django!topic/reviewboard/PhV-bWQlDd8

Set permissions both to database file and to the directory where it is stored for user running the process (typically www-data).

Datetime query in Sqlalchemy

subscriptions = UserCourse.query.filter(UserCourse.next_email_time <=

Add parent directory to sys.path to import modules from it

sys.path.append(os.path.join(os.path.dirname(__file__), '..'))

Alembic error with existing ENUM types in database in Postgres

Use ENUM from postgresql and specify create_type=False:

from sqlalchemy.dialects.postgresql import ENUM

category_enum = ENUM('a', 'b', 'c', name='category_enum', create_type=False)

Auto update and auto default for time attributes in Sqlalchemy

from myapp import db

created_on = db.Column(db.DateTime,
updated_on = db.Column(db.DateTime,,

Jinja macro for WTForms form rendering with errors

{% macro render_field(field) %}
<div class="form_field">
    {{ field.label(class="label") }}
    {% if field.errors %}
    {% set css_class = 'has_error ' + kwargs.pop('class', '') %}
    {{ field(class=css_class, **kwargs) }}
    <ul class="errors">{% for error in field.errors %}<li>{{ error|e }}</li>{% endfor %}</ul>
    {% else %}
    {{ field(**kwargs) }}
    {% endif %}
{% endmacro %}

To specify class

{% from "forms/macros.html" import render_field %}
{{ render_field(form.body, class="email-body") }}


Also see

Use specific browser if webbrowser library is used

export BROWSER='google-chrome'

Convert from string to datetime (string parse time)

Mnemonic rule for remembering the function - string parse time - strptime.

import datetime

DROPBOX_DATE_FORMAT = "%a, %d %b %Y %H:%M:%S +0000"


d = datetime.datetime.strptime(s['modified'], DROPBOX_DATE_FORMAT)):

There is similar function in time module.

Split a string on several delimiters

>>> import re
>>> string = 'this is, a test of splitting; on two delimiters'
>>> re.split(r'[,;]', string)
['this is', ' a test of splitting', ' on two delimiters']
>>> re.split(r'[, ]', string)
['this', 'is', '', 'a', 'test', 'of', 'splitting;', 'on', 'two', 'delimiters']

InlineModelAdmin: Show partially an inline model and link to the complete model

I create a new template called linked.html that is a copy of tabular.html and I added this code to create the link.

{% if %}
          <td class="{{ }}">
              <a href="/admin/{{ app_label }}/{{ inline_admin_formset.opts.admin_model_path }}/{{ }}/">Full record</a>
{% endif %}

then I created a new model LinkedInline inheriting InlineModelAdmin

#override of the InlineModelAdmin to support the link in the tabular inline
class LinkedInline(admin.options.InlineModelAdmin):
    template = "admin/linked.html"
    admin_model_path = None

    def __init__(self, *args):
        super(LinkedInline, self).__init__(*args)
        if self.admin_model_path is None:
            self.admin_model_path = self.model.__name__.lower()

Then when I define a new inline, I have only to use my LinkedInline instead of the normal InlineModelAdmin.

Reseting South migrations!topic/south-users/NKOWJd3xbZE

Add current directory to PYTHONPATH


Delete (remove) files

import os

filelist = [ f for f in os.listdir(".") if f.startswith("progress.") ]
for f in filelist:

XMPP client

Storing (saving) password in keyring

The keyring lib has two functions:

  • get_password(service, username): Returns the password stored in keyring. If the password does not exist, it will return None.
  • set_password(service, username, password): Store the password in the keyring.

Dumping user data in Django

python dumpdata auth.User --indent 4 > users.json

To use s3 storage in Django

another option:

Create User when creating model

from django.db.models.signals import post_save
from django.dispatch import receiver


@receiver(post_save, sender=Bot)
def bot_save_handler(sender, instance, created, **kwargs):
    if created:
        u = User.objects.create_user(, '', '')
        instance.user = u

current transaction is aborted, commands ignored until end of transaction block

from django.db import connection

Makefile settings for migration with South

ifndef APP
    $(MANAGE) migrate
    @echo Starting of migration of $(APP)
    $(MANAGE) schemamigration $(APP) --auto
    $(MANAGE) migrate $(APP)
    @echo Done

ifndef APP
    @echo Please, specify -e APP=appname argument
    @echo Starting init migration of $(APP)
    $(MANAGE) schemamigration $(APP) --initial
    $(MANAGE) migrate $(APP)
    @echo Done

Setup a model to edit in admin

File in app directory:

from django.contrib import admin
from myproject.myapp.models import Author

Setting up db with postgres

Change password if needed

psql -U postgres

postgres=# \password postgres
Enter new password:
Enter it again:

Create user if needed

  $ createuser -U postgres yournewuser -P
Enter password for new role:
Enter it again:
Shall the new role be a superuser? (y/n) n
Shall the new role be allowed to create databases? (y/n) y
Shall the new role be allowed to create more new roles? (y/n) n

Create db

  $ createdb -U yournewuser -E utf8 -O yournewuser yournewdb -T template0


    'default': {
        'ENGINE': 'django.db.backends.postgresql_psycopg2',
        'NAME': 'yournewdb',
        'USER': 'yournewuser',
        'PASSWORD': 'whateverpasswordyouenteredearlier',
        'HOST': '',
        'PORT': '',

or using dj_database_url:

import dj_database_url
DATABASES = {'default': dj_database_url.config(default='postgres://user:password@localhost/name')} for Django

At the very end of your, add this:

    from local_settings import *
except ImportError:

Storing more information about users

from django.contrib.auth.models import User

class UserProfile(models.Model):
    # This field is required.
    user = models.OneToOneField(User)

    # Other fields here
    accepted_eula = models.BooleanField()
    favorite_animal = models.CharField(max_length=20, default="Dragons.")
  AUTH_PROFILE_MODULE = 'accounts.UserProfile'

Django tweek for vim


The only true django tweak I make is before I open vim I'll export the DJANGO_SETTINGS_MODULE environment so that I get code completion for django modules as well:

  export DJANGO_SETTINGS_MODULE=project.settings

I put into Makefile:

    DJANGO_SETTINGS_MODULE=project.settings gvim .

Django Registration



Generating Sphinx docs from source

Documentation is generated from source when adding a path to

sys.path.insert(0, os.path.relpath('../../cghub_api'))

sphinx-apidoc generates completely automatic API documentation for a Python package:

  sphinx-apidoc -o source/ ../cghub_api/

Not capturing standard output with nose

-s: Not capturing stdout

By default, nose captures all output and only presents stdout from tests that fail. By specifying '-s', you can turn this behavior off.

Selecting tests with Nose

To specify which tests to run, pass test names on the command line:


Test names specified may be file or module names, and may optionally indicate the test case to run by separating the module or file name from the test case name with a colon. Filenames may be relative or absolute. Examples:

nosetests test.module
nosetests another.test:TestCase.test_method
nosetests a.test:TestCase
nosetests /path/to/test/

Pip install from Github with -e

There is an issue with virtualenv and pip when installing from github with -e flag. It is reported for instance in this issue with py-github code:

I've encountered it with trying to install from

-e flag means that the source code is editable as described in pip docs.

There are several options:

  • do not use the flag, it works for me for pygpx, in requirements.txt:
  • change directories structure as was done in py-github.

Using nose

cd path/to/project

Testing for exception

def test_afunction_throws_exception(self):
    self.assertRaises(ExpectedException, afunction)

And if afunction takes arguments, just pass them into assertRaises like this:

def test_afunction_throws_exception(self):
    self.assertRaises(ExpectedException, afunction, arg1, arg2)


mkvirtualenv [-a project_path] [-i package] [-r requirements_file] [virtualenv options] ENVNAME


Useful commands

rmvirtualenv ENVNAME

Installation - Shell Startup File

Add three lines to your shell startup file (.bashrc, .profile, etc.) to set the location where the virtual environments should live, the location of your development project directories, and the location of the script installed with this package:

export WORKON_HOME=$HOME/.virtualenvs
source /usr/local/bin/

After editing it, reload the startup file (e.g., run source ~/.bashrc).

Checking that JPEG in PIL works from shell

With a JPG image file_name.jpg in the same directory

from PIL import Image
trial_image ='file_name.jpg')'file_name_test.jpg')

Installing PIL with virtualenv

If PIL is required it is better to install it correctly once in the system and then use virtualenv with --site-packages.

Problems with JPEG

sudo apt-get install libjpeg62-dev
/myEnv/bin/pip install --no-install PIL
cd /myEnv/build/PIL

Change in

JPEG_ROOT = libinclude("/usr/lib")
  /myEnv/bin/pip install PIL

Another possibility

For Ubuntu 11.04, what finally worked for me is:

pip uninstall PIL
sudo apt-get install libjpeg8-dev
pip install PIL

And yet another approach:

Rather than mess with the in PIL, you can also simply create symlinks to the libraries in /usr/lib (assuming you have sudo access).

I did this with the following commands:

sudo ln -s /usr/lib/x86_64-linux-gnu/ /usr/lib/
sudo ln -s /usr/lib/x86_64-linux-gnu/ /usr/lib/
sudo ln -s /usr/lib/x86_64-linux-gnu/ /usr/lib/

And one more good link for Ubuntu 11.10 -

Zen of Python

  • Beautiful is better than ugly.
  • Explicit is better than implicit.
  • Simple is better than complex.
  • Complex is better than complicated.
  • Flat is better than nested.
  • Sparse is better than dense.
  • Readability counts.
  • Special cases aren't special enough to break the rules.
  • Although practicality beats purity.
  • Errors should never pass silently.
  • Unless explicitly silenced.
  • In the face of ambiguity, refuse the temptation to guess.
  • There should be one-- and preferably only one --obvious way to do it.
  • Although that way may not be obvious at first unless you're Dutch.
  • Now is better than never.
  • Although never is often better than right now.
  • If the implementation is hard to explain, it's a bad idea.
  • If the implementation is easy to explain, it may be a good idea.
  • Namespaces are one honking great idea -- let's do more of those!

Nohup and Python

See Python output buffering four ways to do the latter:

  1. Use the -u command line switch
  2. Wrap sys.stdout in an object that flushes after every write
  4. sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 0)

Move from --no-site-packages

Frank Wiles ran into this problem on IRC, where he wanted to add in the site-packages after creating a virtualenv with --no-site-packages. It turns out to be really simple, in that you only have to remove the no-global-site-packages.txt in the lib/python2.x directory inside the virtualenv. After that virtualenv will go ahead and fallback to the global site packages happily.

Test coverage in Python

Coverage utility:

Testing AJAX request with Django

from django.test.client import Client
client = Client()'', {'foo': 'bar'}, **{'HTTP_X_REQUESTED_WITH': 'XMLHttpRequest'})

Conditional expression idiom

Starting with python 2.5, you can use:

x if c else y

In Django 1.4 it will be possible to redefine (override) settings while testing

DEBUG=False in test server in Django

Regardless of the value of the DEBUG setting in your configuration file, all Django tests run with DEBUG=False. This is to ensure that the observed output of your code matches what will be seen in a production setting.

Binding uploaded files to forms

Installing PIP in virtualenv

Template inheritance in Django templates

Using blocks in Django templates

The most powerful -- and thus the most complex -- part of Django's template engine is template inheritance. Template inheritance allows you to build a base "skeleton" template that contains all the common elements of your site and defines blocks that child templates can override.

Step-by-step tutorials for testing in Django

Testing that user logged in and using reverse for URLs

from django.contrib.auth.models import User
from django.core.urlresolvers import reverse
from django.test.client import Client
import unittest

class LoginTestCase(unittest.TestCase):
    def setUp(self):
        self.client = Client()
        self.user = User.objects.create_user('john', '', 'johnpassword')

    def testLogin(self):
        self.client.login(username='john', password='johnpassword')
response = self.client.get(reverse('testlogin-view'))
        self.assertEqual(response.status_code, 200)

I suggest you (if you don't use them already) to use the reverse() function and name your URLs. This way you are sure that you get always the right URL.

Common issue with redirect when testing Django apps

r = self.client.get('/foo')
self.assertEquals(r.status_code, 200)

FAIL Assertion Error: 301 != 200

r = self.client.get('/foo', follow=True)
self.assertEquals(r.status_code, 200)

response.redirect_chain - links visited before a non-redirect was found.

Simple Login Form

By default, the login view renders a template at registration/login.html (you can change this template name by passing an extra view argument ,template_name). This form needs to contain a username and a password field. A simple template might look like this:

{% extends "base.html" %}

{% block content %}

  {% if form.errors %}
    <p class="error">Sorry, that's not a valid username or password</p>
  {% endif %}

  <form action="" method="post">
    <label for="username">User name:</label>
    <input type="text" name="username" value="" id="username">
    <label for="password">Password:</label>
    <input type="password" name="password" value="" id="password">

    <input type="submit" value="login" />
    <input type="hidden" name="next" value="{{ next|escape }}" />

{% endblock %}

When to use django-admin and when

Why do the Django documentation code examples using instead of when demonstrating subcommands such as loaddata and dumpdata?

Well, because these scripts are the same in priciple, with the differences, you already mentioned. The Django docs also mention <subcommand> [options] <subcommand> [options]

side by side. Usually you use to start a new project or application and to do the rest.

Or if really want to use django-admin:

Use virtualenv, and have DJANGO_SETTINGS_MODULE set by bin/activate, and then you can use

Otherwise an error ImportError: Settings cannot be imported, because environment variable DJANGO_SETTINGS_MODULE is undefined.

Django full text search

django-fts on Google code can not be ran right after cloning. I forked and fixed it:

To include with pip in virtualenv add the following line to requirements.txt

-e git+

If you tried Google Code version. Drop fts_ tables. Sync database with Restart the server.

How to determine from which class a method was inherited from?

Use the inspect module.

Multiple Inheritance

Python supports a limited form of multiple inheritance as well. A class definition with multiple base classes looks like this:

class DerivedClassName(Base1, Base2, Base3):

For old-style classes, the only rule is depth-first, left-to-right. Thus, if an attribute is not found in DerivedClassName, it is searched in Base1, then (recursively) in the base classes of Base1, and only if it is not found there, it is searched in Base2, and so on.

With new-style classes, dynamic ordering is necessary because all cases of multiple inheritance exhibit one or more diamond relationships (where at least one of the parent classes can be accessed through multiple paths from the bottommost class). For example, all new-style classes inherit from object, so any case of multiple inheritance provides more than one path to reach object. To keep the base classes from being accessed more than once, the dynamic algorithm linearizes the search order in a way that preserves the left-to-right ordering specified in each class, that calls each parent only once, and that is monotonic (meaning that a class can be subclassed without affecting the precedence order of its parents). Taken together, these properties make it possible to design reliable and extensible classes with multiple inheritance. For more detail, see

New-style and classic classes

Classes and instances come in two flavors: old-style (or classic) and new-style.

Up to Python 2.1, old-style classes were the only flavour available to the user. The concept of (old-style) class is unrelated to the concept of type: if x is an instance of an old-style class, then x.__class__ designates the class of x, but type(x) is always . This reflects the fact that all old-style instances, independently of their class, are implemented with a single built-in type, called instance.

New-style classes were introduced in Python 2.2 to unify classes and types. A new-style class is neither more nor less than a user-defined type. If x is an instance of a new-style class, then type(x) is typically the same as x.__class__ (although this is not guaranteed - a new-style class instance is permitted to override the value returned for x.__class__).

The major motivation for introducing new-style classes is to provide a unified object model with a full meta-model. It also has a number of practical benefits, like the ability to subclass most built-in types, or the introduction of “descriptors”, which enable computed properties.

For compatibility reasons, classes are still old-style by default. New-style classes are created by specifying another new-style class (i.e. a type) as a parent class, or the “top-level type” object if no other parent is needed. The behaviour of new-style classes differs from that of old-style classes in a number of important details in addition to what type() returns. Some of these changes are fundamental to the new object model, like the way special methods are invoked. Others are “fixes” that could not be implemented before for compatibility concerns, like the method resolution order in case of multiple inheritance.

While this manual aims to provide comprehensive coverage of Python’s class mechanics, it may still be lacking in some areas when it comes to its coverage of new-style classes. Please see for sources of additional information.

Old-style classes are removed in Python 3.0, leaving only the semantics of new-style classes.

User Registartion - user registration app for Django using Mongoengine.

Exporting data to Excel in Django

Testing Django from terminal

When testing with Python from terminal

import os
os.environ['DJANGO_SETTINGS_MODULE'] = 'myapp.settings'

Python SMTP server for testing sending emails

import smtpd, asyncore
server = smtpd.DebuggingServer(('localhost', 8000), None)
Last edited by Artem Dudarev, 2013-10-13 02:01:37. Edit