Weapons of mass creation
Notes on Django Tutorial at Pycon

Jacob Kaplan Moss

Querysets are cheap and queries (db hits) are expensive.

Models, Views, Urls and Templates, all you need to know to get started.

Views

Where you gather all the “stuff” that you want to display.

Smugmug exploit.

Your URL is a part of you application, you should design it. URLs are user interface. Some frameworks try to hide URLs, but URLs are part of the web and part of the UI.

URLs are “hackable”. You can edit a URL so you can move up a directory, etc. This used to be a “expert” feature, but nowadays people see a logic to it. People chop URLs to get there. Users are getting mor savvy.

Django doesn’t lock you in to a particular URL structure.

Object mapping

  • Unlike Rails which will by default follow your controllers and build URLs automatically.
  • Relationships defined
  • URIs should be opaque. They should define a resource but not what’s behind it.

ROOT_URLCONF is where Django starts.

import * is evil unless the package is designed for it.

from django.conf.urls.defaults import *
urlpatters = patterns('', ...)

Regular expressions, full re power is there, but you probably won’t need it.

Tips:

  • If you’re repeating yourself in patterns then you probably should move that out into a separate url file i.e. include(‘cheeserater.packages.urls’). Just don’t anchor the pattern with $.
  • You can get a little performance increase to move common URLs to top of patterns list.

Dissecting a request /documentation/url_dispatch/

  • GET /packages/Unipath/
  • ROOT_URLCONF
  • cheeserater.urls (walking down the patterns trying to match against the path)
  • (r’^packages/’,…)
  • include (‘cheeserater.packages.urls’)
  • call the view function with the first arg as request

A view function is a python function that takes a request object and returns a response object.

MVC differences

  • Your view is responsible for collecting and collating data but does not include presentation data.
  • Visual representation is in a template.

Using render_to_response() [view code example]

get_object_or_404(SomeModel, **kwargs)
  try:
        return SomeModel.objects.get(**kwargs)
  except SomeModel.DoesNotExist:
        raise Http404(...)

Templates

Glorified chunk of HTML with some variables.

  • PHP is a template language.
  • Ruby uses RHTML.
  • Python has dozens.
  • Django ships with one because we like it, you can use something else if you like. You can use a different one quite easily.

[showing example template]

Django template language has been designed for someone who is used to writing HTML. i.e. a Designer

The magic dot

  • p[“name”]
  • p.name
  • p.name()

You shouldn’t have to tell your template designers if something is a function or method … cheetah template engine idea.

Template filters
filter chaining with | built-in filters
Tags
usually do something with flow and logic, i.e. for, include, if then … same tags require end tags and others don’t

Template inheritance “big whoop idea / feature”

  • solves the problem of common code
  • base template
  • blocks / block tags
  • holes or places that will get filled in later
  • “the news hole” “i need three hundred words to fill this news hole”
  • putting a title block in the base template and let the child define it’s own if it wants to.

[block.super example]

  • Thought django template language was the stupidest idea when he first started at LJ world.
  • Most people are used to includes, while you can do that, it’s not designed for that.
  • [ inheritance example]
  • base template (absolute smallest piece of design, stuff that never changes)
  • might have base templates for each type of content.
  • requires {% extends %} to be right at the beginning.
  • typically 3 or 4 levels of inheritance

Why?

  • Lawrence world was first site built with Django.
    • It was one template before they rebuilt it.
  • matching divs with includes sucks.
  • every designer he’s worked with thinks template inheritance is brilliant.

Inheritance Tips

  • {% extends %} must be the first thing in your template.
  • More {% block %}s are better. Throw empty blocks in your template.
    • i.e. {% block extrahead %}
  • If you’re duplicating content then you’re missing a block.
  • {{ block.super }} keep it in mind
  • /documentation/templates/
  • /documentation/templates_python/

Croyden (postneo.com) friend

Generic Views

Saves time in doing common tasks.

What’s a generic view?

  • early on they realized same stuff was being repeating.
  • when copy and posting code, you know something is wrong.
  • only writing parts of views that differ.
  • pagination
    • getting that right once and not thinking about it again.
  • what does a list of objects do?
    • takes a request and a queryset. (and has lots of options)
  • generic views are just plain old python functions, just views.
    • nothing that special about them other than just used alot and tested.
  • [view code example]
  • package_list(request, package_name):
    • return list_detail.object_list(…)
  • [template code example]
  • include makes sense for pagination widget.

URLconf revisisted

(pattern, callback, extra)

Testing

Apology for being preachy.

“Tests are the Programmer’s stone, transmitting fear into boredom.” — Kent Beck

A good code base vs bade code base = quality of unit testing

Book: Test Driven Development (TDD)

“Never write any new code unless you have a failed test.” Drives Jacob crazy, TOO BORING.

“I didn’t do test driven developement. I do stupidity driven testing … I wait until I do something stupid, and then write tests to aoid doing it again.” — Titus Brown

“Never let one test break.” — Jacob

“Whatever happens, don’t let your test suite break, you’ll regret it …”

Testing Django

  • load sample data
  • deal with HTTP
  • setup and teardown
  • until recently was thought to be too hard
  • rail shipped with a test framework
    • stubs out test for you when you start a new app
    • testing isn’t THAT hard
  • inspired by rails except more pythonic

Doctest

Unit tests

Fixtures
loading data in a reliable manner

Test client

Email capture

  • allows email to be captured so you don’t actually send out email

Doctests

  • String inside triple quotes that looks like an interactive session.
  • Both tests and documentation.
  • Or create tests.py file.
  • /documentation/testing/
  • Siege (test requests/sec)

What else ya got?

ref: Report Lab (integrating PDF)

Notes on Django Code Lab

Django Code Lab
Jacob / James / Adrian

James = release manager
Practical Django Projects

Adrian = everyblock founder
Benevolant Dictators of Life
Def. Guide to Django

Jacob

Code Lab, part 1

Pim Van Heuvan, URL Design
Justin Lilly, unit testing
Richard House, model design
Peter Herndone, searching, optimizing
J Clifford Dyer, next/previous links

Part 2
Bob Haugen, preparing for deployment
Wiley Kestner, sginals
Eric St-Jean, REST APIs
Sean O Connor, order_by(”?”)
Honza Kral, many questions, many answers

Url Design
—————
Questions by Pim van Heuven, Belgium

The case of the URLconf monster
urls.py 2032 lines long
600+URL patterns
94 dicts of keyword arguments
generated dynamically

Worst-case matching
everytime u try to match a regex and fail, it takes time
worst case is a 404
600 regex matches before u get to 404

Break things up
They are already logically separated
Split into multiple files and use include with prefixes.
Drill down into smaller and smaller urlconfs.

Forms taking etra parameters.
Use cases
pass in an existing object to a form and derive
Getting Initial data
django.newforms.models.model_to_dict()
Passing extra parameters into a form
override init()
take extra arguments u want *args, **kwargs
call superclass init()
super() is important
Other Tricks
after super() call

Testing Django
——————
Justin Lilly

Unit tests

Model Design, Richard House
—————————————
“We just need one more field…”

EventType
EventTypeOption
Event
EventOption

Let’s build a form
pass in event_type

No more schema changes.

Peter Herndon
——————-
Searching: query optimization, executing raw SQL, and choosing the right tool for the job.
Works for research library.
synapse.mxkcc.org
synapse google code

The problem
/people/

Raw SQL

Django’s ORM is by far the weakest ORM available. There’s a certain point at which you will have to write SQL.

from django import db

sql = “SELECT… FROM… WHERE end = %s…”
params = [end_date,…]

cursor = db.connection.cursor()
cursor.execute(sql,params)
for row in coursor:

Read your db optimizer docs.

“Bad programmers worry about the code. Good programmers worry about data structures.”

Fulltext search has been done.
Google Coop, Lucene, Solar, Sphinx

Search Integration with Django
djapian, django-sphinx, djangosearch (developed at LJWorld)

from djangosearch.indexer import ModelIndex

results = Document.index.search(query)

Searching over a flat structure is faster than nested.

J. Clifford Dyer
————————
Handling Previous/Next Links

Dave Lowe
————-
When not to use the admin
Moving from Cold Fusion to Django

Apps don’t need to be part of the project.
Multiple sites that share apps/content
Use Site object to
Editor in Site A should not be allowed to Site B.

new forms admin will help that but for now just write your own views.

Get away from having applications inside a project.

Bob Haugen
——————
Prepping for deployment

Coding defensively.
serving media using builtin static server when DEBUG is true.
template dirs based on
urlresolvers.reverse() in a deployed site
raise Http404(“Invalid order ID”) shows up in debug mode

Develop and Deploy
manage.py runserver
Apache + mod_python or mod_wsgi

Pieces of the rest of the stack that
reverse proxy load balancer
Perlbal
(from livejournal)
Apache mod_proxy

spoonfeeding (reverse proxying)
mod_python is moving fast but webserver is slowly moving bits.
load is 0 but new requests are waiting on IO
allow perlbal to handle trickling of data
abstraction of hardware resources
add and remove servers without downtime
perlbal docs kind of suck/non-existent.
ask questions on the list.

Use Memcached
livejournal

Wiley Kestner
——————-
Signals

Notifications

User subscribes to object
When this object changes i want different ways of being notified
User chooses method of notification

Design Pattern (Observer
dispatcher, django has one built in

Dispatcher 101
some piece of code sends a “signal”
just a predefined python obj
django defines some useful signals
django.dispatch.dispatcher.send()

def save_callback():
print “Something just go saved”

dispatcher.connect(save_callback, signal=…)

need to guarantee it gets excecuted
put in models.py

Let’s write a simple listener

No documentation yet (Jacob’s fault, he’ll finish it during the sprint)

http://toys.jacobian.org/presentations/