Djangocon 2013

September 2-7, 2013

Tuesday 11:30 a.m.–12:15 p.m.

Scaling Your Write-heavy Django App: a case study

Tobias McNulty

Audience level:
Intermediate

Description

Content management systems and other read-heavy Django apps are relatively easy to scale. Scaling write-heavy apps is another matter. In this talk I’ll walk through our experience scaling the University of Chicago’s custom school survey application to handle over 75,000 requests per minute and upwards of 9,500 PostgreSQL transactions per second.

Abstract

Content management systems and other read-heavy Django apps are relatively easy to scale. You add some caching, and maybe a couple extra web servers for good measure. Scaling write-heavy apps is another matter, both from an app server and a database server perspective. In this talk I’ll walk through our experience scaling the University of Chicago’s custom school survey application to handle over 75,000 requests per minute and upwards of 9,500 PostgreSQL transactions per second. This includes everything from django-debug-toolbar and server provisioning to load testing and consolidated logging. The following is a rough outline of topics to be covered:

  • Eliminating gross inefficiencies with django-debug-toolbar
  • Analyzing your Postgres logs for repetitive queries
  • Establishing a platform you can use to test
  • Creating a test script with JMeter
  • Running your test script at scale
  • Interpreting load test results
  • Things we tried that didn’t work
  • Optimizing your web server configuration
  • Final tweaks to your Postgres configuration