Stop letting bot traffic evict your customers' sessions. A simple Redis configuration switch from LRU to LFU solved our crawler problem, with a Django configuration example.

<div class="w-block-html block-html">
 <iframe allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen="" frameborder="0" height="400" referrerpolicy="strict-origin-when-cross-origin" src="https://www.youtube.com/embed/lSNdp7qpqVo?si=dtY_d8ah_hTV6LM8" title="YouTube video player" width="750">
 </iframe>
</div>
<div class="w-block-markdown block-markdown">
 <div class="prose prose-lg max-w-none">
 
 We've seen a dramatic increase in malicious AI/web crawlers with our clients.
 
 
 These crawlers aren't honoring things like
 <code>
 robots.txt
 </code>
 or headers to indicate a page shouldn't be 
crawled (think links to facets like those in search results) that cause crawlers to get lost in a web of mostly 
useless or duplicated content.
 
 
 On top of that annoyance they're making tens if not
 
 hundreds of requests per second
 
 for hours at a time.
 
 
 These sites are reasonably fast and in many cases are behind awesome cache systems like Cloudflare, so 
why is it impacting our clients negatively?
 
 
 
 User sessions!
 
 These bots are creating thousands of new sessions that push the real human user's sessions out of the cache.
 
 
 We're using
 <a href="https://docs.djangoproject.com/en/dev/topics/http/sessions/">
 Django's Sessions
 </a>
 , but this 
technique works for any browser session token where you want to retain the sessions of real human 
users for as long as possible and expire out these robotic sessions.
 
 <h2>
 Let's talk about sessions
 </h2>
 
 Our default Django site template defaults to using the
 <code>
 cached_db
 </code>
 backend. This tries to get the session from 
Django's default cache (redis in most cases), if it's not found it is retrieved from the database and stored in the 
cache for the next request.
 
 
 Because of these asshole crawlers we've had to rethink this default and instead only keep the session in a cache 
for some of our customers.
 
 
 On most sites it doesn't really matter if the session data is retained. The worst case being the user is forced to log in again.
 
 
 With other sites, the user's session is absolutely crucial to the business. A simple example is we have e-commerce clients 
who want to provide a nice experience to their users. They want a random Internet user to be able to add items to their 
cart over time and only make them login to actually checkout.
 
 
 You need a session for that and you need that session to persist in cache for a reasonable amount of time.
 
 <h2>
 Default Redis
 </h2>
 
 By default Redis will use as much memory as possible and not expire keys that have a TTL set, but when it can no longer store keys in 
RAM it will fail to allow you to create a new session. This often causes whatever session system you're using to raise an exception 
and your site is broken.
 
 
 LRU (Least Recently Used) cache expiration is probably the most commonly known way to expire data. It's
 <a href="https://docs.python.org/3/library/functools.html#functools.lru_cache">
 built into Python
 </a>
 and it is the default for systems like memcached. You could configure or customize your session 
handling to use an LRU mechanism to keep RAM usage under control, but then you still have a problem.
 
 
 When you get a half million bogus AI requests in an hour and you only have RAM to hold say half of that, your real human 
customer who painstakingly curated the perfect cart of high dollar premium widgets but had the audacity to take a break to make 
dinner returns to their browser to find an empty cart and a fresh session.
 
 
 And unless you have a monopoly on widgets, they're likely to take their business elsewhere. I mean, if you can't keep my cart for an hour
why would I trust you with more complicated things like widget manufacturing and shipping logistics!?!?
 
 <h2>
 LFU to the Rescue
 </h2>
 
 I didn't know this until recently but Redis can be configured in a "volatile" mode which causes it to remove items even if the 
keys have TTLs on them. Redis provides
 <a href="https://redis.io/docs/latest/develop/reference/eviction/#eviction-policies">
 several options here
 </a>
 one of which is
 <code>
 volatile-lru
 </code>
 which is what most people would probably choose.
 
 
 However, what we really need here is LFU (Least Frequently Used)! Let me explain.
 
 
 Each of these bots generates a new session and uses it exactly once. This means that any regular human user who 
browses around a couple of pages will see their session remain for far longer. Frequent users, such as the website staff, 
likely would never have their session expire if not for the TTL.
 
 
 To illustrate this a bit I used an LLM to help me quickly generate a
 <a href="https://github.com/revsys/lfu-redis-example">
 demo of LRU vs LFU expiration in Redis
 </a>
 .
 
 
 The demo shows how we can keep shoving single use session keys into a Redis configured for LFU and retain our sessions which are 
used as little as a second time.
 
 <h2>
 Configuring Redis for LFU
 </h2>
 
 To adjust Redis for LFU you need to adjust your
 <code>
 redis.conf
 </code>
 with these three parameters:
 
 <div class="codehilite">
 <pre><code>maxmemory 9mb
maxmemory-policy volatile-lfu 
lazyfree-lazy-eviction yes
</code></pre>
 </div>
 
 This constrains Redis to use at most 9MBs of memory. Obviously you're going to want more session storage than that, so 
adjust it to as much RAM as makes sense for your situation. The next two turn on LFU even if the key has a TTL set (volatile) 
and tells Redis to expire keys asynchronously rather than blocking during the expiration.
 
 
 To do this in Docker Compose is fairly easy as well. You just need to write your own config and load it like this:
 
 <div class="codehilite">
 <pre><code>redis-lfu:
 image: "redis:8.2-alpine"
 command: redis-server /usr/local/redis/redis.conf
 ports:
 - "6001:6379"
 init: true
 stop_signal: SIGKILL
 volumes:
 - ./docker/redis-lfu.conf:/usr/local/redis/redis.conf:ro
</code></pre>
 </div>
 <h2>
 Configuring Django Sessions for LFU
 </h2>
 
 I ran into something I didn't think about when configuring Redis this way for a customer and want to point it out so you 
don't make the same mistake. Many Django sites use Redis and the Django cache for other things and NOT just sessions.
 
 Celery is one good example of where this can go wrong. In that case, we actually deploy two small Redis instances
one configured "normally" and one configured for LFU specifically for user sessions.
 
 
 So our local compose has services like this:
 
 <div class="codehilite">
 <pre><code> redis:
 image: "redis:8.2-alpine"
 ports:
 - "6000:6379"
 init: true
 stop_signal: SIGKILL

 redis-sessions:
 image: "redis:8.2-alpine"
 command: redis-server /usr/local/redis/redis.conf
 ports:
 - "6001:6379"
 init: true
 stop_signal: SIGKILL
 volumes:
 - ./docker/redis-lfu.conf:/usr/local/redis/redis.conf:ro
</code></pre>
 </div>
 
 Obviously you'll need to do something slightly different in Kubernetes or however you're ultimately deploying 
your production applications.
 
 
 And our Django settings cache and session changes look like this:
 
 <div class="codehilite">
 <pre><code># Define two different caches
CACHES = {
 "default": {
 "BACKEND": "django.core.cache.backends.redis.RedisCache",
 "LOCATION": f"redis://redis:6379/0",
 },
 "sessions": {
 "BACKEND": "django.core.cache.backends.redis.RedisCache",
 "LOCATION": f"redis://redis-sessions:6379/0",
 },
}

# Use cached sessions and specifically the LFU configured "sessions" cache 
SESSION_ENGINE = "django.contrib.sessions.backends.cache"
SESSION_CACHE_ALIAS = "sessions"
</code></pre>
 </div>
 
 Hopefully knowing that Redis can support LFU key expiration proves helpful to you!
 
 
 P.S. Dear AI/web bot crawlers, can we chat? I know you're in a hurry&mdash;gathering that 
big heap of data for your 'research' or racing to meet your boss's deadline&mdash;but maybe 
honor robots.txt? Or at least keep it to one request at a time?
 
 
 The Internet might hate you less. Hell, we might even say something nice about your new 
cryptocurrency or whatever.
 
 </div>
</div>

maxmemory 9mb
maxmemory-policy volatile-lfu 
lazyfree-lazy-eviction yes

redis-lfu:
  image: "redis:8.2-alpine"
  command: redis-server /usr/local/redis/redis.conf
  ports:
    - "6001:6379"
  init: true
  stop_signal: SIGKILL
  volumes:
    - ./docker/redis-lfu.conf:/usr/local/redis/redis.conf:ro

  redis:
    image: "redis:8.2-alpine"
    ports:
      - "6000:6379"
    init: true
    stop_signal: SIGKILL

  redis-sessions:
    image: "redis:8.2-alpine"
    command: redis-server /usr/local/redis/redis.conf
    ports:
      - "6001:6379"
    init: true
    stop_signal: SIGKILL
    volumes:
      - ./docker/redis-lfu.conf:/usr/local/redis/redis.conf:ro

# Define two different caches
CACHES = {
    "default": {
        "BACKEND": "django.core.cache.backends.redis.RedisCache",
        "LOCATION": f"redis://redis:6379/0",
    },
    "sessions": {
        "BACKEND": "django.core.cache.backends.redis.RedisCache",
        "LOCATION": f"redis://redis-sessions:6379/0",
    },
}

# Use cached sessions and specifically the LFU configured "sessions" cache 
SESSION_ENGINE = "django.contrib.sessions.backends.cache"
SESSION_CACHE_ALIAS = "sessions"

  
  
   Hopefully knowing that Redis can support LFU key expiration proves helpful to you!
  
  
   P.S. Dear AI/web bot crawlers, can we chat? I know you're in a hurry—gathering that 
big heap of data for your 'research' or racing to meet your boss's deadline—but maybe 
honor robots.txt? Or at least keep it to one request at a time?
  
  
   The Internet might hate you less. Hell, we might even say something nice about your new 
cryptocurrency or whatever.

Sometimes LFU > LRU

At REVSYS, our first attempt at adding JSON-LD to our sites relied on embedding the data in the Django template. For the most part, this has worked fine, and we've had good results from an SEO perspective. But in terms of maintainability, it has not been the most efficient approach. We have now started transitioning to generating the structured data using Django Ninja and Pydantic. As a result, we now have cleaner templates and better maintainability.

Boosting SEO with Django Ninja, Pydantic, and JSON-LD

Writing Django management commands can involve a ton of boilerplate code. But Revsys uses two libraries that cut our management command code in half while making it more readable and powerful: django-click and django-typer.

# management/commands/load_movies_django.py
import json
from pathlib import Path

from django.conf import settings
from django.core.management.base import BaseCommand

from movies.utils import clear_movie_data, load_movies_from_data


class Command(BaseCommand):
    help = "Load movies data from JSON file into the database."

    def add_arguments(self, parser):
        parser.add_argument(
            "--file", default="data/movies.json", help="Path to movies JSON file"
        )
        parser.add_argument(
            "--clear", action="store_true", help="Clear existing data before loading"
        )
        parser.add_argument(
            "count", type=int, nargs="?", help="Number of movies to load (optional)"
        )

    def handle(self, *args, **options):
        if options["clear"]:
            self.stdout.write(self.style.WARNING("Clearing existing movie data..."))
            clear_movie_data()
            self.stdout.write(self.style.SUCCESS("Existing data cleared."))

        file_path = Path(settings.BASE_DIR) / options["file"]

        if not file_path.exists():
            self.stdout.write(self.style.ERROR(f"Error: File {file_path} not found"))
            return

        self.stdout.write(self.style.NOTICE(f"Loading movies from {file_path}..."))

        with open(file_path) as f:
            movies_data = json.load(f)

        count = options["count"]
        if count is not None:
            movies_data = movies_data[:count]
            self.stdout.write(self.style.NOTICE(f"Loading first {count} movies..."))

        total_created_movies, total_created_genres, total_created_cast = (
            load_movies_from_data(movies_data)
        )

        self.stdout.write(self.style.SUCCESS("\nLoading complete!"))
        self.stdout.write(self.style.SUCCESS(f"Created {total_created_movies} movies"))
        self.stdout.write(self.style.SUCCESS(f"Created {total_created_genres} genres"))
        self.stdout.write(
            self.style.SUCCESS(f"Created {total_created_cast} cast members")
        )

pip install django-click

# management/commands/load_movies_click.py
import djclick as click
from django.conf import settings

from movies.utils import clear_movie_data, load_movies_from_data


@click.command()
@click.option("--file", default="data/movies.json", help="Path to movies JSON file")
@click.option("--clear", is_flag=True, help="Clear existing data before loading")
@click.argument("count", type=int, required=False)
def command(file, clear, count):
    """Load movies data from JSON file into the database."""

    if clear:
        click.secho("Clearing existing movie data...", fg="yellow")
        clear_movie_data()
        click.secho("Existing data cleared.", fg="green")

    file_path = Path(settings.BASE_DIR) / file

    if not file_path.exists():
        click.secho(f"Error: File {file_path} not found", fg="red", err=True)
        return

    click.secho(f"Loading movies from {file_path}...", fg="blue")

    with open(file_path) as f:
        movies_data = json.load(f)

    if count is not None:
        movies_data = movies_data[:count]
        click.secho(f"Loading first {count} movies...", fg="cyan")

    total_created_movies, total_created_genres, total_created_cast = (
        load_movies_from_data(movies_data)
    )

    click.secho("\nLoading complete!", fg="green", bold=True)
    click.secho(f"Created {total_created_movies} movies", fg="green")
    click.secho(f"Created {total_created_genres} genres", fg="green")
    click.secho(f"Created {total_created_cast} cast members", fg="green")

import djclick as click
from myapp.models import User

@click.command()
@click.argument('user', type=click.ModelInstance(User))
def command(user):
    """Do something with a user."""
    click.echo(f"Processing user: {user.username}")

# Lookup by username field
@click.argument('user', type=click.ModelInstance(User, lookup='username'))

pip install django-typer

from django_typer.management import Typer
app = Typer(help="Load movies data from JSON file into the database.")

@app.command()
def main(
    count: int | None = typer.Argument(None, help="Number of movies to load"),
    file: str = typer.Option("data/movies.json", help="Path to movies JSON file"),
    clear: bool = typer.Option(False, help="Clear existing data before loading"),
):
    if clear:
        typer.secho("Clearing existing movie data...", fg=typer.colors.YELLOW)
        clear_movie_data()
        typer.secho("Existing data cleared.", fg=typer.colors.GREEN)

    file_path = Path(settings.BASE_DIR) / file

    if not file_path.exists():
        typer.secho(f"Error: File {file_path} not found", fg=typer.colors.RED, err=True)
        raise typer.Exit(1)

    typer.secho(f"Loading movies from {file_path}...", fg=typer.colors.BLUE)

    with open(file_path) as f:
        movies_data = json.load(f)

    total_created_movies, total_created_genres, total_created_cast = (
        load_movies_from_data(movies_data)
    )

    typer.secho("\nLoading complete!", fg=typer.colors.GREEN, bold=True)

def main(count: int = None, file: str = "data/movies.json", clear: bool = False):

from rich.console import Console
from rich.panel import Panel
from rich.progress import Progress, SpinnerColumn, TextColumn
from rich.table import Table

app = Typer(help="Load movies data from JSON file into the database.")

@app.command()
def main(
    # same function definition as before
):
    """Load movies data from JSON file into the database."""
    console = Console()

    # Display a pretty welcome banner
    console.print(Panel.fit("🎬 Movie Database Loader", style="bold blue"))

    with open(file_path) as f:
        movies_data = json.load(f)

    if count is not None:
        movies_data = movies_data[:count]
        console.print(f"🔢 Loading first [bold yellow]{count}[/bold yellow] movies...")

    # Add a progress bar
    with Progress(console=console) as progress:
        task = progress.add_task("🎭 Processing movies...", total=len(movies_data))
        total_created_movies, total_created_genres, total_created_cast = (
            load_movies_from_data(movies_data)
        )
        progress.update(task, completed=len(movies_data))

    # Add a table to summarize the output
    table = Table(title="📊 Loading Summary", style="green")
    table.add_column("Category", style="cyan", no_wrap=True)
    table.add_column("Count", style="magenta", justify="right")
    table.add_column("Icon", justify="center")

    table.add_row("Movies", str(total_created_movies), "🎬")
    table.add_row("Genres", str(total_created_genres), "🎭")
    table.add_row("Cast Members", str(total_created_cast), "👥")

    console.print()
    console.print(table)

# Load movies data into database
load-fresh-movie-data:
    just load-genres --clear 1000
    just load-people --clear 1000
    just load-movies --clear 1000

load-genres *args:
    docker compose exec web python manage.py load_genres {{args}}

load-people *args:
    docker compose exec web python manage.py load_people {{args}}

load-movies *args:
    docker compose exec web python manage.py load_movies {{args}}

  
  
   This pattern lets you create shortcuts for your management commands, and link them together.
  
  
   My honest take
  
  
   I use django-click for most of the management commands I need to write. It's clean, fast to write, and gets out of my way. But when I need to build something that helps me understand complex data or provides structured feedback during long-running operations, django-typer is the better choice.
  
  
   The next time you need to write a management command, try one of these libraries and let me know what you think!

Better Django management commands with django-click and django-typer

Stop link rot before users ever see it: this guide shows how one lightweight app scans your models, flags broken URLs, and slots neatly into any Django workflow.

<div class="w-block-markdown block-markdown">
 <div class="prose prose-lg max-w-none">
 
 Most Django projects store some sort of URLs. Whether it's in a
 <code>
 URLField()
 </code>
 on 
some of your models or deep in some
 <code>
 TextField()
 </code>
 s full of HTML.
 
 
 We all know
 <a href="https://www.w3.org/Provider/Style/URI">
 cool URLs don't change
 </a>
 , but 
not all URLs are cool and some rot over time. How do you find out which of your links 
have disappeared on you?
 
 
 Turns out there is a Django app for that named
 <a href="https://pypi.org/project/django-linkcheck/">
 django-linkcheck
 </a>
 . It will check your URLs on a schedule you set and give you a nice admin page to manage dealing with them.
 
 </div>
</div>
<div class="w-block-content block-content">
 
 
 <img alt="Screenshot of django-linkcheck admin page" class="richtext-image full-width" height="550" src="https://sfo2.digitaloceanspaces.com/revsys-com-prod-media/images/CleanShot_2025-05-29_at_09.55.26.width-800.png" width="800"/>
 
 
</div>
<div class="w-block-markdown block-markdown">
 <div class="prose prose-lg max-w-none">
 
 It helpfully gives you the status it received, links to recheck or ignore this item and quick 'Edit' links to take you right to the model object in question!
 
 
 It hasn't had a release for awhile, but I can confirm it works just fine with Django 
5.1.x. The docs are a bit sparse, so I stumbled around a little bit getting it to work. This post is here to save you from having to also stumble around in the dark.
 
 <h2>
 Install and Setup
 </h2>
 
 Installing the app is straight forward, you just add it to
 <code>
 INSTALLED_APPS
 </code>
 and 
include the special admin view in your main
 <code>
 urls.py
 </code>
 .
 
 
 It then needs to create it's own tables so run:
 
 <div class="codehilite">
 <pre><code>$ ./manage.py migrate 
</code></pre>
 </div>
 
 You then need to tell it which models and fields you want it to handle. The docs 
suggest you run:
 
 <div class="codehilite">
 <pre><code>$ ./manage.py linkcheck_suggest_config --model myapp.SomeModel &gt; myapp/linklinks.py
</code></pre>
 </div>
 
 Which is a great idea, but it's a bit broken actually. This generates something like:
 
 <div class="codehilite">
 <pre><code>from myapp.models import SomeModel

class SomeModelLinklist(Linklist):
 model = SomeModel
 url_fields = [myapp.SomeModel.url]
 ignore_empty = [myapp.SomeModel.url]
 object_filter = {"active": True}

linklists = {
 "SomeModel": SomeModelLinklist,
}
</code></pre>
 </div>
 
 It fails to import
 <code>
 Linklist
 </code>
 for one, but the larger issue is it leads you believe you
set the fields in the form
 <code>
 &lt;app&gt;.&lt;model&gt;.&lt;fieldname&gt;
 </code>
 when in fact it really needs to be 
just a list of modelfield strings. The correct version would be:
 
 <div class="codehilite">
 <pre><code>from linkcheck import Linklist # actually import Linklist
from myapp.models import SomeModel

class SomeModelLinklist(Linklist):
 model = SomeModel
 url_fields = ["url"] # list fields as strings 
 ignore_empty = ["url""]
 object_filter = {"active": True}

linklists = {
 "SomeModel": SomeModelLinklist,
}
</code></pre>
 </div>
 
 This could be very confusing to a first time user or newer Django developer, but don't 
let that stop you. It's a really nice app that solves a real world issue.
 
 <h2>
 Checking your links
 </h2>
 
 To actually check your URLs from your models you then need to run two commands:
 
 <div class="codehilite">
 <pre><code>$ ./manage.py findlinks 
</code></pre>
 </div>
 
 This populates linkcheck's tables with the TODO list of sorts.
 
 
 You can then
 
 actually check
 
 them with:
 
 <div class="codehilite">
 <pre><code>$ ./manage.py checklinks 
</code></pre>
 </div>
 
 
 NOTE:
 
 Both of these should be re-run periodically, perhaps via a cron job, based on 
your personal needs.
 
 
 If you have any broken links they should be visible and ready to handle at
 <code>
 /admin/linkcheck/
 </code>
 now.
 
 
 Check out the
 <a href="https://github.com/DjangoAdminHackers/django-linkcheck?tab=readme-ov-file#settings">
 README
 </a>
 for more information on how you can set this up including settings for how frequently you would like the links to be rechecked.
 
 
 Hopefully these small tips make it easier to start using this great Django app. Cheers!
 
 </div>
</div>

$ ./manage.py migrate 

$ ./manage.py linkcheck_suggest_config --model myapp.SomeModel > myapp/linklinks.py

from myapp.models import SomeModel

class SomeModelLinklist(Linklist):
    model = SomeModel
    url_fields = [myapp.SomeModel.url]
    ignore_empty = [myapp.SomeModel.url]
    object_filter = {"active": True}

linklists = {
    "SomeModel": SomeModelLinklist,
}

from linkcheck import Linklist   # actually import Linklist
from myapp.models import SomeModel

class SomeModelLinklist(Linklist):
    model = SomeModel
    url_fields = ["url"]   # list fields as strings 
    ignore_empty = ["url""]
    object_filter = {"active": True}

linklists = {
    "SomeModel": SomeModelLinklist,
}

$ ./manage.py findlinks 

$ ./manage.py checklinks 

  
  
   
    NOTE:
   
   Both of these should be re-run periodically, perhaps via a cron job, based on 
your personal needs.
  
  
   If you have any broken links they should be visible and ready to handle at
   
    /admin/linkcheck/
   
   now.
  
  
   Check out the
   
    README
   
   for more information on how you can set this up including settings for how frequently you would like the links to be rechecked.
  
  
   Hopefully these small tips make it easier to start using this great Django app.  Cheers!

Give Your Django Admin X‑Ray Vision—Automatic Dead‑Link Detection

The 4 Docker Sins Killing Your Python Productivity. You changed a single line of code and Docker has to rebuild everything! Dramatically improve both fresh builds and rebuilds when working in Docker.

<div class="w-block-markdown block-markdown">
 <div class="prose prose-lg max-w-none">
 
 If you're waiting more than 10 seconds for your Docker rebuilds, you're wasting hours of your life!
 
 
 So that you keep reading, here are the results!
 
 <h2>
 Results
 </h2>
 <ul>
 <li>
 Fresh builds: 50s &rarr; 18s (2.7x faster)
 </li>
 <li>
 Rebuilds: 47s &rarr; 0.4s (117x faster)
 </li>
 <li>
 Adding dependencies: 50s &rarr; 6s (8x faster)
 </li>
 </ul>
 <h2>
 Overview
 </h2>
 
 We work with Python development teams of all sizes and levels of sophistication. One area where 
many teams struggle is optimizing their Docker image build times.
 
 
 I was reminded of this yesterday replying to a recent
 <a href="https://old.reddit.com/r/django/comments/1klljn8/too_many_installed_django_apps/">
 r/django reddit thread
 </a>
 where the author assumed they needed to break their Django monolith up into a few services to reduce 
their build time, which isn't accurate, but is a common mistaken assumption.
 
 
 There are a few small things you can do to
 
 dramatically
 
 reduce the amount of time it takes to 
build (and more importantly REBUILD) a Python or Django docker image.
 
 
 The TL;DR is you need to:
 
 <ul>
 <li>
 Only rebuild the dependency layer when your dependencies actually change
 </li>
 <li>
 Cache your PyPI downloads locally and in CI
 </li>
 <li>
 Switch to using uv which is stupid stupid fast
 </li>
 <li>
 Use a multi-stage build
 </li>
 </ul>
 
 If you're already doing all of those things, you can safely skip the rest of this post.
 
 <h2>
 Naive Approach
 </h2>
 
 The most common problem and impactful issue is not ordering the lines in your Dockerfile 
to only rebuild the layers that need to be rebuilt.
 
 
 Not having to do anything at all is the fastest thing there is! It's a one 
million percent improvement! 🤣
 
 
 Here is what many people start with:
 
 <div class="codehilite">
 <pre><code>FROM python:3.13-slim

RUN mkdir /code
WORKDIR /code

COPY . /code/

RUN pip install -r requirements.txt

# ... rest of lines ending in a `CMD` to run
</code></pre>
 </div>
 
 So what's wrong with this? Because every time you change ANYTHING in your git repository 
you're re-installing your pip dependencies.
 
 
 This is thankfully easy to resolve. Instead of the above, you should be doing this:
 
 <div class="codehilite">
 <pre><code>FROM python:3.13-slim

RUN mkdir /code
WORKDIR /code

# Copy just the requirements first 
COPY ./requirements.txt /code/requirements.txt 

# Install dependencies 
RUN pip install -r requirements.txt

# Copy in everything else 
COPY . /code/

# ... rest of lines ending in a `CMD` to run
</code></pre>
 </div>
 
 Now when you rebuild this image it will only need to perform the
 <code>
 pip install
 </code>
 step when there has actually been a change to
your
 <code>
 requirements.txt
 </code>
 !
 
 
 Dependencies change somewhat frequently, but no where near as frequently as you change code, docs, tests, and your README this
stops wasting time rebuilding that particular Docker layer on every single change.
 
 <h2>
 Caching the PyPI dependencies
 </h2>
 
 Ok so now we're only doing this when there is really something new to do. The next thing to do is not bother downloading all of 
these dependencies each and every time we build our Docker image. By default pip caches your downloads when using it locally, 
so this little optimization is overlooked. Python developers either assume it IS happening inside Docker or that it is hard 
or impossible to make it do so.
 
 <h3>
 Where does pip cache things?
 </h3>
 
 You can
 <a href="https://pip.pypa.io/en/latest/cli/pip_cache/">
 manage your pip cache
 </a>
 but the most useful thing is to simply know where 
this cache exists. So run
 <code>
 pip cache dir
 </code>
 (or
 <code>
 uv cache dir
 </code>
 if you're already using
 <code>
 uv
 </code>
 we'll talk about it more later). If you 
look into that directory hopefully you'll see a bunch of files.
 
 
 Now this is the cache on your HOST OS, not inside of Docker. There are a couple of ways to expose this into your Docker image, but it's much easier to just have your Docker daemon cache it for you.
 
 
 If you're using a default Python docker image, you're running in Debian and by default everything is running as the root user. FYI there are security implications to this and you look into running your code as another user non-root user, but that's a topic for another post.
 
 
 So for the
 <code>
 root
 </code>
 user on a Debian system this makes the pip and uv cache locations are going to be in
 <code>
 /root/.cache/
 </code>
 so we need to make a small change to your
 <code>
 RUN
 </code>
 that installs everything.
 
 
 Instead of:
 
 
 <code>
 RUN pip install -r requirements.txt
 </code>
 
 
 We need to use:
 
 <div class="codehilite">
 <pre><code>RUN --mount=type=cache,target=/root/.cache,id=pip \
 python -m pip install -r /code/requirements.txt
</code></pre>
 </div>
 
 This is instructing the Docker daemon to cache this folder with the id
 <code>
 pip
 </code>
 and it will then be reused across builds.
 
 <h3>
 What about in CI?
 </h3>
 
 Things are a bit harder in CI. Depending on what CI system you're using it's sometimes built in, sometimes you need to make configuration adjustments. In any case, the goal you're after here is that the
 <code>
 /root/.cache/
 </code>
 folder is preserved and reused across builds so that the 
downloads are cached between CI runs.
 
 
 You can read up on all of the details of to
 <a href="https://docs.docker.com/build/cache/optimize/">
 optimize Docker cache usage
 </a>
 in the Docker docs.
 
 <h2>
 Use uv
 </h2>
 
 If you're not familiar with
 <a href="https://docs.astral.sh/uv/">
 uv
 </a>
 it's a near drop-in replacement 
for pip from the folks at Astral who also brought us the great
 <code>
 ruff
 </code>
 linting and formatting tool and the soon to be beta
 <a href="https://github.com/astral-sh/ty">
 ty
 </a>
 type checker.
 
 
 For most things you just prefix your normal pip command with
 <code>
 uv
 </code>
 and it works as expected, 
just a
 
 HELL OF A LOT
 
 faster.
 
 
 Switching to
 <code>
 uv
 </code>
 and adding in the cache mount makes our example Dockerfile now look like
this:
 
 <div class="codehilite">
 <pre><code>FROM python:3.13-slim

RUN mkdir /code
WORKDIR /code

# Install uv 
RUN --mount=type=cache,target=/root/.cache,id=pip \
 python -m pip install uv 

# Copy just the requirements first 
COPY ./requirements.txt /code/requirements.txt 

# Run uv pip install with caching! 
RUN --mount=type=cache,target=/root/.cache,id=pip \
 uv pip install --system -r /code/requirements.txt

# Copy in everything else 
COPY . /code/

# ... rest of lines ending in a `CMD` to run
</code></pre>
 </div>
 <h2>
 So how fast is it now?
 </h2>
 
 Things are quite a bit faster at the small expensive of a slightly more complicated
 <code>
 Dockerfile
 </code>
 .
 
 
 Naive Fresh - 50 seconds
 
 
 Naive Rebuild - 47 seconds.
 
 
 The difference here is just the speed of downloading the pip dependencies between runs.
 
 
 After we've fixed things to only re-run
 <code>
 pip install
 </code>
 when those requirements change gives us 
the biggest benefit.
 
 
 Naive Fixed Fresh - 50 seconds
 
 
 Naive Fixed Rebuild -
 
 10 seconds
 
 
 <h3>
 With Caching
 </h3>
 
 Caching our downloads improves our situation even further!
 
 
 Cached Fresh - 44 seconds
 
 
 Cached Rebuild -
 
 0.4 seconds
 
 
 <h3>
 With Caching and uv
 </h3>
 
 UV Fresh - 18.5 seconds
 
 
 UV Rebuild -
 
 0.4 seconds
 
 
 
 Why isn't
 <code>
 uv
 </code>
 faster? Well it IS faster downloading the files initially, I'm guessing it is doing something in parallel better or just being written in 
rust is making this aspect twice as fast as normal pip. But for these last two we're really just testing how faster Docker is able to 
create the layer since there is really no calls to pip or uv going on.
 
 <h2>
 Adding a new pip dependency into the mix
 </h2>
 
 The
 
 real speed up
 
 is when you need to add a new dependency. In our original requirements.txt we neglected to add the very useful
 <code>
 django-debug-toolbar
 </code>
 package. So I added it and re-ran all of these.
 
 <h3>
 Naive
 </h3>
 
 Naive Fresh - 50 seconds
 
 
 Naive Rebuild - 47 seconds.
 
 
 Naive Rebuild w/ new dependency -
 
 50 seconds
 
 
 
 Naive Fixed Fresh - 50 seconds
 
 
 Naive Fixed Rebuild - 10 seconds
 
 
 Naive Fixed Rebuild w/ new dependency -
 
 51 seconds
 
 
 <h3>
 With Caching
 </h3>
 
 Cached Fresh - 44 seconds
 
 
 Cached Rebuild - 0.4 seconds
 
 
 Cached Rebuild w/ new dependency -
 
 24 seconds
 
 
 <h3>
 With Caching and uv
 </h3>
 
 UV Fresh - 18.5 seconds
 
 
 UV Rebuild - 0.4 seconds
 
 
 UV Rebuild w/ new dependency -
 
 6 seconds
 
 
 
 So we went from a consistent 50ish seconds per build to 18 seconds for a fresh build, 6 seconds when adding a new dependency and 
nearly instant for rebuilds with no dependency changes.
 
 <h2>
 Bonus info
 </h2>
 <h3>
 Multi-stage Docker Builds with Python
 </h3>
 
 What are multi-stage builds? In short, they are
 <code>
 Dockerfile
 </code>
 s with multiple
 <code>
 FROM
 </code>
 lines.
 
 
 Why would I want to do that? Well size and security mainly.
 
 
 On the security front, using a multi-stage build allows you to deploy an image that does not include any 
compilers or build tools, but still use those tools to build the dependencies you use. In terms of size,
the your final image only includes the initial runtime environment, your built dependencies, but not any of 
the tools or dev packages needed to build those dependencies.
 
 
 So you get a smaller and more secure image, which are good things and they add just a
 
 BIT
 
 more complexity to 
your Dockerfile. Once you've been walked through it, it should be fairly clear.
 
 <div class="codehilite">
 <pre><code>FROM python:3.13-slim AS builder-py

RUN mkdir /code
WORKDIR /code

# Install uv 
RUN --mount=type=cache,target=/root/.cache,id=pip \
 python -m pip install uv 

# Copy just the requirements first 
COPY ./requirements.txt /code/requirements.txt 

# Run uv pip install with caching! 
RUN --mount=type=cache,target=/root/.cache,id=pip \
 uv pip install --system -r /code/requirements.txt

FROM python:3.13-slim AS release 

# Copy our system wide installed pip dependencies from builder-py
COPY --from=builder-py /usr/local /usr/local

# Copy in everything else 
COPY . /code/

# ... rest of lines ending in a `CMD` to run
</code></pre>
 </div>
 <h3>
 Benchmark / Testing done here
 </h3>
 
 You can find the exact Docker files and bits I used to do this testing here in
 <a href="https://github.com/revsys/blog-faster-docker-builds">
 this repo
 </a>
 .
 
 
 I did this testing on a M4 Max MacBook Pro with 128GBs of RAM on a 1.2 Gbps fiber internet connection while catching up on 
watching some PyCon 2025 talks. I'm also using Orbstack which improves the overall performance of Docker on MacOS. Your 
results will almost certainly vary, but doing any of these steps will save you and your team time in your CI pipelines and 
when building images locally. The small differences in download speed or available CPU don't really matter, we aren't doing a
CPU heavy micro-benchmark here.
 
 
 Our time on this planet is short, too short to spend it waiting for Docker to needlessly rebuild images.
 
 
 Do yourself a favor a start using these tips now!
 
 </div>
</div>

FROM python:3.13-slim

RUN mkdir /code
WORKDIR /code

COPY . /code/

RUN pip install -r requirements.txt

# ... rest of lines ending in a `CMD` to run

FROM python:3.13-slim

RUN mkdir /code
WORKDIR /code

# Copy just the requirements first 
COPY ./requirements.txt /code/requirements.txt 

# Install dependencies 
RUN pip install -r requirements.txt

# Copy in everything else 
COPY . /code/

# ... rest of lines ending in a `CMD` to run

RUN --mount=type=cache,target=/root/.cache,id=pip \
    python -m pip install -r /code/requirements.txt

FROM python:3.13-slim

RUN mkdir /code
WORKDIR /code

# Install uv 
RUN --mount=type=cache,target=/root/.cache,id=pip \
    python -m pip install uv 

# Copy just the requirements first 
COPY ./requirements.txt /code/requirements.txt 

# Run uv pip install with caching! 
RUN --mount=type=cache,target=/root/.cache,id=pip \
    uv pip install --system -r /code/requirements.txt

# Copy in everything else 
COPY . /code/

# ... rest of lines ending in a `CMD` to run

FROM python:3.13-slim AS builder-py

RUN mkdir /code
WORKDIR /code

# Install uv 
RUN --mount=type=cache,target=/root/.cache,id=pip \
    python -m pip install uv 

# Copy just the requirements first 
COPY ./requirements.txt /code/requirements.txt 

# Run uv pip install with caching! 
RUN --mount=type=cache,target=/root/.cache,id=pip \
    uv pip install --system -r /code/requirements.txt

FROM python:3.13-slim AS release 

# Copy our system wide installed pip dependencies from builder-py
COPY --from=builder-py /usr/local /usr/local

# Copy in everything else 
COPY . /code/

# ... rest of lines ending in a `CMD` to run

  
  
   Benchmark / Testing done here
  
  
   You can find the exact Docker files and bits I used to do this testing here in
   
    this repo
   
   .
  
  
   I did this testing on a M4 Max MacBook Pro with 128GBs of RAM on a 1.2 Gbps fiber internet connection while catching up on 
watching some PyCon 2025 talks. I'm also using Orbstack which improves the overall performance of Docker on MacOS. Your 
results will almost certainly vary, but doing any of these steps will save you and your team time in your CI pipelines and 
when building images locally. The small differences in download speed or available CPU don't really matter, we aren't doing a
CPU heavy micro-benchmark here.
  
  
   Our time on this planet is short, too short to spend it waiting for Docker to needlessly rebuild images.
  
  
   Do yourself a favor a start using these tips now!

Faster Python Docker Builds

To kick off a semi-regular series of blog posts about useful Django apps, we start with some definitions

<div class="w-block-markdown block-markdown">
 <div class="prose prose-lg max-w-none">
 <h2>
 Introduction
 </h2>
 
 We'll be doing a semi-regular series of blog posts about various useful Django apps. However, it seemed like a good idea to lay some groundwork first. That will help us do a few things:
 
 <ul>
 <li>
 define
 <code>
 app
 </code>
 along with a few other similar terms, and get some namespace confusion out of the way
 </li>
 <li>
 understand the value, as a
 
 developer
 
 , of creating separate apps in your Django projects
 </li>
 <li>
 understand the value, as a
 
 developer
 
 or
 
 end-user
 
 of a Django project, of using separate apps (third-party or internally-created) in your Django projects
 </li>
 </ul>
 <h2>
 What is an App? Context matters.
 </h2>
 
 If you go to the Django home page, https://www.djangoproject.com, the blurb at the top says:
 
 Django makes it easier to build better web apps more quickly and with less code.
 
 Django is often described as a
 <a href="https://en.wikipedia.org/wiki/Web_framework">
 web application framework
 </a>
 or something that you use to create web applications (web apps). I've used this description myself. I have also purposefully distinguished between a
 <code>
 web app
 </code>
 and a
 <code>
 website
 </code>
 during a Django tutorial I taught. I've also written a
 <a href="https://www.revsys.com/tidbits/website-vs-web-app-hidden-complexities/">
 blog post
 </a>
 on that topic. So, if Django is used to BUILD web apps, why are we talking about apps IN Django? The context shifts a bit if you're looking at something built with Django from the outside vs. from the inside.
 
 <h2>
 A View From The Inside
 </h2>
 
 From the outside, the specific nomenclature is much less important. But from the inside, as someone building something with Django, the specific terminology has more meaning. But, it can still be a little confusing.
 
 
 Internally, Django views an entire collection of code, what most would call a "website," as a
 <a href="https://docs.djangoproject.com/en/5.1/intro/tutorial01/#creating-a-project">
 project
 </a>
 . This is where the settings and configuration for a specific Django website are contained. Inside that project are 1 or more
 <a href="https://docs.djangoproject.com/en/5.1/intro/tutorial01/#creating-the-polls-app">
 apps
 </a>
 . An app is usually a specific sub-system used within the project. It's a collection of related code that provides specific functionality that's available to the rest of the project.
 
 <h3>
 A Github Example
 </h3>
 
 We can look at GitHub and how its functionality is split up as an example.
 
 Note
 
 : Github is NOT built with Django. But it's a web application that's known to the people who'll probably be reading this post. So, we'll pretend Github is built in Django for this example.
 
 
 Looking at the
 <a href="https://github.com/python/cpython">
 repository for Python
 </a>
 , we can see that
 <a href="https://github.com/python/cpython/issues">
 the issue tracker
 </a>
 ,
 <a href="https://github.com/python/cpython/pulls">
 management of pull requests
 </a>
 , and
 <a href="https://github.com/python/cpython/actions">
 tracking CI/CD activity
 </a>
 are all handling different tasks in different parts of the GitHub web application. But those bits of functionality are available to be reused across the web app on each different repository. When I talk about "bits of functionality" from a Django perspective, I'm talking about the database tables, views, other code and endpoints related to each task. So, the tables, views, other code and endpoints needed to manage Github Issues would all live in one app, perhaps called
 <code>
 issues
 </code>
 . It would be the same for the
 <code>
 pull_requests
 </code>
 and
 <code>
 ci_actions
 </code>
 apps. As such, that functionality is available to any other part of the larger Github project. At the same time, any changes that needed to be made to one of those apps (model updates, view changes, new endpoints, etc.) can be localized in that specific app.
 
 
 In Django, as with most web application frameworks, visiting a certain URL causes a specific section of code to be run. In this example, that would be the code in the
 <code>
 issues
 </code>
 ,
 <code>
 pull_requests
 </code>
 or
 <code>
 ci_actions
 </code>
 apps.
 
 
 One of the best things about apps is that they can be used between
 
 different
 
 projects. It's this
 <a href="https://docs.djangoproject.com/en/5.1/intro/reusable-apps/#reusability-matters">
 reusability
 </a>
 that makes Django apps especially valuable. In many ways, a Django app can be seen as Django's analogy to Python packages. And they serve a very similar role: you don't have to re-invent something that already exists. While most 3rd party Django apps get downloaded from
 <a href="https://pypi.org">
 PyPI
 </a>
 like other Python packages, you can search for a Django app to fit your needs there or at
 <a href="https://djangopackages.org">
 Django Packages
 </a>
 .
 
 <h2>
 Building Your Own Django App
 </h2>
 <h3>
 Developer-Facing Benefits
 </h3>
 
 The benefits of using a third-party app are clear: you get new functionality added to your project. And you don't have to write/test/debug new code. You usually only have to make a few configuration changes.
 
 
 But when it comes to the code you're writing yourself, some developers may wonder why they should bother creating separate apps inside their projects. Won't that just add extra complexity? If not, then what are the benefits?
 
 <h3>
 Code Modularity
 </h3>
 
 By having code that performs a specific set of functions separated into an app, it's not mixed in with your larger project monolith. As a result, you end up with code that's easier to test, document, and update. Instead of having to dig through your entire code base, you can focus on JUST the parts you need to look at. You may also improve the effectiveness of your testing by realizing that some subsystems (apps) need more or less testing than others. You can tune your coverage strategy on a per-app basis. This modularity can also provide clearer internal imports due to app namespacing.
 
 <h3>
 Reusability
 </h3>
 
 You may find yourself writing code that is nearly identical across multiple projects. Instead of copying/pasting that code several times and hoping you've made the right changes, you can install your app into that project. This way, any changes that need to be made to that app can be made in one place instead of several. This might not be one you've considered, given that we don't always know what the full design will be as we're working on something, and it can take time to tell if a certain chunk of code will be useful in other projects.
 
 
 On the topic of reusability, this doesn't mean you have to share this app with the world. You can put it on Github and
 <code>
 pip install
 </code>
 it
 <a href="https://adamj.eu/tech/2019/03/11/pip-install-from-a-git-repository/">
 from a Git repository
 </a>
 . That way, it's easily available to all your other projects and other developers on your team.
 
 <h3>
 A Microservices Analogy
 </h3>
 
 Here's another way of looking at the relationship between a Django project and an app. We can make a comparison to the monolithic and microservice approaches to software architecture. An app in Django can be compared to a microservice but with MUCH less overhead. But an app and a microservice fill similar roles. They help deal with complexity by decomposing your solution into smaller, more manageable pieces. So, if you're working on a Django project and thinking you need a microservice, that's probably a sign that you should convert part of your monolith into an app. And, if at some point you realize you truly DO need a microservice, it's much easier to convert an app into a microservice. You already have the encapsulated structure in place. In short, breaking the codebase of a monolithic Django project into apps
 
 (where appropriate, not just for the sake of doing it)
 
 can provide many of the benefits of distributed systems without most of the problems.
 
 
 There may be a question about how much "overhead" there is in turning your working code into a reusable app? The answer is, "not much". I won't go into detail here, but the Django docs have you covered from
 <a href="https://docs.djangoproject.com/en/5.1/intro/tutorial01/">
 start
 </a>
 to
 <a href="https://docs.djangoproject.com/en/5.1/intro/reusable-apps/">
 finish
 </a>
 .
 
 <h2>
 End-User Facing Benefits
 </h2>
 
 The assumption here is you are building a Django project for other people to use, not just a personal project. Perhaps it's a client you're doing freelance or agency work for. Or, you may be a developer inside a larger organization, building web apps for your co-workers to use. In situations like these, there may be concerns about "adding complexity" or "slowing down development" by creating separate apps within a larger project. As noted above, that's not really a problem, but here are a few things to keep in mind if a concern like that is raised.
 
 
 There are two main benefits users of your Django project will see from creating separate apps within the project. First, all users will benefit from better naming conventions and clarity about which part of the web app they're using. Most won't care about the implementation details, but knowing that they can go directly to
 <code>
 https://CompanyName/task_1/
 </code>
 or
 <code>
 https://CompanyName/task_2/
 </code>
 to do what they need to will simplifiy things for them. Contrast this with needing to go to
 <code>
 https://CompanyName/all_the_tasks/
 </code>
 and then search for what they need.
 
 
 Second, there may be a group of staff or superusers who will have access to the Django Admin. These may be people such as managers, department heads, and the like. For those users, having your code separated into apps will create automatic separation in the Django admin. This will make it easier for those folks to find and focus on the specific app they need, instead of sorting through the entire project.
 
 <h2>
 Next: Django App Of The
 <code>
 {{ variable_time_period }}
 </code>
 </h2>
 
 Now that we've set a bit of foundation, we'll be looking at various Django apps and how they can be used. Some will be apps we're currently using on projects. Some will be apps we think are interesting. We probably won't be doing an app every week, but we'll try to keep the cadence regular enough to be compelling! Stay tuned!
 
 </div>
</div>

Introduction
  
  
   We'll be doing a semi-regular series of blog posts about various useful Django apps. However, it seemed like a good idea to lay some groundwork first. That will help us do a few things:
  
  
   
    define
    
     app
    
    along with a few other similar terms, and get some namespace confusion out of the way
   
   
    understand the value, as a
    
     developer
    
    , of creating separate apps in your Django projects
   
   
    understand the value, as a
    
     developer
    
    or
    
     end-user
    
    of a Django project, of using separate apps (third-party or internally-created) in your Django projects
   
  
  
   What is an App? Context matters.
  
  
   If you go to the Django home page, https://www.djangoproject.com, the blurb at the top says:
   
    Django makes it easier to build better web apps more quickly and with less code.
   
   Django is often described as a
   
    web application framework
   
   or something that you use to create web applications (web apps). I've used this description myself. I have also purposefully distinguished between a
   
    web app
   
   and a
   
    website
   
   during a Django tutorial I taught. I've also written a
   
    blog post
   
   on that topic.  So, if Django is used to BUILD web apps, why are we talking about apps IN Django? The context shifts a bit if you're looking at something built with Django from the outside vs. from the inside.
  
  
   A View From The Inside
  
  
   From the outside, the specific nomenclature is much less important. But from the inside, as someone building something with Django, the specific terminology has more meaning. But, it can still be a little confusing.
  
  
   Internally, Django views an entire collection of code, what most would call a "website," as a
   
    project
   
   . This is where the settings and configuration for a specific Django website are contained. Inside that project are 1 or more
   
    apps
   
   .  An app is usually a specific sub-system used within the project. It's a collection of related code that provides specific functionality that's available to the rest of the project.
  
  
   A Github Example
  
  
   We can look at GitHub and how its functionality is split up as an example.
   
    Note
   
   : Github is NOT built with Django. But it's a web application that's known to the people who'll probably be reading this post. So, we'll pretend Github is built in Django for this example.
  
  
   Looking at the
   
    repository for Python
   
   , we can see that
   
    the issue tracker
   
   ,
   
    management of pull requests
   
   , and
   
    tracking CI/CD activity
   
   are all handling different tasks in different parts of the GitHub web application. But those bits of functionality are available to be reused across the web app on each different repository. When I talk about "bits of functionality" from a Django perspective, I'm talking about the database tables, views, other code and endpoints related to each task. So, the tables, views, other code and endpoints needed to manage Github Issues would all live in one app, perhaps called
   
    issues
   
   . It would be the same for the
   
    pull_requests
   
   and
   
    ci_actions
   
   apps. As such, that functionality is available to any other part of the larger Github project. At the same time, any changes that needed to be made to one of those apps (model updates, view changes, new endpoints, etc.) can be localized in that specific app.
  
  
   In Django, as with most web application frameworks, visiting a certain URL causes a specific section of code to be run. In this example, that would be the code in the
   
    issues
   
   ,
   
    pull_requests
   
   or
   
    ci_actions
   
   apps.
  
  
   One of the best things about apps is that they can be used between
   
    different
   
   projects. It's this
   
    reusability
   
   that makes Django apps especially valuable. In many ways, a Django app can be seen as Django's analogy to Python packages. And they serve a very similar role: you don't have to re-invent something that already exists. While most 3rd party Django apps get downloaded from
   
    PyPI
   
   like other Python packages, you can search for a Django app to fit your needs there or at
   
    Django Packages
   
   .
  
  
   Building Your Own Django App
  
  
   Developer-Facing Benefits
  
  
   The benefits of using a third-party app are clear: you get new functionality added to your project. And you don't have to write/test/debug new code. You usually only have to make a few configuration changes.
  
  
   But when it comes to the code you're writing yourself, some developers may wonder why they should bother creating separate apps inside their projects. Won't that just add extra complexity? If not, then what are the benefits?
  
  
   Code Modularity
  
  
   By having code that performs a specific set of functions separated into an app, it's not mixed in with your larger project monolith. As a result, you end up with code that's easier to test, document, and update. Instead of having to dig through your entire code base, you can focus on JUST the parts you need to look at. You may also improve the effectiveness of your testing by realizing that some subsystems (apps) need more or less testing than others. You can tune your coverage strategy on a per-app basis. This modularity can also provide clearer internal imports due to app namespacing.
  
  
   Reusability
  
  
   You may find yourself writing code that is nearly identical across multiple projects. Instead of copying/pasting that code several times and hoping you've made the right changes, you can install your app into that project. This way, any changes that need to be made to that app can be made in one place instead of several. This might not be one you've considered, given that we don't always know what the full design will be as we're working on something, and it can take time to tell if a certain chunk of code will be useful in other projects.
  
  
   On the topic of reusability, this doesn't mean you have to share this app with the world. You can put it on Github and
   
    pip install
   
   it
   
    from a Git repository
   
   . That way, it's easily available to all your other projects and other developers on your team.
  
  
   A Microservices Analogy
  
  
   Here's another way of looking at the relationship between a Django project and an app. We can make a comparison to the monolithic and microservice approaches to software architecture. An app in Django can be compared to a microservice but with MUCH less overhead. But an app and a microservice fill similar roles. They help deal with complexity by decomposing your solution into smaller, more manageable pieces. So, if you're working on a Django project and thinking you need a microservice, that's probably a sign that you should convert part of your monolith into an app. And, if at some point you realize you truly DO need a microservice, it's much easier to convert an app into a microservice. You already have the encapsulated structure in place. In short, breaking the codebase of a monolithic Django project into apps
   
    (where appropriate, not just for the sake of doing it)
   
   can provide many of the benefits of distributed systems without most of the problems.
  
  
   There may be a question about how much "overhead" there is in turning your working code into a reusable app? The answer is, "not much". I won't go into detail here, but the Django docs  have you covered from
   
    start
   
   to
   
    finish
   
   .
  
  
   End-User Facing Benefits
  
  
   The assumption here is you are building a Django project for other people to use, not just a personal project. Perhaps it's a client you're doing freelance or agency work for. Or, you may be a developer inside a larger organization, building web apps for your co-workers to use. In situations like these, there may be concerns about "adding complexity" or "slowing down development" by creating separate apps within a larger project. As noted above, that's not really a problem, but here are a few things to keep in mind if a concern like that is raised.
  
  
   There are two main benefits users of your Django project will see from creating separate apps within the project. First, all users will benefit from better naming conventions and clarity about which part of the web app they're using. Most won't care about the implementation details, but knowing that they can go directly to
   
    https://CompanyName/task_1/
   
   or
   
    https://CompanyName/task_2/
   
   to do what they need to will simplifiy things for them. Contrast this with needing to go to
   
    https://CompanyName/all_the_tasks/
   
   and then search for what they need.
  
  
   Second, there may be a group of staff or superusers who will have access to the Django Admin. These may be people such as managers, department heads, and the like. For those users, having your code separated into apps will create automatic separation in the Django admin. This will make it easier for those folks to find and focus on the specific app they need, instead of sorting through the entire project.
  
  
   Next: Django App Of The
   
    {{ variable_time_period }}
   
  
  
   Now that we've set a bit of foundation, we'll be looking at various Django apps and how they can be used. Some will be apps we're currently using on projects. Some will be apps we think are interesting. We probably won't be doing an app every week, but we'll try to keep the cadence regular enough to be compelling! Stay tuned!

What IS a Django App?

Step-by-step guide to integrating Meilisearch with Django, complete with automatic indexing, typo tolerance, and relevant filtering capabilities.

# compose.yml 
services:
 search:
 image: getmeili/meilisearch:v1.7
 volumes:
 - ./meili-data:/meili_data
 ports:
 - "7700:7700"

docker-compose up -d search

pip install meilisearch

# settings.py
SEARCH_API_URL = env("SEARCH_API_URL", default="http://search:7700/")
SEARCH_API_KEY = env("SEARCH_API_KEY", default=None)
SEARCH_INDEXES = {
 "main": "main_search",
 # You can define additional indexes here for different purposes
 # "autocomplete": "autocomplete_search",
 # "admin": "admin_search",
}
# Control whether we index the search or not
INDEX_SEARCH = env.bool("INDEX_SEARCH", default=True)

# products/models.py
from django.db import models

class Product(models.Model):
 name = models.CharField(max_length=255)
 description = models.TextField(blank=True)
 price = models.DecimalField(max_digits=10, decimal_places=2)
 sku = models.CharField(max_length=50, unique=True)
 brand = models.CharField(max_length=100, blank=True)
 active = models.BooleanField(default=True)
 created_at = models.DateTimeField(auto_now_add=True)
 updated_at = models.DateTimeField(auto_now=True)

 def get_absolute_url(self):
 return f"/products/{self.id}/"

 def __str__(self):
 return self.name

class Product(models.Model):
 name = models.CharField(max_length=255)
 sku = models.CharField(max_length=50, unique=True)
 description = models.TextField()
 price = models.DecimalField(max_digits=10, decimal_places=2)

class Category(models.Model):
 name = models.CharField(max_length=255)
 slug = models.SlugField(max_length=100)
 description = models.TextField()
 active = models.BooleanField(default=True)

# search/schemas.py
from pydantic import BaseModel

class SearchSchema(BaseModel):
 """
 Pydantic model for our main search schema
 """
 id: str
 title: str | None = None
 type: str # 'product', 'category', etc.
 url: str
 active: bool = True

 # Product fields
 product_name: str | None = None
 product_description: str | None = None
 brand_name: str | None = None
 sku: str | None = None

# products/models.py
from django.db import models
from search.schemas import SearchSchema

class Product(models.Model):
 # fields and methods as before

 def search_data(self):
 """Return data for the search index"""
 return SearchSchema(
 # prefix the ID with the "type" for the SearchSchema
 id=f"product-{self.id}",
 title=self.name,
 type="product",
 url=self.get_absolute_url(),
 active=self.active,
 product_name=self.name,
 product_description=self.description,
 brand_name=self.brand,
 sku=self.sku,
 )

def get_meilisearch_client():
 """Return a meilisearch client"""
 url = settings.SEARCH_API_URL
 api_key = settings.SEARCH_API_KEY
 client = meilisearch.Client(url, api_key)
 return client

def update_object(obj):
 """Update a single object in the search index or delete if inactive"""
 client = get_meilisearch_client()
 index_name = settings.SEARCH_INDEXES["main"]
 index = client.index(index_name)

 if obj["active"]:
 # Update the object in the index
 index.add_documents([obj])
 else:
 # Delete the object from the index
 index.delete_document(obj["id"])

def build_main_indexes(verbose=False):
 """Build the main search index with all searchable models"""
 from products.models import Product # Import your models

 client = get_meilisearch_client()
 index_name = settings.SEARCH_INDEXES["main"]
 index = client.index(index_name)

 # Configure the index settings
 index_settings = {
 "filterableAttributes": ["type", "active"],
 "searchableAttributes": [
 "title",
 "product_name",
 "product_description",
 "sku",
 "brand_name",
 ],
 }
 index.update_settings(index_settings)

 # Index all active products
 products = Product.objects.filter(active=True)
 if verbose:
 print(f"Indexing {products.count()} products...")

 product_docs = [product.search_data().dict() for product in products]
 if product_docs:
 index.add_documents(product_docs)

def rebuild_main_indexes(verbose=False):
 """Completely rebuild the main search index"""
 client = get_meilisearch_client()
 index_name = settings.SEARCH_INDEXES["main"]

 # Delete the index if it exists
 try:
 client.delete_index(index_name)
 except meilisearch.errors.MeiliSearchApiError:
 pass

 # Create the index and build it
 client.create_index(index_name)
 build_main_indexes(verbose=verbose)

# search/mixins.py
from django.conf import settings
from search.utils import update_object

class UpdateSearchMixin:
 """
 Mixin to update the search index when a model is saved
 """
 def update_search(self):
 if settings.INDEX_SEARCH:
 update_object(self.search_data().dict())

 def save(self, *args, **kwargs):
 """ Override the save method so we can call update_search after save """
 # Call the original save method
 super().save(*args, **kwargs)
 # Update the search index
 self.update_search()

# products/models.py
from django.db import models
from search.schemas import SearchSchema
from search.mixins import UpdateSearchMixin

# Add UpdateSearchMixin
class Product(UpdateSearchMixin, models.Model):
 ...

# search/management/commands/index_search.py
from __future__ import annotations

import djclick as click

from search.utils import rebuild_main_indexes


@click.command()
@click.option("--verbose", is_flag=True, default=False)
def main(verbose):
 """Index objects in Meilisearch."""
 click.secho("Rebuilding main indexes...", fg="green", nl=False)
 rebuild_main_indexes(verbose=verbose)
 click.echo("Indexes built.")

# search/management/commands/index_search.py
from __future__ import annotations

import time
import djclick as click
from django.conf import settings
from products.models import Product, Category
from search.utils import get_meilisearch_client

@click.command()
@click.option("--verbose", is_flag=True, default=False)
@click.option("--batch-size", default=1000, help="Batch size for indexing")
def main(verbose, batch_size):
 """Rebuild search indexes in Meilisearch with batch processing."""
 start_time = time.time()
 click.secho("Rebuilding search indexes...", fg="green")

 # Get client and recreate index
 client = get_meilisearch_client()
 index_name = settings.SEARCH_INDEXES["main"]

 # Delete and recreate index
 try:
 client.delete_index(index_name)
 except Exception:
 pass

 client.create_index(index_name)
 index = client.index(index_name)

 # Configure index settings
 click.echo("Configuring index settings...")
 index_settings = {
 "filterableAttributes": ["type", "active"],
 "searchableAttributes": [
 "title", "product_name", "product_description", 
 "sku", "brand_name", "category_name"
 ],
 }
 index.update_settings(index_settings)

 # Index products in batches
 products = Product.objects.filter(active=True)
 total_products = products.count()

 if verbose:
 click.echo(f"Indexing {total_products} products in batches of {batch_size}...")

 # Process in batches to avoid memory issues with large datasets
 batch_count = 0
 for i in range(0, total_products, batch_size):
 batch = products[i:i+batch_size]
 product_docs = [p.search_data().dict() for p in batch]
 if product_docs:
 index.add_documents(product_docs)

 batch_count += 1
 if verbose:
 click.echo(f"Processed batch {batch_count}, {min(i+batch_size, total_products)}/{total_products} products")

 # Index categories (typically smaller, so we do it all at once)
 categories = Category.objects.filter(active=True)
 category_docs = [cat.search_data().dict() for cat in categories]
 if category_docs:
 index.add_documents(category_docs)

 elapsed_time = time.time() - start_time
 click.secho(f"✓ Indexing completed in {elapsed_time:.2f} seconds", fg="green")

# products/models.py
class Category(UpdateSearchMixin, models.Model):
 name = models.CharField(max_length=255)
 description = models.TextField(blank=True)
 active = models.BooleanField(default=True)

 def get_absolute_url(self):
 return f"/categories/{self.id}/"

 def search_data(self):
 return SearchSchema(
 id=f"category-{self.id}",
 title=self.name,
 type="category",
 url=self.get_absolute_url(),
 active=self.active,
 category_name=self.name,
 category_description=self.description,
 products_count=self.products.count(),
 )

# search/schemas.py
class SearchSchema(BaseModel):
 """
 Pydantic model for our main search schema
 """
 # other fields as before... 

 # Category fields
 category_name: str | None = None
 category_description: str | None = None
 products_count: int | None = None

def build_main_indexes(verbose=False):
 """Build the main search index with all searchable models"""
 from products.models import Product, Category # Import your models

 client = get_meilisearch_client()
 index_name = settings.SEARCH_INDEXES["main"]
 index = client.index(index_name)

 # Configure the index settings
 index_settings = {
 "filterableAttributes": ["type", "active"],
 "searchableAttributes": [
 ...
 "category_name",
 "category_description",
 ],
 }
 index.update_settings(index_settings)

 # Index all active products as before... 

 # Index all active categories
 categories = Category.objects.filter(active=True)
 if verbose:
 print(f"Indexing {categories.count()} categories...")

 category_docs = [cat.search_data().dict() for cat in categories]
 if category_docs:
 index.add_documents(category_docs)

# search/views.py
from django.shortcuts import render
from django.conf import settings
from search.utils import get_meilisearch_client

def search(request):
 """Search view"""
 query = request.GET.get("q", "").strip()
 type_filter = request.GET.get("type", None)
 results = []

 if query:
 client = get_meilisearch_client()
 index = client.index(settings.SEARCH_INDEXES["main"])

 # Build filter string using your filterable attributes 
 filter_str = "active = true"
 if type_filter:
 filter_str += f" AND type = {type_filter}"

 # Perform search
 search_results = index.search(
 query, 
 {
 "filter": filter_str,
 "limit": 50
 }
 )
 results = search_results["hits"]

 return render(
 request,
 "search/results.html",
 {
 "results": results,
 "query": query,
 "type_filter": type_filter
 },
 )


{% extends "base.html" %}

{% block content %}
<div class="search-container">
 
 <form method="get" action="{% url 'search' %}">
 <input type="text" name="q" value="{{ query }}" placeholder="Search...">
 <button type="submit">Search</button>
 </form>

 
 {% if query %}
 <h2>Results for "{{ query }}"</h2>
 {% if results %}
 <div class="search-results">
 {% for result in results %}
 <div class="search-result">
 <h3><a href="{{ result.url }}">{{ result.title }}</a></h3>
 </div>
 {% endfor %}
 </div>
 {% else %}
 No results found.
 {% endif %}
 {% endif %}
</div>
{% endblock %}

# urls.py
from django.urls import path
from search.views import search

urlpatterns = [
 # ... other URLs
 path('search/', search, name='search'),
]

 
 
 For a production application, you might want to enhance this with:
 
 
 
 
 Pagination
 
 : For handling large result sets
 
 
 
 Highlighting
 
 : To show users where their search terms matched
 
 
 
 Faceted search
 
 : To allow filtering by multiple attributes
 
 
 
 Autocomplete
 
 : To suggest search terms as users type
 
 
 
 Analytics
 
 : To track popular searches and improve your search experience
 
 
 
 Meilisearch supports all these features through its API, making it easy to build a sophisticated search experience as your application grows.
 
 
 See It In Action: Surplus Sales Case Study
 
 
 Want to see what a production Meilisearch implementation looks like? Visit our client
 
 Surplus Sales
 
 and try out their search functionality. Pay special attention to:
 
 
 
 
 Typo tolerance
 
 : Try searching for "trnasistors" (misspelled) and notice how it still finds what you need
 
 
 
 Speed
 
 : The millisecond response times, even when filtering through their 70,000+ item catalog
 
 
 
 Relevance
 
 : How the most relevant parts appear at the top of results
 
 
 
 Complex metadata handling
 
 : Search by part numbers, specifications, or descriptions
 
 
 
 Their implementation handles all the concepts we've covered in this tutorial. It's a great example of how these techniques scale to real-world applications.
 
 
 Alternative Approach: Using django-meili
 
 
 If you prefer a more Django-native approach with less custom code, you might want to check out
 
 django-meili
 
 . This package provides a more Haystack-like experience for integrating Meilisearch with Django. I haven't used it, but from what I can tell, these are the main differences between using django-meili and implementing something more like our custom Meilisearch implementation:
 
 
 
 
 Model-focused
 
 : django-meili is designed around indexing individual Django models, which works well if you're searching just one model at a time
 
 
 
 Less flexible
 
 : Our unified schema approach gives you more control over how different models are represented in search results
 
 
 
 Easier setup
 
 : django-meili requires less boilerplate code to get started
 
 
 
 Django-native
 
 : It follows Django conventions more closely with a familiar API
 
 
 
 django-meili might be a good choice if:
 
 
 
 You're primarily searching within a single model type
 
 
 You prefer a more Django-integrated approach with less custom code
 
 
 You don't need the flexibility of a unified schema across different model types
 
 
 
 Further Reading
 
 
 
 
 Meilisearch Official Documentation
 
 
 
 
 Meilisearch Python Client
 
 
 
 
 Meilisearch Settings Reference
 
 
 
 
 Meilisearch Search Parameters
 
 
 
 
 django-meili GitHub Repository
 
 
 
 
 Need help implementing advanced search for your Django project?
 
 Contact our team at Revsys
 
 for expert
 
 Django consulting
 
 . We've implemented search solutions for clients ranging from small startups to large enterprises, and we'd be happy to help you build a search experience that delights your users.

How to Add Blazing Fast Search to Your Django Site with Meilisearch

Test output is often an afterthought. You can improve your flow and slightly gamify your bug hunting by combining the wonderful Rich Python library and pytest.

<div class="w-block-markdown block-markdown">
 <div class="prose prose-lg max-w-none">
 <h2>
 Debugging complex code is not usually fun
 </h2>
 
 It's hard to get motivated when faced with a big pile of numbers that just aren't lining up in the ways they are supposed to. Stupid numbers, do what you're told!
 
 <h2>
 The task at hand
 </h2>
 
 I've been rebuilding a 15 to 20-year-old ColdFusion currency trading learning simulation for a client. The math involved is mostly contained in SQL Server stored procedures, some values in tables are really constants (or at least most of the time), and variable names between steps of the process aren't consistent. This has made the rebuilding effort harder than it originally appeared. Add in finding some undesired behaviors in the old app and the desire to improve these situations going forward has been a bit of a slog actually.
 
 
 I knew that today was the day I had to just dig in and iterate on this code until all of the numbers were correct now that I had a known good set of values to work against.
 
 
 Since I needed to compare several numbers together, I thought I might use a
 <a href="https://rich.readthedocs.io/en/stable/tables.html">
 Rich Table
 </a>
 to make the output a little better organized. And I'm VERY glad I did.
 
 
 Here is what I started with:
 
 </div>
</div>
<div class="w-block-content block-content">
 
 
 <img alt="Initial boring pytest output" class="richtext-image full-width" height="742" src="https://sfo2.digitaloceanspaces.com/revsys-com-prod-media/images/CleanShot_2024-02-03_at_08.18.27.width-800.png" width="800"/>
 
 
</div>
<div class="w-block-markdown block-markdown">
 <div class="prose prose-lg max-w-none">
 
 Not exactly pretty. It spit out the numbers it was calculating and I could compare them against the known good values visually. Looking and re-re-reading the numbers a half dozen times I was already bored and feeling tired despite having too much caffeine in my system.
 
 
 So I implemented all of the expected, calculated and differences between the two in a Rich Table. I took a couple of moments to adjust some colors and right justifying all of these numbers because I knew I was going to be staring at this
 
 all freaking day
 
 .
 
 </div>
</div>
<div class="w-block-content block-content">
 
 
 <img alt="Initial Python Rich table formatting" class="richtext-image full-width" height="475" src="https://sfo2.digitaloceanspaces.com/revsys-com-prod-media/images/CleanShot_2024-02-03_at_09.15.54.width-800.png" width="800"/>
 
 
</div>
<div class="w-block-markdown block-markdown">
 <div class="prose prose-lg max-w-none">
 <h2>
 The debugging
 </h2>
 
 I then went about debugging the first
 
 Total Unhedged
 
 number because all of the following calculations depended on this number being correct. And I quickly found the bug!
 
 
 I would love to say it was because of this great new visualization, but it was just closely re-reading my Python code and comparing it (
 
 yet again
 
 ) to the original portion of a stored procedure to find my subtle mistake.
 
 
 One down, 6 more to go I thought. This is going to be a long day, but it turned out two more of the values were
 
 NEARLY
 
 correct now, just off due to rounding. This
 
 WAS
 
 easier to spot because of the tabular layout. I treated myself to a bit more table improvement, I wrote a little function named
 <code>
 calc_diff()
 </code>
 that took the expected and actual values and color coded the difference red if there was a difference and green if not.
 
 </div>
</div>
<div class="w-block-content block-content">
 
 
 <img alt="Improved table with red/green color coding" class="richtext-image full-width" height="341" src="https://sfo2.digitaloceanspaces.com/revsys-com-prod-media/images/CleanShot_2024-02-03_at_09.36.37.width-800.png" width="800"/>
 
 
</div>
<div class="w-block-markdown block-markdown">
 <div class="prose prose-lg max-w-none">
 <h2>
 Table code example
 </h2>
 
 The code for this is surprisingly easy.
 <a href="https://github.com/Textualize/rich">
 Rich
 </a>
 was already a dependency of something else in my project's virtualenv so I didn't even need to install it. Here is a simplified example of the code for you:
 
 </div>
</div>
<div class="w-block-code block-code">
 <div class="codehilite">
 <pre>from decimal import Decimal

from rich.console import Console
from rich.table import Table
from rich.text import Text 

def calc_diff(expected, actual):
 diff = expected - actual

 color = "red"
 if diff == 0:
 color = "green"

 text = Text(str(diff))
 text.stylize(color)

 return text

# My actual pytest test
def test_ex3_math(ex3_params):
 expected_total_unhedged = Decimal("3.394148")

 # call a method on a fixture class that does the math calculation for us, yours
 # would obviously be different 
 total_unhedged = ex3_params.total_unhedged()

 # Build our table
 table = Table(
 title="Exercise 3 Answers", title_style="gold1", style="deep_sky_blue3"
 )
 table.add_column(
 "Name", style="turquoise2", header_style="turquoise2", no_wrap=True
 )
 table.add_column("Expected", header_style="turquoise2", justify="right")
 table.add_column("Actual", header_style="turquoise2", justify="right")
 table.add_column("Difference", header_style="turquoise2", justify="right")

 # Total Unhedged - just showing you one row here for sake of brevity
 table.add_row(
 "Total Unhedged",
 str(expected_total_unhedged),
 str(total_unhedged),
 calc_diff(expected_total_unhedged, total_unhedged),
 )

 # Output the table 
 console = Console(color_system="256")
 console.print(table)

 # Trigger our test to fail artificially 
 assert False
</pre>
 </div>
</div>
<div class="w-block-markdown block-markdown">
 <div class="prose prose-lg max-w-none">
 <h2>
 Conclusion
 </h2>
 
 Was it worth the extra time?
 
 DEFINITELY!
 
 I not only made it easier to compare the values as I worked but also motivated me a little more than usual as I turned each red line green.
 
 
 Every little bit helps, right? Hopefully, you enjoyed this protip and can put it use in one of your upcoming debugging sessions.
 
 
 If you or your company struggles with testing your Python or Django apps, check out
 <a href="/teststart/">
 TestStart
 </a>
 our testing consulting package to help jump start your team into better productivity.
 
 </div>
</div>

from decimal import Decimal

from rich.console import Console
from rich.table import Table
from rich.text import Text 

def calc_diff(expected, actual):
        diff = expected - actual

        color = "red"
        if diff == 0:
            color = "green"

        text = Text(str(diff))
        text.stylize(color)

        return text

# My actual pytest test
def test_ex3_math(ex3_params):
    expected_total_unhedged = Decimal("3.394148")

    # call a method on a fixture class that does the math calculation for us, yours
   # would obviously be different 
    total_unhedged = ex3_params.total_unhedged()

    # Build our table
    table = Table(
        title="Exercise 3 Answers", title_style="gold1", style="deep_sky_blue3"
    )
    table.add_column(
        "Name", style="turquoise2", header_style="turquoise2", no_wrap=True
    )
    table.add_column("Expected", header_style="turquoise2", justify="right")
    table.add_column("Actual", header_style="turquoise2", justify="right")
    table.add_column("Difference", header_style="turquoise2", justify="right")

    # Total Unhedged - just showing you one row here for sake of brevity
    table.add_row(
        "Total Unhedged",
        str(expected_total_unhedged),
        str(total_unhedged),
        calc_diff(expected_total_unhedged, total_unhedged),
    )

   # Output the table 
   console = Console(color_system="256")
   console.print(table)

   # Trigger our test to fail artificially 
   assert False

 


 
  
   Conclusion
  
  
   Was it worth the extra time?
   
    DEFINITELY!
   
   I not only made it easier to compare the values as I worked but also motivated me a little more than usual as I turned each red line green.
  
  
   Every little bit helps, right? Hopefully, you enjoyed this protip and can put it use in one of your upcoming debugging sessions.
  
  
   If you or your company struggles with testing your Python or Django apps, check out
   
    TestStart
   
   our testing consulting package to help jump start your team into better productivity.

"En-Rich" your Python testing

A simple import trick to make it easy to keep your django-ninja view functions in different Django apps and files.

<div class="w-block-markdown block-markdown">
 <div class="prose prose-lg max-w-none">
 
 I've been evaluating
 <a href="https://django-ninja.rest-framework.com/">
 django-ninja
 </a>
 recently, using it in a personal project to see when it might be appropriate for
our clients.
 
 <h2>
 The Problem - How to have multiple view files
 </h2>
 
 One thing that was not clear was how to break up your API views and sprinkle
them around in multiple files. The default tutorial and documentation is not
bad, but just does not cover this small thing.
 
 
 The way django-ninja works is that you decorate functional views with
 <code>
 @api
 </code>
 where you indicate what HTTP method it is for and some other parameters.
Everything is perfect until you realize your views.py has grown huge or you
want to include api views from several Django applications.
 
 
 For those of us who are very familiar with DRF, it is not immediately clear how
you can break your views into multiple files.
 
 
 Let's use an example of having a Django apps named
 <code>
 core
 </code>
 ,
 <code>
 users
 </code>
 , and
 <code>
 posts
 </code>
 to mimic some sort of blog like software and our project directory, which contains
our settings and urls files will be named
 <code>
 config/
 </code>
 which is the standard way
we roll at REVSYS.
 
 
 Your first guess might be that you should create another instance of
 <code>
 NinjaAPI
 </code>
 and use that to decorate your views in each file. Unfortunately, because of how
the internal registry works for these, that would create multiple django-ninja
"sites", each with their own docs, etc. which is obviously not what we want.
 
 <h2>
 The Solution
 </h2>
 
 The easiest way to do this is to pass around your API decorator and ensure
your view files are imported before you call on Ninja to generate your URLs.
 
 
 While I hate having an app named
 <code>
 core
 </code>
 , I used it in this case. So we
create our API instance with the following:
 
 <div class="codehilite">
 <pre><code># File: core/api.py
from ninja import NinjaAPI

api = NinjaAPI(csrf=True)
</code></pre>
 </div>
 
 And then in our views we import this and use it:
 
 <div class="codehilite">
 <pre><code># File: users/views.py
from core.api import api

@api.get("users/something/"):
def get_user(request):
 # ... do something interesting here ...

# File: posts/views.py
from core.api import api

@api.get("posts/{username}/"):
def get_posts(requests, username: str):
 # ... do something interesting here ...
</code></pre>
 </div>
 
 Then the final bit is to make sure we import these files in our
 <code>
 urls.py
 </code>
 before
we call
 <code>
 api.urls
 </code>
 like this:
 
 <div class="codehilite">
 <pre><code># File: config/urls.py
import users.views
import posts.views
from core.api import api

urlpatterns = [
 path("api/", api.urls),
]
</code></pre>
 </div>
 <h2>
 Alternative Solution
 </h2>
 
 Turns out there is also a DRF like concept of
 <a href="https://django-ninja.rest-framework.com/guides/routers/">
 routers
 </a>
 which is honestly probably a better more
maintainable solution than the above.
 
 
 Thanks to
 <a href="https://twitter.com/uninen/status/1622007355067056132?s=46&amp;t=6Jl-2SgFBle7KLjsma1uQw">
 Ville S&auml;&auml;vuori
 </a>
 for point this out after I posted this.
 
 
 Hopefully, this saves you a bit of time when you first start using django-ninja!
 
 
 If you need help creating Django REST APIs don't hesitate to
 <a href="/contact/">
 reach out to us
 </a>
 !
 
 </div>
</div>

# File: core/api.py
from ninja import NinjaAPI

api = NinjaAPI(csrf=True)

# File: users/views.py
from core.api import api

@api.get("users/something/"):
def get_user(request):
   # ... do something interesting here ...

# File: posts/views.py
from core.api import api

@api.get("posts/{username}/"):
def get_posts(requests, username: str):
   # ... do something interesting here ...

# File: config/urls.py
import users.views
import posts.views
from core.api import api

urlpatterns = [
    path("api/", api.urls),
]

  
  
   Alternative Solution
  
  
   Turns out there is also a DRF like concept of
   
    routers
   
   which is honestly probably a better more
maintainable solution than the above.
  
  
   Thanks to
   
    Ville Säävuori
   
   for point this out after I posted this.
  
  
   Hopefully, this saves you a bit of time when you first start using django-ninja!
  
  
   If you need help creating Django REST APIs don't hesitate to
   
    reach out to us
   
   !

Breaking up django-ninja views

Sometimes you want your FloatFields to be nullable, but your JS code isn&#x27;t quite smart enough to handle it.

<div class="w-block-content block-content">
 
 I run into this situation every couple of years, and I always forget the solution. Even asked around the office and everyone had the same "Oh yeah, I remember doing that once... but now how".
 
 
 When these situations happen to us, it is often time to write a blog post!
 
 
 The situation I was faced with was I had:
 
 <ul>
 <li data-block-key="7lj4t">
 A model with a few nullable Django Model FloatFields
 </li>
 <li data-block-key="b2ksr">
 Django Rest Framework, using a ModelSerializer
 </li>
 <li data-block-key="rfo1">
 react-hook-form, react-query and standard fetch calls to POST the data to the endpoint
 </li>
 </ul>
 
 The issue is that we have a simple HTML text input to collect a possible float value. It's easy enough to mark this as not required on the form and any client-side form validation. However, it is NOT always that easy to handle what happens next. Because it's a text field, the JS tooling wants to send it as an empty string. So we end up with a payload that looks like:
 
</div>
<div class="w-block-code block-code">
 <div class="codehilite">
 <pre>{
 "name": "Frank Wiles",
 "weight": "",
}
</pre>
 </div>
</div>
<div class="w-block-content block-content">
 
 When what we'd really like to have is:
 
</div>
<div class="w-block-code block-code">
 <div class="codehilite">
 <pre>{
 "name": "Frank Wiles",
 "weight": null,
}
</pre>
 </div>
</div>
<div class="w-block-content block-content">
 
 DRF rejects this saying that an empty string is not a number. It is correct of course, just not being super helpful in this exact situation. So what is the solution? Enter BlankableFloatField
 
</div>
<div class="w-block-code block-code">
 <div class="codehilite">
 <pre>from rest_framework.serializers import FloatField


class BlankableFloatField(FloatField):
 """
 This accepts an empty string as None
 """

 def to_internal_value(self, data):
 if data == "":
 return None
 return super().to_internal_value(data)
</pre>
 </div>
</div>
<div class="w-block-content block-content">
 
 You can override this on your ModelSerializer by just specifying the fields manually like this:
 
</div>
<div class="w-block-code block-code">
 <div class="codehilite">
 <pre>class PersonSerializer(serializers.ModelSerializer):
 # ... your other fields here ... 
 weight = BlankableFloatField(required=False, allow_null=True, default=None)
</pre>
 </div>
</div>
<div class="w-block-content block-content">
 
 The solution isn't hard to execute, but it's hard to reason about and figure out the
 
 best
 
 place to adjust this in your Django project.
 
 
 Hopefully, the next time you find yourself with Javascript error in your console saying that an empty string is not a number this helps you out!
 
</div>

{
  "name": "Frank Wiles",
  "weight": "",
}

 
{
  "name": "Frank Wiles",
  "weight": null,
}

 
from rest_framework.serializers import FloatField


class BlankableFloatField(FloatField):
    """
    This accepts an empty string as None
    """

    def to_internal_value(self, data):
        if data == "":
            return None
        return super().to_internal_value(data)

 
class PersonSerializer(serializers.ModelSerializer):
    # ... your other fields here ... 
    weight = BlankableFloatField(required=False, allow_null=True, default=None)

 


 
  The solution isn't hard to execute, but it's hard to reason about and figure out the
  
   best
  
  place to adjust this in your Django project.
 
 
  Hopefully, the next time you find yourself with Javascript error in your console saying that an empty string is not a number this helps you out!

Sometimes you want your FloatFields to be nullable, but your JS code isn't quite smart enough to handle it.

Nullable DRF FloatFields and Javascript

I needed to structure permissions in an API view that had multiple user roles, and I learned about using the &amp; (and), | (or) and ~ (not) operators with Django REST Framework permissions.

<div class="w-block-markdown block-markdown">
 <div class="prose prose-lg max-w-none">
 
 Recently, I needed to structure permissions in an API view so that someone could access an API if they were a superuser or they had some other status, like being a manager. The
 <a href="https://www.django-rest-framework.org/api-guide/permissions/#how-permissions-are-determined">
 Django Rest Framework Permissions docs
 </a>
 say that &ldquo;If any permission check fails an exceptions.PermissionDenied or exceptions.NotAuthenticated exception will be raised,&rdquo; which made me think I&rsquo;d have to override the
 <a href="http://www.cdrf.co/3.9/rest_framework.views/APIView.html">
 <code>
 get_permissions()
 </code>
 </a>
 method of the view or write a series of complicated
 <code>
 IsUserTypeA
 </code>
 ,
 <code>
 IsUserTypeAOrUserTypeB
 </code>
 permissions classes, in order to do what I wanted. That didn&rsquo;t seem ideal.
 
 
 Then a
 <a href="https://twitter.com/webology">
 colleague
 </a>
 pointed out one line in the DRF docs further down on the
 <a href="https://www.django-rest-framework.org/api-guide/permissions/#setting-the-permission-policy">
 same page
 </a>
 : &ldquo;Note: it supports &amp; (and), | (or) and ~ (not).&rdquo; This was exactly what I needed.
 
 
 Here&rsquo;s how it works.
 
 <h2>
 All permissions must return True
 </h2>
 
 The expected way that DRF permissions work is as stated above; when you list multiple permission classes in
 <code>
 permission_classes
 </code>
 , each of them must return True in order for the user to have access to that view. This is most likely what you&rsquo;re used to when implementing DRF permissions.
 
 <div class="codehilite">
 <pre><code>from rest_framework.permissions import IsAuthenticated, IsAdminUser, ReadOnly
from rest_framework.viewsets import ModelViewSet

from my_app.permissions import IsStudent, IsFaculty


class MyModelViewSet(ModelViewSet):
 permission_classes = (IsAuthenticated, IsAdminUser)
</code></pre>
 </div>
 
 In this case, the user must be authenticated
 
 and
 
 be an admin user, or else they cannot access any part of the view.
 
 <h2>
 The | (or) operator
 </h2>
 
 Using the
 <code>
 |
 </code>
 (or) operator in DRF permissions allows you to set multiple conditions the user might meet and be able to access this view. For instance, for an educational app, a view might be available to students
 
 or
 
 faculty.
 
 <div class="codehilite">
 <pre><code>from rest_framework.permissions import IsAuthenticated, IsAdminUser, ReadOnly
from rest_framework.viewsets import ModelViewSet

from my_app.permissions import IsStudent, IsFaculty


class MyModelViewSet(ModelViewSet):
 permission_classes = (IsAdminUser | IsStudent | IsFaculty)
</code></pre>
 </div>
 
 This expression
 <code>
 (IsAdminUser | IsStudent | IsFaculty)
 </code>
 allows a user who is an admin, a student, or a faculty member to access the view. Instead of needing to pass all three of these conditions, a user only needs to pass one.
 
 
 The way I accomplished this before was a custom permissions class named something like
 <code>
 IsAdminStudentOrFaculty
 </code>
 , but with multiple roles and complex permissions, this meant creating custom permission classes for nearly every combination of user role. (In this example, I refer to custom permissions
 <code>
 IsStudent
 </code>
 and
 <code>
 IsFaculty
 </code>
 ; there are
 <a href="https://www.django-rest-framework.org/api-guide/permissions/#examples">
 some examples in the docs
 </a>
 of how to implement custom permissions like these.)
 
 <h2>
 The &amp; (and) operator
 </h2>
 
 Especially combined with the
 <code>
 |
 </code>
 (or) operator, the
 <code>
 &amp;
 </code>
 (and) operator is really powerful. It enables you to set clear permissions boundaries without writing increasingly complex custom permissions classes.
 
 <div class="codehilite">
 <pre><code>from rest_framework.permissions import IsAuthenticated, IsAdminUser, ReadOnly
from rest_framework.viewsets import ModelViewSet

from my_app.permissions import IsStudent, IsFaculty


class MyModelViewSet(ModelViewSet):
 permission_classes = (IsAuthenticated &amp; (IsAdminUser | IsFaculty | ReadOnly))
</code></pre>
 </div>
 
 This example,
 <code>
 (IsAuthenticated &amp; (IsAdminUser | IsFaculty | ReadOnly))
 </code>
 , requires that all users accessing this view be authenticated. In addition, it sets other permissions: a user must be authenticated
 
 and
 
 be an admin or a faculty member, or they only have &ldquo;read&rdquo; access to this view.
 
 <h2>
 The
 <code>
 ~
 </code>
 (not) operator
 </h2>
 
 The
 <code>
 ~
 </code>
 (not) operator allows you to simplify your permissions to permit everyone
 
 except
 
 users who meet specific criteria. Personally, I prefer to set permissions with positive language, like &ldquo;allow admins and faculty&rdquo; rather than &ldquo;don&rsquo;t allow students,&rdquo; but there are probably cases where the
 <code>
 ~
 </code>
 operator is a great option.
 
 <div class="codehilite">
 <pre><code>from rest_framework.permissions import IsAuthenticated, IsAdminUser, ReadOnly
from rest_framework.viewsets import ModelViewSet

from my_app.permissions import IsStudent, IsFaculty


class MyModelViewSet(ModelViewSet):
 permission_classes = (~IsStudent &amp; IsAuthenticated)
</code></pre>
 </div>
 
 <code>
 (~IsStudent &amp; IsAuthenticated)
 </code>
 says that anyone who is authenticated and who is
 
 not
 
 a student can access this view.
 
 
 I hope you, like me, now can free yourself from the cumbersome task of writing too many custom permission classes in your apps with several different types of user roles and instead make use of simpler permission classes and the
 <code>
 &amp;
 </code>
 ,
 <code>
 |
 </code>
 and
 <code>
 ~
 </code>
 operators!
 
 <h2>
 Further Reading
 </h2>
 <ul>
 <li>
 <a href="https://www.django-rest-framework.org/api-guide/permissions">
 Django REST Framework Permissions
 </a>
 docs
 </li>
 <li>
 The
 <a href="https://github.com/encode/django-rest-framework/pull/5753">
 pull request
 </a>
 that added the
 <code>
 &amp;
 </code>
 (and) and
 <code>
 |
 </code>
 (or) operators. This, and the next pull request, are also the inspiration for some of my examples.
 </li>
 <li>
 The
 <a href="https://github.com/encode/django-rest-framework/pull/6361">
 pull request
 </a>
 that added the
 <code>
 ~
 </code>
 (not) operator
 </li>
 </ul>
 
 
 Thanks to
 <a href="https://twitter.com/webology">
 Jeff Triplett
 </a>
 for reviewing a draft of this article and finding this tip in the docs in the first place!
 
 
 </div>
</div>

from rest_framework.permissions import IsAuthenticated, IsAdminUser, ReadOnly
from rest_framework.viewsets import ModelViewSet

from my_app.permissions import IsStudent, IsFaculty


class MyModelViewSet(ModelViewSet):
    permission_classes = (IsAuthenticated, IsAdminUser)

from rest_framework.permissions import IsAuthenticated, IsAdminUser, ReadOnly
from rest_framework.viewsets import ModelViewSet

from my_app.permissions import IsStudent, IsFaculty


class MyModelViewSet(ModelViewSet):
    permission_classes = (IsAdminUser | IsStudent | IsFaculty)

from rest_framework.permissions import IsAuthenticated, IsAdminUser, ReadOnly
from rest_framework.viewsets import ModelViewSet

from my_app.permissions import IsStudent, IsFaculty


class MyModelViewSet(ModelViewSet):
    permission_classes = (IsAuthenticated & (IsAdminUser | IsFaculty | ReadOnly))

from rest_framework.permissions import IsAuthenticated, IsAdminUser, ReadOnly
from rest_framework.viewsets import ModelViewSet

from my_app.permissions import IsStudent, IsFaculty


class MyModelViewSet(ModelViewSet):
    permission_classes = (~IsStudent & IsAuthenticated)

  
  
   
    (~IsStudent & IsAuthenticated)
   
   says that anyone who is authenticated and who is
   
    not
   
   a student can access this view.
  
  
   I hope you, like me, now can free yourself from the cumbersome task of writing too many custom permission classes in your apps with several different types of user roles and instead make use of simpler permission classes and the
   
    &
   
   ,
   
    |
   
   and
   
    ~
   
   operators!
  
  
   Further Reading
  
  
   
    
     Django REST Framework Permissions
    
    docs
   
   
    The
    
     pull request
    
    that added the
    
     &
    
    (and) and
    
     |
    
    (or) operators. This, and the next pull request, are also the inspiration for some of my examples.
   
   
    The
    
     pull request
    
    that added the
    
     ~
    
    (not) operator
   
  
  
   
    Thanks to
    
     Jeff Triplett
    
    for reviewing a draft of this article and finding this tip in the docs in the first place!

I needed to structure permissions in an API view that had multiple user roles, and I learned about using the & (and), | (or) and ~ (not) operators with Django REST Framework permissions.

A Tip About DRF Permissions

I was working on a project with a coworker recently and I noticed in one of their pull requests that they used a custom exception in one of our Django REST Framework viewsets. I prefer this way to what I was doing before, so I wanted to share it with you!

<div class="w-block-markdown block-markdown">
 <div class="prose prose-lg max-w-none">
 
 I was working on a project with a
 <a href="https://twitter.com/fwiles">
 coworker
 </a>
 recently and I noticed in one of their pull requests that they used a custom exception in one of our Django REST Framework viewsets. I prefer this way to what I was doing before, so I wanted to share it with you!
 
 <h2>
 My Less Clean, Less Consistent Exception Handling
 </h2>
 
 I made good use of the
 <a href="https://www.django-rest-framework.org/api-guide/status-codes/">
 status codes
 </a>
 page of the DRF docs. For example, if a user uploaded a CSV file that my view couldn&rsquo;t read, I might have had something like this:
 
 <div class="codehilite">
 <pre><code>from rest_framework import status
from rest_framework.response import Response
from rest_framework.views import APIView


class MyCSVUploadView(APIView):
 def post(self, request, *args, **kwargs):
 &hellip;
 if file_error:
 return Response(
 {"detail": "Unable to read file."}, 
 status=status.HTTP_400_BAD_REQUEST
 )
</code></pre>
 </div>
 
 This works just fine. But in some cases, I might return the same type of error response in multiple places. For a university that is no longer accepting applications, I might have multiple places where I am essentially passing the same &ldquo;Deadline passed&rdquo; message. Although the response is just one line of code, I have to find it, copy it, and paste it to make sure I&rsquo;m using the same wording, or deal with inconsistency in my exception messages.
 
 <h2>
 Custom Exception Classes for Clean Consistency
 </h2>
 
 For exception conditions that you encounter frequently in your code, you can override DRF&rsquo;s
 <code>
 APIException
 </code>
 class and create your own exceptions! Read more in
 <a href="https://www.django-rest-framework.org/api-guide/exceptions/#apiexception">
 the docs
 </a>
 .
 
 
 Import the
 <code>
 APIException
 </code>
 class and create a new class that inherits from it. Then set the
 <code>
 status_code
 </code>
 ,
 <code>
 default_detail
 </code>
 , and
 <code>
 default_code
 </code>
 attributes.
 
 <div class="codehilite">
 <pre><code># exceptions.py
from rest_framework.exceptions import APIException


class UnreadableCSVFile(APIException):
 status_code = 400
 default_detail = "Unable to read file."
 default_code = "unreadable_csv_file"
</code></pre>
 </div>
 
 The
 <code>
 status_code
 </code>
 attribute defines the HTTP status code you want to send in the response with this exception; refer to the
 <a href="https://www.django-rest-framework.org/api-guide/status-codes/">
 DRF list of status codes
 </a>
 for help. The
 <code>
 default_detail
 </code>
 attribute is the human-readable exception message you want to pass in the response. The
 <code>
 default_code
 </code>
 is a string that represents this specific exception.
 
 
 In your view, instead of manually returning a
 <code>
 Response()
 </code>
 object, raise your custom exception.
 
 <div class="codehilite">
 <pre><code># views.py
from rest_framework.views import APIView

From my_app.exceptions import UnreadableCSVFile

class MyCSVUploadView(APIView):
 def post(self, request, *args, **kwargs):
 &hellip;
 if file_error:
 raise UnreadableCSVFile()
</code></pre>
 </div>
 
 This allows my exceptions to be reusable across my app, which means the language I use in exception messages is always consistent.
 
 <h2>
 Custom Exception Handlers
 </h2>
 
 You can further customize your exceptions by creating a custom handler function (
 <a href="https://www.django-rest-framework.org/api-guide/exceptions/#custom-exception-handling">
 docs on custom exception handling
 </a>
 ). This is especially helpful if there are pieces of data, like the status code, that you want to ensure are included in all of your HTTP responses. This is a slightly different goal than what I refer to above, but it&rsquo;s worth knowing about. Here is the example from the docs:
 
 <div class="codehilite">
 <pre><code>from rest_framework.views import exception_handler

def custom_exception_handler(exc, context):
 # Call REST framework's default exception handler first, 
 # to get the standard error response.
 response = exception_handler(exc, context)

 # Now add the HTTP status code to the response.
 if response is not None:
 response.data['status_code'] = response.status_code

 return response
</code></pre>
 </div>
 
 This adds the
 <code>
 status_code
 </code>
 to each exception response sent in your app.
 
 
 You would then add your custom exception handler to the
 <code>
 REST_FRAMEWORK
 </code>
 settings in your
 <code>
 settings.py
 </code>
 so this custom exception handler will be executed every time there is an API exception:
 
 <div class="codehilite">
 <pre><code>REST_FRAMEWORK = {
 'EXCEPTION_HANDLER': 'my_project.my_app.utils.custom_exception_handler'
}
</code></pre>
 </div>
 
 For my example above, with the unreadable CSV file, the response would look like this:
 
 <div class="codehilite">
 <pre><code>{"status_code": 400, "detail": "Unable to read file."}
</code></pre>
 </div>
 
 With the status code returned as part of the response dictionary.
 
 
 
 Thanks to
 <a href="https://twitter.com/fwiles">
 Frank Wiles
 </a>
 for teaching me about custom exceptions, and
 <a href="https://twitter.com/webology">
 Jeff Triplett
 </a>
 for reviewing a draft of this article.
 
 
 </div>
</div>

from rest_framework import status
from rest_framework.response import Response
from rest_framework.views import APIView


class MyCSVUploadView(APIView):
    def post(self, request, *args, **kwargs):
        …
        if file_error:
            return Response(
                {"detail": "Unable to read file."}, 
                status=status.HTTP_400_BAD_REQUEST
            )

# exceptions.py
from rest_framework.exceptions import APIException


class UnreadableCSVFile(APIException):
    status_code = 400
    default_detail = "Unable to read file."
    default_code = "unreadable_csv_file"

# views.py
from rest_framework.views import APIView

From my_app.exceptions import UnreadableCSVFile

class MyCSVUploadView(APIView):
    def post(self, request, *args, **kwargs):
        …
        if file_error:
            raise UnreadableCSVFile()

from rest_framework.views import exception_handler

def custom_exception_handler(exc, context):
    # Call REST framework's default exception handler first, 
    # to get the standard error response.
    response = exception_handler(exc, context)

    # Now add the HTTP status code to the response.
    if response is not None:
        response.data['status_code'] = response.status_code

    return response

REST_FRAMEWORK = {
    'EXCEPTION_HANDLER': 'my_project.my_app.utils.custom_exception_handler'
}

{"status_code": 400, "detail": "Unable to read file."}

  
  
   With the status code returned as part of the response dictionary.
  
  
   
    Thanks to
    
     Frank Wiles
    
    for teaching me about custom exceptions, and
    
     Jeff Triplett
    
    for reviewing a draft of this article.

Custom Exceptions in Django REST Framework

When working with Wagtail, you might find that you&#x27;re using Wagtail Page models for some of your database models, but regular Django models for others. In this post, learn how to add your Django models to the Wagtail admin to make managing multiple types of models easier.

<div class="w-block-markdown block-markdown">
 <div class="prose prose-lg max-w-none">
 <h2>
 Versions
 </h2>
 <ul>
 <li>
 Python 3.7
 </li>
 <li>
 Django 2.2
 </li>
 <li>
 Wagtail 2.6
 </li>
 </ul>
 
 When working with Wagtail, you might find that you're using Wagtail Page models for some of your database models, but regular Django models for others.
 
 
 A built-in example of this is the Django
 <code>
 User
 </code>
 model. When you log into the Wagtail admin, you can see the Django
 <code>
 User
 </code>
 model in the
 <code>
 Settings
 </code>
 submenu. The
 <code>
 User
 </code>
 model is not a Wagtail model; it's the same
 <code>
 User
 </code>
 model you see in a Django project that doesn't use Wagtail. Wagtail just exposes it to the Admin for you.
 
 </div>
</div>
<div class="w-block-content block-content">
 
 
 <img alt="users_admin.png" class="richtext-image full-width" height="360" src="https://sfo2.digitaloceanspaces.com/revsys-com-prod-media/images/users_admin.width-800.png" width="404"/>
 
 
</div>
<div class="w-block-markdown block-markdown">
 <div class="prose prose-lg max-w-none">
 
 We can do the same thing with our Django models: we can expose them to the Wagtail admin so we don't have to maintain two separate admin interfaces to manage our website content.
 
 
 For this example, let's assume we're working with these models:
 
 <div class="codehilite">
 <pre><code>from django.db import models 


class Pizza(models.Model):

 name = models.CharField(max_length=30)
 toppings = models.ManyToManyField("Topping")


class Topping(models.Model):

 name = models.CharField(max_length=30)
</code></pre>
 </div>
 <h2>
 Adding a single model
 </h2>
 
 The
 <a href="http://docs.wagtail.io/en/v2.6.1/reference/contrib/modeladmin/">
 Wagtail docs
 </a>
 are pretty clear on how to accomplish this, but let's walk through the steps.
 
 
 First, make sure
 <code>
 wagtail.contrib.modeladmin
 </code>
 is in your
 <code>
 INSTALLED_APPS
 </code>
 :
 
 <div class="codehilite">
 <pre><code># settings.py 

INSTALLED_APPS = [
 ...
 "wagtail.contrib.modeladmin",
]
</code></pre>
 </div>
 
 Next, in the same app as the model you want to expose to the Wagtail admin, add a file called
 <code>
 wagtail_hooks.py
 </code>
 .
 
 <div class="codehilite">
 <pre><code># wagtail_hooks.py

from wagtail.contrib.modeladmin.options import ModelAdmin, modeladmin_register 

from .models import Pizza


class PizzaAdmin(ModelAdmin):
 model = Pizza 
 menu_label = "Pizza" 
 menu_icon = "pick" 
 menu_order = 200 
 add_to_settings_menu = False 
 exclude_from_explorer = False 
 list_display = ("name",)
 list_filter = ("toppings",)
 search_fields = ("name",)


modeladmin_register(PizzaAdmin)
</code></pre>
 </div>
 
 Let's step through these options in the
 <code>
 ModelAdmin
 </code>
 class:
 
 <ul>
 <li>
 <code>
 model
 </code>
 : The name of the model you're adding.
 </li>
 <li>
 <code>
 menu_label
 </code>
 : Leave this blank to use the
 <code>
 verbose_name_plural
 </code>
 from your model. Give it a value to specify a new label for the Wagtail menu.
 </li>
 <li>
 <code>
 menu_icon
 </code>
 : Every menu item in the Wagtail admin has an icon, and you can specify the one you want to use. Here is a
 <a href="https://thegrouchy.dev/general/2015/12/06/wagtail-streamfield-icons.html">
 list of the available icons
 </a>
 .
 </li>
 <li>
 <code>
 menu_order
 </code>
 : What order you want this model to appear in. 000 is first, 100 is second, etc. Note: if you add multiple models to the admin, you won't get an error if two of them have the same
 <code>
 menu_order
 </code>
 ; Wagtail will just pick for you.
 </li>
 <li>
 <code>
 add_to_settings_menu
 </code>
 : Whether you want this menu item to appear in the
 
 Settings
 
 submenu in the Wagtail admin.
 </li>
 <li>
 <code>
 exclude_from_explorer
 </code>
 : Set to True if you
 
 do not
 
 want the explorer (the search box in the admin) to return results from this model. Set to False if you
 
 do
 
 want the explorer to return results from this model. (It's confusing.)
 </li>
 <li>
 <code>
 list_display
 </code>
 : Same as the Django admin; list the fields you want to display on the listing page for this model in the Wagtail admin.
 </li>
 <li>
 <code>
 list_filter
 </code>
 : Same as the Django admin; supply the fields you want to use to filter in the sidebar of the Wagtail admin.
 </li>
 <li>
 <code>
 search_fields
 </code>
 : Same as the Django admin; supply the fields that you want the explorer to use to return search results.
 </li>
 </ul>
 
 The final step is to register the admin class. Once you've done that and started your server, you'll be able to see your model in the Wagtail admin:
 
 </div>
</div>
<div class="w-block-content block-content">
 
 
 <img alt="pizzas_admin.png" class="richtext-image full-width" height="328" src="https://sfo2.digitaloceanspaces.com/revsys-com-prod-media/images/pizzas_admin.width-800.png" width="800"/>
 
 
</div>
<div class="w-block-markdown block-markdown">
 <div class="prose prose-lg max-w-none">
 <h2>
 Adding related models
 </h2>
 
 In our example models, we have two models:
 <code>
 Pizza
 </code>
 and
 <code>
 Toppings
 </code>
 . We could manually add the
 <code>
 Topping
 </code>
 model to the Wagtail admin and have it appear just below the
 <code>
 Pizza
 </code>
 model. We just learned how!
 
 
 But it's so closely related to the
 <code>
 Pizza
 </code>
 model that it might be nice if we were able to relate those two models together in a submenu, kind of like how
 
 Settings
 
 is its own submenu in the admin that contains Users, Redirects, Sites, etc.
 
 
 Go back to
 <code>
 wagtail_hooks.py
 </code>
 :
 
 <div class="codehilite">
 <pre><code># wagtail_hooks.py

from wagtail.contrib.modeladmin.options import (
 ModelAdmin, 
 ModelAdminGroup, 
 modeladmin_register 
)

from .models import Pizza, Topping


class PizzaAdmin(ModelAdmin):
 ...
 menu_order = 000 
 ...


class ToppingAdmin(ModelAdmin):
 model = Topping 
 menu_label = "Toppings" 
 menu_icon = "edit" 
 menu_order = 100 
 add_to_settings_menu = False 
 exclude_from_explorer = False 
 list_display = ("name",)
 search_fields = ("name",)
</code></pre>
 </div>
 
 Relating our two models together starts off in the same way: we create a class that inherits from
 <code>
 ModelAdmin
 </code>
 for each model and identify the necessary attributes like
 <code>
 model
 </code>
 and
 <code>
 menu_icon
 </code>
 to control things like their listing pages and search behavior.
 
 
 Then, we add a new class that inherits from
 <code>
 ModelAdminGroup
 </code>
 :
 
 <div class="codehilite">
 <pre><code># wagtail_hooks.py

from wagtail.contrib.modeladmin.options import (
 ModelAdmin, 
 ModelAdminGroup, 
 modeladmin_register 
)

from .models import Pizza, Topping


class PizzaAdmin(ModelAdmin):
 ...
 menu_order = 000 
 ...


class ToppingAdmin(ModelAdmin):
 ...
 menu_order = 100 
 ... 

class PizzaGroup(ModelAdminGroup):
 menu_label = "Pizzas" 
 menu_icon = "pick"
 menu_order = 500 
 items = (PizzaAdmin, ToppingAdmin)


modeladmin_register(PizzaGroup)
</code></pre>
 </div>
 
 In the
 <code>
 PizzaGroup
 </code>
 class, we have some of the same attributes:
 
 <ul>
 <li>
 <code>
 menu_label
 </code>
 : We set what we want this group of related models to be called in the Wagtail admin menu
 </li>
 <li>
 <code>
 menu_icon
 </code>
 : Which icon we want to use for this menu
 </li>
 <li>
 <code>
 menu_order
 </code>
 : Where we want this menu to appear in the sidebar, in relation to the other menu items
 </li>
 </ul>
 
 We also add a new attribute,
 <code>
 items
 </code>
 , where we list which
 <code>
 ModelAdmin
 </code>
 classes we want to be part of this group. In our case, we want
 <code>
 PizzaAdmin
 </code>
 and
 <code>
 ToppingAdmin
 </code>
 to be in this group, so we add those.
 
 
 Note the change we made to
 <code>
 menu_order
 </code>
 in
 <code>
 PizzaAdmin
 </code>
 and
 <code>
 ToppingAdmin
 </code>
 : Now those are set to
 <code>
 000
 </code>
 and
 <code>
 100
 </code>
 . When the
 <code>
 ModelAdmin
 </code>
 classes will be part of a group, set the
 <code>
 menu_order
 </code>
 how you want them to relate to each other, not to the other menu items in the Wagtail admin. Then set the
 <code>
 menu_order
 </code>
 for the
 <code>
 ModelAdminGroup
 </code>
 class to the proper value for the order you want it to appear in the side menu in the admin.
 
 
 Then we register the whole group, instead of the
 <code>
 ModelAdmin
 </code>
 classes individually, to the Wagtail admin. When we reload the admin, we see this:
 
 </div>
</div>
<div class="w-block-content block-content">
 
 
 <img alt="pizzas_group_admin.png" class="richtext-image full-width" height="300" src="https://sfo2.digitaloceanspaces.com/revsys-com-prod-media/images/pizzas_group_admin.width-800.png" width="800"/>
 
 
</div>
<div class="w-block-markdown block-markdown">
 <div class="prose prose-lg max-w-none">
 
 On the far left, there is a new menu item
 
 Pizzas
 
 that expands a submenu. The submenu contains links to the admin interfaces for
 
 Pizzas
 
 and
 
 Toppings
 
 !
 
 
 
 Note
 
 : If you have the Django admin enabled and have your models already in the Django admin, this doesn't disable them from the regular Django admin. You are free to access your models in both the Wagtail admin and the Django admin, or at this point you can choose to remove your models from the Django admin (or disable the Django admin altogether, if you prefer).
 
 <h2>
 Helpful Links
 </h2>
 <ul>
 <li>
 <a href="http://docs.wagtail.io/en/v2.6.1/reference/contrib/modeladmin/">
 Wagtail docs on ModelAdmin
 </a>
 </li>
 <li>
 <a href="https://thegrouchy.dev/general/2015/12/06/wagtail-streamfield-icons.html">
 A List of Wagtail StreamField Icons
 </a>
 </li>
 </ul>
 
 
 Special thanks to Jeff Triplett and Jacob Burch for their help with this post.
 
 
 </div>
</div>

from django.db import models 


class Pizza(models.Model):

    name = models.CharField(max_length=30)
    toppings = models.ManyToManyField("Topping")


class Topping(models.Model):

    name = models.CharField(max_length=30)

# settings.py 

INSTALLED_APPS = [
    ...
    "wagtail.contrib.modeladmin",
]

# wagtail_hooks.py

from wagtail.contrib.modeladmin.options import ModelAdmin, modeladmin_register 

from .models import Pizza


class PizzaAdmin(ModelAdmin):
    model = Pizza 
    menu_label = "Pizza"  
    menu_icon = "pick" 
    menu_order = 200 
    add_to_settings_menu = False 
    exclude_from_explorer = False 
    list_display = ("name",)
    list_filter = ("toppings",)
    search_fields = ("name",)


modeladmin_register(PizzaAdmin)

# wagtail_hooks.py

from wagtail.contrib.modeladmin.options import (
    ModelAdmin, 
    ModelAdminGroup, 
    modeladmin_register 
)

from .models import Pizza, Topping


class PizzaAdmin(ModelAdmin):
    ...
    menu_order = 000 
    ...


class ToppingAdmin(ModelAdmin):
    model = Topping 
    menu_label = "Toppings"  
    menu_icon = "edit" 
    menu_order = 100 
    add_to_settings_menu = False 
    exclude_from_explorer = False 
    list_display = ("name",)
    search_fields = ("name",)

# wagtail_hooks.py

from wagtail.contrib.modeladmin.options import (
    ModelAdmin, 
    ModelAdminGroup, 
    modeladmin_register 
)

from .models import Pizza, Topping


class PizzaAdmin(ModelAdmin):
    ...
    menu_order = 000 
    ...


class ToppingAdmin(ModelAdmin):
    ...
    menu_order = 100 
    ... 

class PizzaGroup(ModelAdminGroup):
    menu_label = "Pizzas" 
    menu_icon = "pick"
    menu_order = 500 
    items = (PizzaAdmin, ToppingAdmin)


modeladmin_register(PizzaGroup)

  
  
   In the
   
    PizzaGroup
   
   class, we have some of the same attributes:
  
  
   
    
     menu_label
    
    : We set what we want this group of related models to be called in the Wagtail admin menu
   
   
    
     menu_icon
    
    : Which icon we want to use for this menu
   
   
    
     menu_order
    
    : Where we want this menu to appear in the sidebar, in relation to the other menu items
   
  
  
   We also add a new attribute,
   
    items
   
   , where we list which
   
    ModelAdmin
   
   classes we want to be part of this group. In our case, we want
   
    PizzaAdmin
   
   and
   
    ToppingAdmin
   
   to be in this group, so we add those.
  
  
   Note the change we made to
   
    menu_order
   
   in
   
    PizzaAdmin
   
   and
   
    ToppingAdmin
   
   : Now those are set to
   
    000
   
   and
   
    100
   
   . When the
   
    ModelAdmin
   
   classes will be part of a group, set the
   
    menu_order
   
   how you want them to relate to each other, not to the other menu items in the Wagtail admin. Then set the
   
    menu_order
   
   for the
   
    ModelAdminGroup
   
   class to the proper value for the order you want it to appear in the side menu in the admin.
  
  
   Then we register the whole group, instead of the
   
    ModelAdmin
   
   classes individually, to the Wagtail admin. When we reload the admin, we see this:
  
 


 
 
 
 
 


 
  
   On the far left, there is a new menu item
   
    Pizzas
   
   that expands a submenu. The submenu contains links to the admin interfaces for
   
    Pizzas
   
   and
   
    Toppings
   
   !
  
  
   
    Note
   
   : If you have the Django admin enabled and have your models already in the Django admin, this doesn't disable them from the regular Django admin. You are free to access your models in both the Wagtail admin and the Django admin, or at this point you can choose to remove your models from the Django admin (or disable the Django admin altogether, if you prefer).
  
  
   Helpful Links
  
  
   
    
     Wagtail docs on ModelAdmin
    
   
   
    
     A List of Wagtail StreamField Icons
    
   
  
  
   
    Special thanks to Jeff Triplett and Jacob Burch for their help with this post.

When working with Wagtail, you might find that you're using Wagtail Page models for some of your database models, but regular Django models for others. In this post, learn how to add your Django models to the Wagtail admin to make managing multiple types of models easier.

How to Add Django Models to the Wagtail Admin

On a recent project, we needed to use different serializers for GET vs POST/PUT/PATCH requests to our API. Read on to learn how we used a mixin to accomplish this goal.

<div class="w-block-markdown block-markdown">
 <div class="prose prose-lg max-w-none">
 
 
 Versions
 
 :
 
 <ul>
 <li>
 Python 3.7
 </li>
 <li>
 Django 2.2
 </li>
 <li>
 Django REST Framework 3.10
 </li>
 </ul>
 
 On a recent project, we needed to use different serializers for GET vs. POST/PUT/PATCH requests to our Django REST Framework API. In our case, this was because the GET serializer contained a lot of nested data; for example, it contained expanded fields from other serializers to foreign-key relationships. The requests to update data via the API, though, didn't need these expanded fields.
 
 
 The first way we approached using different serializers for read and update actions was to override
 <code>
 get_serializer_class()
 </code>
 on each viewset to decide which serializer to return depending on the action in the request. We returned the "read" serializer for
 <code>
 list
 </code>
 and
 <code>
 retrieve
 </code>
 actions, and the "update" serializer for everything else. (The full list of API actions is
 <a href="https://github.com/encode/django-rest-framework/blob/335054a5d36b352a58286b303b608b6bf48152f8/rest_framework/schemas/coreapi.py#L39">
 in the DRF codebase
 </a>
 .) But we wound up repeating ourselves across several viewsets, so we wrote a mixin to take care of some of this work for us!
 
 
 A mixin is a Python class that contains custom attributes and methods (
 <a href="https://easyaspython.com/mixins-for-fun-and-profit-cb9962760556?gi=3875fd6a6ff8">
 more explanation
 </a>
 ). It's not very useful on its own, but when it's inherited into a class, that class has access to the mixin's special attributes and methods.
 
 
 This was our mixin:
 
 <div class="codehilite">
 <pre><code>class ReadWriteSerializerMixin(object):
 """
 Overrides get_serializer_class to choose the read serializer
 for GET requests and the write serializer for POST requests.

 Set read_serializer_class and write_serializer_class attributes on a
 viewset. 
 """

 read_serializer_class = None
 write_serializer_class = None

 def get_serializer_class(self): 
 if self.action in ["create", "update", "partial_update", "destroy"]:
 return self.get_write_serializer_class()
 return self.get_read_serializer_class()

 def get_read_serializer_class(self):
 assert self.read_serializer_class is not None, (
 "'%s' should either include a `read_serializer_class` attribute,"
 "or override the `get_read_serializer_class()` method."
 % self.__class__.__name__
 )
 return self.read_serializer_class

 def get_write_serializer_class(self):
 assert self.write_serializer_class is not None, (
 "'%s' should either include a `write_serializer_class` attribute,"
 "or override the `get_write_serializer_class()` method."
 % self.__class__.__name__
 )
 return self.write_serializer_class
</code></pre>
 </div>
 
 This mixin defines two new attributes,
 <code>
 read_serializer_class
 </code>
 and
 <code>
 write_serializer_class
 </code>
 . Each attribute has a corresponding method to catch the error where the mixin is being used, but those attributes haven't been set. The
 <code>
 get_*_serializer_class()
 </code>
 methods will raise an
 <code>
 AssertionError
 </code>
 if your viewset hasn't set the appropriate attribute or overridden the necessary method.
 
 
 The
 <code>
 get_serializer_class
 </code>
 method makes the final decision on which serializer to use. For the "update" actions to the API, it returns
 <code>
 write_serializer_class
 </code>
 ; otherwise it returns
 <code>
 read_serializer_class
 </code>
 .
 
 
 The mixin gets used in a viewset like this:
 
 <div class="codehilite">
 <pre><code>from rest_framework import viewsets

from .mixins import ReadWriteSerializerMixin
from .models import MyModel
from .serializers import ModelReadSerializer, ModelWriteSerializer


class MyModelViewSet(ReadWriteSerializerMixin, viewsets.ModelViewSet):
 queryset = MyModel.objects.all() 
 read_serializer_class = ModelReadSerializer 
 write_serializer_class = ModelWriteSerializer
</code></pre>
 </div>
 
 Now the viewset
 <code>
 MyModelViewSet
 </code>
 has access to the attributes and methods from the mixin
 <code>
 ReadWriteSerializerMixin
 </code>
 . This means that when a call is made to the API that uses
 <code>
 MyModelViewSet
 </code>
 , the
 <code>
 get_serializer_class()
 </code>
 method from
 <code>
 ReadWriteSerializerMixin
 </code>
 will automatically be called and will decide, based on the kind of API request being made, which serializer to use. If we needed to make even more granular decisions about the serializer returned (maybe we want to use a more limited serializer for a
 <code>
 list
 </code>
 request and one with more data in a
 <code>
 retrieve
 </code>
 request), then our viewset can override
 <code>
 get_write_serializer_class()
 </code>
 to add that logic.
 
 
 Note: Custom DRF actions will contain actions that aren't part of the DRF list of accepted actions (because they are custom actions you're creating), so when you call
 <code>
 get_serializer_class
 </code>
 from inside your action method, it will return whatever your "default" serializer class is. In the example above, the "default" serializer is the
 <code>
 read_serializer_class
 </code>
 because it's what we return when we fall through the other conditional.
 
 
 Depending on your action, you will want to override
 <code>
 get_serializer_class
 </code>
 to change your default method or explicitly account for your custom action.
 
 
 Mixins are a DRY (Don't Repeat Yourself) way to add functionality that you wind up needing to use across several viewsets. We hope you get to experiment with using them soon!
 
 
 
 Thanks to Jeff Triplett for his help with this post.
 
 
 </div>
</div>

class ReadWriteSerializerMixin(object):
    """
    Overrides get_serializer_class to choose the read serializer
    for GET requests and the write serializer for POST requests.

    Set read_serializer_class and write_serializer_class attributes on a
    viewset. 
    """

    read_serializer_class = None
    write_serializer_class = None

    def get_serializer_class(self):        
        if self.action in ["create", "update", "partial_update", "destroy"]:
            return self.get_write_serializer_class()
        return self.get_read_serializer_class()

    def get_read_serializer_class(self):
        assert self.read_serializer_class is not None, (
            "'%s' should either include a `read_serializer_class` attribute,"
            "or override the `get_read_serializer_class()` method."
            % self.__class__.__name__
        )
        return self.read_serializer_class

    def get_write_serializer_class(self):
        assert self.write_serializer_class is not None, (
            "'%s' should either include a `write_serializer_class` attribute,"
            "or override the `get_write_serializer_class()` method."
            % self.__class__.__name__
        )
        return self.write_serializer_class

from rest_framework import viewsets

from .mixins import ReadWriteSerializerMixin
from .models import MyModel
from .serializers import ModelReadSerializer, ModelWriteSerializer


class MyModelViewSet(ReadWriteSerializerMixin, viewsets.ModelViewSet):
    queryset = MyModel.objects.all() 
    read_serializer_class = ModelReadSerializer 
    write_serializer_class = ModelWriteSerializer

  
  
   Now the viewset
   
    MyModelViewSet
   
   has access to the attributes and methods from the mixin
   
    ReadWriteSerializerMixin
   
   . This means that when a call is made to the API that uses
   
    MyModelViewSet
   
   , the
   
    get_serializer_class()
   
   method from
   
    ReadWriteSerializerMixin
   
   will automatically be called and will decide, based on the kind of API request being made, which serializer to use. If we needed to make even more granular decisions about the serializer returned (maybe we want to use a more limited serializer for a
   
    list
   
   request and one with more data in a
   
    retrieve
   
   request), then our viewset can override
   
    get_write_serializer_class()
   
   to add that logic.
  
  
   Note: Custom DRF actions will contain actions that aren't part of the DRF list of accepted actions (because they are custom actions you're creating), so when you call
   
    get_serializer_class
   
   from inside your action method, it will return whatever your "default" serializer class is. In the example above, the "default" serializer is the
   
    read_serializer_class
   
   because it's what we return when we fall through the other conditional.
  
  
   Depending on your action, you will want to override
   
    get_serializer_class
   
   to change your default method or explicitly account for your custom action.
  
  
   Mixins are a DRY (Don't Repeat Yourself) way to add functionality that you wind up needing to use across several viewsets. We hope you get to experiment with using them soon!
  
  
   
    Thanks to Jeff Triplett for his help with this post.

Using Different Read and Write Serializers in Django REST Framework

Python 3.7 introduced dataclasses, which design is based on the &quot;attrs&quot; library. This article will show the way I use dataclasses and attrs, why I think you should use both, and why I think attrs is still very relevant.

def render_document(title: str, caption: str, data: List[Dict[str, Any]]):
    return template.render({
        "title" : title,
        "caption": caption,
        "data": data,
})

{
    "title": "My Table",
    "caption": "2019 Earnings",
    "data": [
        {"Period": "QT1", "Europe": 500, "USA": 467},
        {"Period": "QT2", "Europe": 345, "USA": 765},
    ]
}

def render_document(title: str, description: str, status: str, tables: List[Dict[str, Any]]):
    return template.render({
        "title": title,
        "description": description,
        "status": status,
        "tables": tables,
    })

@dataclass
class Table:
    title: str
    data: List[Dict[str, Any]]
    caption: str = ""

def render_document(title: str, description: str, tables: List[Table]):
    return template.render({
        "title": title,
        "description": description,
        "tables": tables,
    })

@attr.s
class Table(object):
    title: str = attr.ib(validator=attr.validators.instance_of(str))  # don't you pass no bytes!
    data: List[Dict[str, Any]] = attr.ib(validator=...)
    description: str = attr.ib(validator=attr.validators.instance_of(str), default="")


def render_document(title: str, description: str, tables: List[Table]):
    return template.render({
        "title": title,
        "description": description,
        "tables": tables,
    })

@dataclass
class Table:
    title: str
    data: List[Dict[str, Any]]
    caption: str = ""

@attr.s
class Document(object):
    status: str = attr.ib(validators=attr.validators.in_(
        ["draft", "in review", "approved"]
    ))
    tables: List[Table] = attr.ib(default=[])

def render_report(self, title: str, documents: List[Document]):
    return template.render({
        "title": title,
        "documents": documents,
    })

  
  
   Note how I am validating that
   
    Document.status
   
   is one of the allowed values. This comes particularly handy when you're building abstractions on top of Django models with a field that uses
   
    choices
   
   . Dataclasses can't do that.
  
  
   A couple of patterns I keep finding myself in are the following:
  
  
   
    Write a function that accepts some arguments
   
   
    Group some of the arguments into a
    
     tuple
    
   
   
    Hm, I want field names ->
    
     namedtuple
    
    .
   
   
    Hm, I want types ->
    
     dataclass
    
    .
   
   
    Hm, I want validation ->
    
     attrs
    
    .
   
  
  
   Another situation that happens quite often is this:
  
  
   
    write a function that accepts some arguments
   
   
    add typing so my IDE can help me out
   
   
    oh, by the way, it needs to support a list of those things, not just one at a time!
   
   
    refactor to use dataclasses
   
   
    This argument can only be one of those values, or
    
     
      I ask myself: How do I make sure other developers are passing the right type and/or values?
     
    
   
   
    switch to attrs
   
  
  
   Sometimes I stop at the dataclasses. Lots of times I get to the attrs step.
  
  
   And sometimes, this happens:
1. one half of this legacy codebase uses
   
    -1
   
   as special value for
   
    False
   
   , that other half uses
   
    False
   
   . Switch to
   
    attr.s
   
   so I can use
   
    converter=
   
   to normalize.
  
  
   Comparison
  
  
   The two libraries do appear very similar. To get a clearer picture of how they compare, I've made a table of the features I use most:
  
  
   
    
     
      feature
     
     
      dataclasses
     
     
      attrs
     
    
   
   
    
     
      frozen
     
     
      ✓
     
     
      ✓
     
    
    
     
      defaults
     
     
      ✓
     
     
      ✓
     
    
    
     
      totuple
     
     
      ✓
     
     
      ✓
     
    
    
     
      todict
     
     
      ✓
     
     
      ✓
     
    
    
     
      validators
     
     
      ✗
     
     
      ✓
     
    
    
     
      converters
     
     
      ✗
     
     
      ✓
     
    
    
     
      slotted classes
     
     
      ✗
     
     
      ✓
     
    
   
  
  
   As you can see, there's a lot of overlap. But the additional features on
   
    attrs
   
   provide functionality that I need more often than not.
  
  
   When to use dataclasses
  
  
   Dataclasses are just about the "shape" of the data.
Choose dataclasses if:
  
  
   
    You don't care about values in the fields, only their type
   
   
    adding a dependency is not trivial
   
  
  
   When to use attrs
  
  
   attrs is about the shape
   
    and
   
   the values.
Choose attrs if:
  
  
   
    you want to validate values. A common case would be the equivalent of a ChoiceField.
   
   
    you want to normalize, or sanitize the input
   
   
    whenever you want more formalization than dataclasses alone can offer
   
   
    you are concerned about memory and performances.
    
     attrs
    
    can create
    
     slotted classes
    
    , which are
    
     optimized by CPython
    
    .
   
  
  
   I often find myself using dataclasses and later switching to attr.s because the requirements changed or I find out I need to guard against some particular value. I think that's a normal aspect of developing software and what I call "continuous refactoring".
  
  
   Why I like dataclasses
  
  
   I'm glad dataclasses have been added to the standard library, and I think it's a beneficial addition. It's a very convenient thing to have at your disposal whenever you need.
  
  
   For one, it will encourage a more structured style of programming from the beginning.
  
  
   But I think the most compelling case is a practical one. Some high-risk corporate environments (eg: financial institutions) require every package to be vetted (with good reason: we've already had incidents of malicious code in libraries). That means that adding attrs is not as simple as adding a line to your
   
    requirements.txt
   
   , and will involve waiting on approval from your corpops team. Those developers can use dataclasses right away and their code will immediately benefit from using more formalized data types.
  
  
   Why I like attrs
  
  
   Most people don't work in such strictly-controlled environments.
  
  
   And sure, sometimes you don't need all the features from attrs, but it doesn't hurt having them.
  
  
   More often than not, I end up needing them anyway, as I formalize more and more of my code's API. Dataclasses only gets half-way of where I want to go.
  
  
   Conclusion
  
  
   I think dataclasses encompass only a subset of what attrs has to offer. Admittedly, it is a
   
    big
   
   subset. But the features that are not covered are important enough and needed often enough that they make attrs not only still relevant and useful, but also necessary.
  
  
   In my mind, using both allows developers to progressively refactor their code, moving the contracts amongst functions from loosely-defined arguments all the way up to formally described data structures as the requirements of the app stabilize over time.
  
  
   One nice effect of having dataclasses is that now developers are more incentivized to refactor their code toward more formalization. At some point dataclasses is not going to be enough, and that's when developers will refactor to use attrs. In this way, dataclasses actually acts as an
   
    introduction
   
   to attrs. I wouldn't be surprised if attrs becomes
   
    more
   
   popular thanks to dataclasses.
  
  
   References
  
  
   
    
     Documentation on dataclasses
    
    +
    
     PEP557
    
   
   
    
     Documentation on attrs
    
   
   
    
     "Why not dataclasses" on attrs' docs
    
   
  
  
   Acknowledgments + Thanks
  
  
   I want to thank the following people for revising drafts and providing input and insights:
  
  
   
    Hynek Schlawack
   
   
    Jacob Kaplan-Moss
   
   
    Jacob Burch
   
   
    Jeff Triplett

Python 3.7 introduced dataclasses, which design is based on the "attrs" library. This article will show the way I use dataclasses and attrs, why I think you should use both, and why I think attrs is still very relevant.

Dataclasses and attrs: when and why

How to smartly keep a set of Django ORM objects in order for drag-n-drop like situations.

class Task(models.Model):
 name = models.CharField(max_length=100)

class Step(models.Model):
 task = models.ForeignKey(Task, related_name='steps', on_delete=models.CASCADE)
 name = models.CharField(max_length=100)
 order = models.IntegerField(default=1)

 objects = StepManager()

from django.db import models, transaction
from django.db.models import F

class StepManager(models.Manager):
 """ Manager to encapsulate bits of business logic """

 def move(self, obj, new_order):
 """ Move an object to a new order position """

 qs = self.get_queryset()

 with transaction.atomic():
 if obj.order > int(new_order):
 qs.filter(
 task=obj.task,
 order__lt=obj.order,
 order__gte=new_order,
 ).exclude(
 pk=obj.pk
 ).update(
 order=F('order') + 1,
 )
 else:
 qs.filter(
 task=obj.task,
 order__lte=new_order,
 order__gt=obj.order,
 ).exclude(
 pk=obj.pk,
 ).update(
 order=F('order') - 1,
 )

 obj.order = new_order
 obj.save()

Step.objects.move(step_3_obj, 5)

from django.db import models, transaction
from django.db.models import F, Max

class StepManager(models.Manager):
 # ... previous code here ...

 def create(self, **kwargs):
 instance = self.model(**kwargs)

 with transaction.atomic():
 # Get our current max order number
 results = self.filter(
 task=instance.task
 ).aggregate(
 Max('order')
 )

 # Increment and use it for our new object
 current_order = results['order__max']
 if current_order is None:
 current_order = 0

 value = current_order + 1
 instance.order = value
 instance.save()

 return instance

class Step(models.Model):

 # ... original fields and manager definitions ... 

 class Meta:
 index_together = ('task', 'order')

from rest_framework import viewsets, status
from rest_framework.decorators import action
from rest_framework.response import Response
from django_filters.rest_framework import DjangoFilterBackend

from . import models
from . import serializers

class StepViewSet(viewsets.ModelViewSet):
 queryset = models.Step.objects.all().order_by('order')
 serializer_class = serializers.StepSerializer
 filter_backends = (DjangoFilterBackend, )
 filter_fields = ('task', )

 @action(methods=['post'], detail=True)
 def move(self, request, pk):
 """ Move a single Step to a new position """
 obj = self.get_object()
 new_order = request.data.get('order', None)

 # Make sure we received an order 
 if new_order is None:
 return Response(
 data={'error': 'No order given'},
 status=status.HTTP_400_BAD_REQUEST,
 )

 # Make sure our new order is not below one
 if int(new_order) < 1:
 return Response(
 data={'error': 'Order cannot be zero or below'},
 status=status.HTTP_400_BAD_REQUEST,
 )

 models.Step.objects.move(obj, new_order)

 return Response({'success': True, 'order': new_order})

 
 
 Where these endpoints reside is going to depend on how you configure things in your Django project, but assuming you're using a DRF
 
 SimpleRouter
 
 and have things setup on
 
 /api/
 
 your URL patterns are going to be something like:
 
 
 
 
 GET /api/steps/
 
 for a list of all steps
 
 
 
 GET /api/steps/?task=<task_id>
 
 for all Steps by Task (in Step order)
 
 
 
 POST /api/steps/<step_id>/move/
 
 with a JSON body of
 
 {'order': <int>}
 
 of the that Step's new order position.
 
 
 
 
 NOTE
 
 This API example excludes any authentication and authorization concerns which you will need to address before using in production. You don't want just anyone reordering your steps!
 
 
 Hopefully, you've found this informative and can put this pattern to use
in your next project. If you need help implementing things of this
nature at your company
 
 reach out to us
 
 to check on our
availability.

Keeping Django Models Ordered

ManyToManyFields confuse a lot of people. The way you relate objects to each other is just different enough from dealing with ForeignKeys and just uncommon enough in day-to-day Django development that it&#x27;s easy to forget all the little tricks for dealing with them.

from django.db import models 


class Pizza(models.Model):

 name = models.CharField(max_length=30)
 toppings = models.ManyToManyField('Topping')

 def __str__(self):
 return self.name


class Topping(models.Model):

 name = models.CharField(max_length=30)

 def __str__(self):
 return self.name

>> from pizzas.models import Pizza, Topping
>> hawaiian_pizza = Pizza.objects.create(name='Hawaiian')
>> pineapple = Topping(name='pineapple')
>> hawaiian_pizza.toppings.add(pineapple)
Traceback (most recent call last):
...
ValueError: Cannot add "<Topping: pineapple>": instance is on database "default", 
value is on database "None"
>> 

>> pineapple.save() 
>> hawaiian_pizza.toppings.add(pineapple)
>> hawaiian_pizza.toppings.all()
<QuerySet [<Topping: pineapple>]>

>> pepperoni = Topping.objects.create(name='pepperoni')
>> pepperoni_pizza = Pizza(name='Pepperoni')
>> pepperoni_pizza.toppings.add(pepperoni)
Traceback (most recent call last):
...
ValueError: "<Pizza: Pepperoni>" needs to have a value for field "id" before this many-to-many 
relationship can be used.

>> hawaiian_pizza.toppings.all()
<QuerySet [<Topping: pineapple>, <Topping: Canadian bacon>]>

>> canadian_bacon.pizzas.all()
Traceback (most recent call last):
...
AttributeError: 'Topping' object has no attribute 'pizzas'

>> canadian_bacon.pizza_set.all()
<QuerySet [<Pizza: Hawaiian>]>

class Pizza(models.Model):
 ...
 toppings = models.ManyToManyField('Topping', related_name='pizzas')

>> canadian_bacon.pizza_set.all()
<QuerySet [<Pizza: Hawaiian>]>

>> canadian_bacon.pizzas.all()
<QuerySet [<Pizza: Hawaiian>]>

>> cheese_pizza = Pizza.objects.create(name='Cheese')
>> mozzarella = Topping.objects.create(name='mozzarella')
>> mozzarella.pizzas.add(cheese_pizza)
>> mozzarella.pizzas.all()
<QuerySet [<Pizza: Cheese>]>

>> Pizza.objects.filter(toppings__name__startswith='p')
<QuerySet [<Pizza: Pepperoni Pizza>, <Pizza: Hawaiian Pizza>]>

>> Topping.objects.filter(pizzas__name__contains='Hawaiian')
<QuerySet [<Topping: pineapple>, <Topping: Canadian bacon>]>

>> pepperoni_pizza.toppings.all()
<QuerySet [<Topping: pepperoni>]>
>> pepperoni_pizza.toppings.add(pepperoni)
>> pepperoni_pizza.toppings.all()
<QuerySet [<Topping: pepperoni>]>

ValueError: Cannot alter field pizzas.Pizza.toppings into pizzas.Pizza.toppings - 
they are not compatible types (you cannot alter to or from M2M fields, or add or remove through= on M2M fields)

class ToppingAmount(models.Model):

 REGULAR = 1
 DOUBLE = 2
 TRIPLE = 3
 AMOUNT_CHOICES = (
 (REGULAR, 'Regular'),
 (DOUBLE, 'Double'),
 (TRIPLE, 'Triple'),
 )

 pizza = models.ForeignKey('Pizza', related_name='topping_amounts', on_delete=models.SET_NULL, null=True)
 topping = models.ForeignKey('Topping', related_name='topping_amounts', on_delete=models.SET_NULL, null=True, blank=True)
 amount = models.IntegerField(choices=AMOUNT_CHOICES, default=REGULAR)

class Pizza(models.Model):
 ...
 toppings = models.ManyToManyField('Topping', through='ToppingAmount', related_name='pizzas')

>> super_pep = Pizza.objects.create(name='Super Pepperoni')
>> pepperoni = Topping.objects.create(name='pepperoni')
>> super_pep.toppings.add(pepperoni)
Traceback (most recent call last):
...
AttributeError: Cannot use add() on a ManyToManyField which specifies an intermediary model. 
Use pizzas.ToppingAmount's Manager instead.

>>> super_pep_amount = ToppingAmount.objects.create(pizza=super_pep, topping=pepperoni, amount=ToppingAmount.DOUBLE)

>> super_pep.toppings.all()
<QuerySet [<Topping: pepperoni>]>

>> pepperoni.pizzas.all()
<QuerySet [<Pizza: Super Pepperoni>]>

for top_amt in ToppingAmount.objects.filter(pizza=super_pep):
 print(top_amt.topping.name, top_amt.get_amount_display())

pepperoni Double

for top_amt in super_pep.topping_amounts.all():
 print(top_amt.topping.name, top_amt.get_amount_display()) 

pepperoni Double

for topping in super_pep.toppings.all():
 for top_amt in topping.topping_amount.all():
 print(topping.name, top_amt.amount, top_amt.get_amount_display())

pepperoni 2 Double

 
 
 You could extend this
 
 ToppingAmount
 
 through model to hold information about the left and right halves of the pizza, or notes about topping preparation ("put peppers under cheese"). You can also add methods to the through model or the source/target models to more easily access some of the topping amount information.
 
 
 A through model is also useful for relationships between players and teams; the through model could contain information about the players' positions, jersey numbers, and dates they joined the team. A through model joining movie theatres and films could contain the number of screens the film is showing on and the start and end run dates. Students' relationships to their Degree Programs could track information like GPA, whether the program is the student's major or minor, whether it's a double major, and start/end semesters the student was in a program.
 
 
 
 Further Reading
 
 
 
 The
 
 many-to-many relationships
 
 section of the Django docs contains a lot of great examples of using
 
 ManyToManyField
 
 .
 
 
 The explanation of
 
 
 ManyToManyField
 
 
 in the model field reference.
 
 
 Jacob Kaplan-Moss also has a
 
 great set of examples
 
 of using a many-to-many relationship with a custom "through" model.
 
 
 
 
 Thank you to
 
 Monique Murphy
 
 and
 
 Jeff Triplett
 
 for their assistance.

ManyToManyFields confuse a lot of people. The way you relate objects to each other is just different enough from dealing with ForeignKeys and just uncommon enough in day-to-day Django development that it's easy to forget all the little tricks for dealing with them.

Tips for Using Django's ManyToManyField

A very common pattern in a Django Project is to have some kind of &#x27;model-type&#x27; relationship, where you have some kind of object that can only belong to one of the types defined in the database.

<div class="w-block-markdown block-markdown">
 <div class="prose prose-lg max-w-none">
 <h1>
 Slugs as primary keys
 </h1>
 
 A very common pattern I've seen in Django Project is to have some kind of 'model-type' relationship, where you have some kind of object that can only belong to one of the types defined in the database.
 
 
 A typical implementation would look something like this:
 
 <div class="codehilite">
 <pre><code>class EventType(models.Model):
 slug = models.SlugField(unique=True)
 name = models.CharField(max_length=100)


class Event(models.Model):
 type = models.ForeignKey(EventType, on_delete=models.CASCADE)
 title = models.CharField(max_length=100)
 day = models.DateField()
</code></pre>
 </div>
 
 A subtle issue with this implementation is that you may have to query the DB for an
 <code>
 EventType
 </code>
 when creating an
 <code>
 Event
 </code>
 :
 
 <div class="codehilite">
 <pre><code>work_type = EventType.objects.get(slug='work')
Event.objects.create(
 type=work_type,
 title='release',
 day=datetime(2018, 12, 31),
)
</code></pre>
 </div>
 
 In other words, the minimum amount of queries to create an
 <code>
 Event
 </code>
 is 2: a
 <code>
 SELECT
 </code>
 for fetching the type, and the
 <code>
 INSERT
 </code>
 that saves the instance.
 
 
 If somehow you already have the
 <code>
 pk
 </code>
 of the
 <code>
 EventType
 </code>
 (for example, it might come from an API payload or from the URL), then you can easily avoid the lookup by setting the primary key directly into the
 <code>
 type_id
 </code>
 column:
 
 <div class="codehilite">
 <pre><code>def post(self, request, *args, **kwargs):
 type_id = self.kwargs['pk']
 Event.objects.create(
 type_id=type_id,
 title='release',
 day=datetime(2018, 12, 31),
 )
</code></pre>
 </div>
 
 But dealing directly with column names is
 <a href="https://docs.djangoproject.com/en/2.1/ref/models/fields/#database-representation">
 discouraged by the docs
 </a>
 
 <blockquote>
 
 However, your code should never have to deal with the database column name
 
 </blockquote>
 
 We can get around this by instantiating an
 <code>
 EventType
 </code>
 instance with just the primary key:
 
 <div class="codehilite">
 <pre><code>def post(self, request, *args, **kwargs):
 type_id = self.kwargs['pk']
 Event.objects.create(
 type=EventType(pk=type_id),
 title='release',
 day=datetime(2018, 12, 31),
 )
</code></pre>
 </div>
 
 But this requires us to know the numerical
 <code>
 id
 </code>
 beforehand.
 
 
 We already have
 <code>
 slug
 </code>
 as source of uniqueness for the
 <code>
 event_eventtypes
 </code>
 table, and it's URL-friendly. We could just use that as the
 <code>
 EventType
 </code>
 primary key.
 
 <div class="codehilite">
 <pre><code>class EventType(models.Model):
 slug = models.SlugField(primary_key=True)
 name = models.CharField(max_length=100)


class Event(models.Model):
 type = models.ForeignKey(
 EventType, 
 on_delete=models.CASCADE,
 )
 title = models.CharField(max_length=100)
 day = models.DateField()
</code></pre>
 </div>
 <div class="codehilite">
 <pre><code>def post(self, request, *args, **kwargs):
 type_slug = self.kwargs['slug']
 Event.objects.create(
 type=EventType(pk=type_slug),
 title='release',
 day=datetime(2018, 12, 31),
 )
</code></pre>
 </div>
 
 This also allows to easily create
 <code>
 Event
 </code>
 s of a specific type without having to fetching any
 <code>
 EventType
 </code>
 . This is especially useful in data migrations, or tests:
 
 <div class="codehilite">
 <pre><code>types = ['home', 'work', 'community']
for type_slug in types:
 Event.objects.create(
 type=EventType(pk=type_slug),
 title='release',
 day=datetime(2018, 12, 31),
 )
</code></pre>
 </div>
 
 In cases like this, you may want to consider using a slug as primary key, rather than the default integer-based default. It's more performant and just as straight forward.
 
 
 There are some caveat to consider, though: You won't be able to modify the slug from the Django admin.
 
 
 Another thing to consider is that changing the type of your primary key is a one-way road: once you made the slug your pk, you won't be able to convert it back to a regular
 <code>
 AutoField
 </code>
 .
 
 </div>
</div>

class EventType(models.Model):
    slug = models.SlugField(unique=True)
    name = models.CharField(max_length=100)


class Event(models.Model):
    type = models.ForeignKey(EventType, on_delete=models.CASCADE)
    title = models.CharField(max_length=100)
    day = models.DateField()

work_type = EventType.objects.get(slug='work')
Event.objects.create(
    type=work_type,
    title='release',
    day=datetime(2018, 12, 31),
)

def post(self, request, *args, **kwargs):
    type_id = self.kwargs['pk']
    Event.objects.create(
        type_id=type_id,
        title='release',
        day=datetime(2018, 12, 31),
    )

def post(self, request, *args, **kwargs):
    type_id = self.kwargs['pk']
    Event.objects.create(
       type=EventType(pk=type_id),
       title='release',
       day=datetime(2018, 12, 31),
    )

class EventType(models.Model):
    slug = models.SlugField(primary_key=True)
    name = models.CharField(max_length=100)


class Event(models.Model):
    type = models.ForeignKey(
        EventType, 
        on_delete=models.CASCADE,
    )
    title = models.CharField(max_length=100)
    day = models.DateField()

def post(self, request, *args, **kwargs):
    type_slug = self.kwargs['slug']
    Event.objects.create(
       type=EventType(pk=type_slug),
       title='release',
       day=datetime(2018, 12, 31),
    )

types =  ['home', 'work', 'community']
for type_slug in types:
    Event.objects.create(
       type=EventType(pk=type_slug),
       title='release',
       day=datetime(2018, 12, 31),
    )

  
  
   In cases like this, you may want to consider using a slug as primary key, rather than the default integer-based default. It's more performant and just as straight forward.
  
  
   There are some caveat to consider, though: You won't be able to modify the slug from the Django admin.
  
  
   Another thing to consider is that changing the type of your primary key is a one-way road: once you made the slug your pk, you won't be able to convert it back to a regular
   
    AutoField
   
   .

A very common pattern in a Django Project is to have some kind of 'model-type' relationship, where you have some kind of object that can only belong to one of the types defined in the database.

Slugs as primary keys

It is often necessary to differentiate between an argument that has not been provided, and an argument provided with the value `None`. For that purpose, we create what&#x27;s called a &#x27;sentinel value&#x27;.

<div class="w-block-markdown block-markdown">
 <div class="prose prose-lg max-w-none">
 
 Sometimes it is necessary to differentiate between an argument that has not
been provided, and an argument provided with the value
 <code>
 None
 </code>
 . For that purpose, we create what's called a 'sentinel value'.
 
 
 For example, let's assume you want to define a Field class. Field instances must have a value or declare a default,
and
 <code>
 None
 </code>
 could be a perfectly valid value to have:
 
 <div class="codehilite">
 <pre><code>class Field:
 def __init__(self, default=sentinel):
 self.value = default

 def set(self, value):
 self.value = value

 def get(self):
 if self.value is sentinel:
 raise ValueError("this field has no value!")

eula_accepted = Field()
eula_accepted.get() # raises `ValueError`

eula_accepted = Field(default=None)
eula_accepted.get() # doesn't raise. `None` means the EULA hasn't been neither accepted or reject yet.
</code></pre>
 </div>
 
 The most common approach is to declare the sentinel value with
 <code>
 object()
 </code>
 :
 
 <div class="codehilite">
 <pre><code>sentinel = object()
</code></pre>
 </div>
 
 This approach is the quickest and most common, but it has some issues. To quote
 <a href="https://mail.python.org/pipermail/python-ideas/2018-July/051852.html">
 Joseph Jevnik
 </a>
 :
 
 <blockquote>
 
 One is that they don't repr well so they make debugging harder. Another issue
is that they cannot be pickled or copied. You also cannot take a weak
reference to a sentinel which can break some caching code and makes
them harder to use.
 
 </blockquote>
 
 For example:
 
 <div class="codehilite">
 <pre><code>sentinel = object()

repr(sentinel)
# '&lt;object object at 0x10823e8d0&gt;'
</code></pre>
 </div>
 
 To work around this issue, some people create their own
 <code>
 Sentinel
 </code>
 class. But I've found a quicker way in the
 <code>
 unittest.mock
 </code>
 module.
 
 <div class="codehilite">
 <pre><code>from unittest.mock import sentinel

NotSet = sentinel.NotSet

repr(NotSet)
# 'sentinel.NotSet'
</code></pre>
 </div>
 
 If you don't feel like importing from
 <code>
 unittest
 </code>
 in your application code, you could install the
 <code>
 mock
 </code>
 package, or "hide it under the rug" by aliasing somewhere in your code base and importing it from there:
 
 <div class="codehilite">
 <pre><code># in `myproject.types`
from unittest.mock import sentinel
</code></pre>
 </div>
 <div class="codehilite">
 <pre><code># somewhere else in your project:
from myproject.types import sentinel
</code></pre>
 </div>
 
 An alternative to
 <code>
 unittest.mock.sentinel
 </code>
 is to declare your own sentinel class and use it as a value:
 
 <div class="codehilite">
 <pre><code>class NotSet:
 pass

# PS: Remember to use the class _itself_.
def fn(default=NotSet):
 pass
</code></pre>
 </div>
 
 This will give you a not really pretty, but useful enough
 <code>
 repr
 </code>
 :
 
 <div class="codehilite">
 <pre><code>repr(NotSet)
# "&lt;class '__main__.NotSet'&gt;"
</code></pre>
 </div>
 
 Of course, you could go one step further and declare your own
 <code>
 repr
 </code>
 :
 
 <div class="codehilite">
 <pre><code>class NotSet:
 def __repr__(self):
 return 'NotSet'

repr(NotSet)
# 'NotSet'
</code></pre>
 </div>
 
 Of all the options, I think using
 <code>
 unittest.mock.Sentinel
 </code>
 is my favorite. Importing from
 <code>
 unittest
 </code>
 in my application code is a compromise that I'm willing to make in exchange for having something ready to use.
 
 </div>
</div>

class Field:
 def __init__(self, default=sentinel):
 self.value = default

 def set(self, value):
 self.value = value

 def get(self):
 if self.value is sentinel:
 raise ValueError("this field has no value!")

eula_accepted = Field()
eula_accepted.get() # raises `ValueError`

eula_accepted = Field(default=None)
eula_accepted.get() # doesn't raise. `None` means the EULA hasn't been neither accepted or reject yet.

sentinel = object()

sentinel = object()

repr(sentinel)
# '<object object at 0x10823e8d0>'

from unittest.mock import sentinel

NotSet = sentinel.NotSet

repr(NotSet)
# 'sentinel.NotSet'

# in `myproject.types`
from unittest.mock import sentinel

# somewhere else in your project:
from myproject.types import sentinel

class NotSet:
 pass

# PS: Remember to use the class _itself_.
def fn(default=NotSet):
 pass

repr(NotSet)
# "<class '__main__.NotSet'>"

class NotSet:
 def __repr__(self):
 return 'NotSet'

repr(NotSet)
# 'NotSet'

 
 
 Of all the options, I think using
 
 unittest.mock.Sentinel
 
 is my favorite. Importing from
 
 unittest
 
 in my application code is a compromise that I'm willing to make in exchange for having something ready to use.

It is often necessary to differentiate between an argument that has not been provided, and an argument provided with the value `None`. For that purpose, we create what's called a 'sentinel value'.

Sentinel values in Python

In this post, you will learn how to create a Celery task inside a Django project in a Docker container. Sounds awesome, right?

proj/
├── app/
 ├── __init__.py
 ├── tasks.py
├── proj/
 ├── __init__.py
 ├── celery.py
 ├── settings.py
├── docker-compose.yml
├── Dockerfile
├── manage.py
└── requirements.txt

celery==4.2.1
redis==2.10.6

import os
from celery import Celery


os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'proj.settings')

app = Celery('proj')
app.config_from_object('django.conf:settings', namespace='CELERY')
app.autodiscover_tasks()

@app.task(bind=True)
def debug_task(self):
 print('Request: {0!r}'.format(self.request))

from celery import shared_task


@shared_task
def hello():
 print(“Hello there!”) 

from .celery import app as celery_app

__all__ = ['celery_app']

>>> from app.tasks import hello
>>> hello()
Hello there! 

>>> hello.delay()
<AsyncResult: ba845cf3-e60b-4432-a9d8-9943621cb8a0>

CELERY_BROKER_URL = 'redis://redis:6379'
CELERY_RESULT_BACKEND = 'redis://redis:6379'
CELERY_ACCEPT_CONTENT = ['application/json']
CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_SERIALIZER = 'json'

from celery.schedules import crontab

CELERY_BEAT_SCHEDULE = {
 'hello': {
 'task': 'app.tasks.hello',
 'schedule': crontab() # execute every minute
 }
}

version: '3'

services:
 db:
 image: postgres:9.6.5
 volumes:
 - postgres_data:/var/lib/postgresql/data/
 redis:
 image: "redis:alpine"
 web:
 build: .
 command: bash -c "python /code/manage.py migrate --noinput && python /code/manage.py runserver 0.0.0.0:8000"
 volumes:
 - .:/code
 ports:
 - "8000:8000"
 depends_on:
 - db
 - redis
 celery:
 build: .
 command: celery -A proj worker -l info
 volumes:
 - .:/code
 depends_on:
 - db
 - redis
 celery-beat:
 build: .
 command: celery -A proj beat -l info
 volumes:
 - .:/code
 depends_on:
 - db
 - redis

volumes:
 postgres_data:

redis:
 image: "redis:alpine"

depends_on:
 - db
 - redis

celery:
 build: .
 command: celery -A proj worker -l info
 volumes:
 - .:/code
 depends_on:
 - db
 - redis

celery-beat:
 build: .
 command: celery -A proj beat -l info
 volumes:
 - .:/code
 depends_on:
 - db
 - redis

celery_1 | [2018-02-01 22:44:00,957: WARNING/ForkPoolWorker-1] Hello there!

 
 
 And there you have it! For even more fun, you might try:
 
 
 
 Adding more complex tasks, like tasks that take arguments
 
 
 Setting up different schedules
 
 
 Setting up a non-periodic task
 
 
 
 What next?
 
 
 Review the
 
 Celery Tasks Checklist
 
 for a great introduction to Celery best practices. In particular, pay attention to:
 
 
 
 Set a
 
 retry_limit
 
 so that failed tasks don’t keep trying forever.
 
 
 Set a high default
 
 task_time_limit
 
 to avoid tasks that block the entire Celery worker.
 
 
 For tasks that need to take in a Django model object as a parameter, pass in a primary key and not the object itself.
 
 
 Do a lot of logging!
 
 
 
 You will also want to monitor your tasks for success or failure. A great tool for this is
 
 Flower
 
 , Celery’s monitoring tool. Flower will show you a dashboard of all your workers and tasks and let you drill down into specific tasks, show you task statistics, let you restart workers, and let you rate-limit tasks (among many other things).
 
 
 If you use an error-tracking system like
 
 Rollbar
 
 or
 
 Sentry
 
 , you can also set Celery up to report exceptions to those services.
 
 
 Further Reading
 
 
 
 
 Celery Task Checklist
 
 
 
 
 Message Queues and Async Workers
 
 , Frank Wiles
 
 
 
 Celery Docs - Periodic Tasks
 
 
 
 
 Celery Docs - First steps with Django
 
 
 
 
 Celery Docs - task_always_eager
 
 
 
 
 Celery Docs - Logging
 
 
 
 
 Celery 4 Periodic Task in Django
 
 , Yehan Djoehartono
 
 
 
 Asynchronous Tasks with Django and Celery
 
 (Celery 3.1.18)
 
 
 
 
 * Thanks to
 
 kurashu89
 
 for their correction on an earlier version of this article.

Celery and Django and Docker: Oh My!

django-test-plus has long been a useful helper library with Django. Now we&#x27;ve added some simple pytest fixture support to make it even more useful!

<div class="w-block-content block-content">
 
 We're obviously fans of
 <a href="https://github.com/revsys/django-test-plus">
 django-test-plus
 </a>
 , having written it, and have found it very valuable over the last few years at cutting down on all of the boilerplate test code we have to write with our clients.
 
 
 
 
 It even helps sell certain stubborn clients on the idea that testing can be done in a not so painful way!
 
 
 Today we're happy to announce that you can now use django-test-plus as a real pytest fixture. Here's a quick example:
 
</div>
<div class="w-block-code block-code">
 <div class="codehilite">
 <pre>def test_some_view(tp):
 tp.get('some-view-name')
 tp.response_200()


def test_some_url_stuff(tp):
 expected_url = '/api/'
 reversed_url = tp.reverse('api')
 assert expected_url == reversed_url
</pre>
 </div>
</div>
<div class="w-block-content block-content">
 
 And you get all of the other test plus methods as part of that "tp" fixture. This is a new experimental feature, so we'd love to hear how we can make this better and more "pytest like".
 
 
 
 
 To start using all you need to do is install it and pull in "tp" as a fixture, you install it with just:
 
</div>
<div class="w-block-code block-code">
 <div class="codehilite">
 <pre>pip install django-test-plus
</pre>
 </div>
</div>

def test_some_view(tp):
    tp.get('some-view-name')
    tp.response_200()


def test_some_url_stuff(tp):
    expected_url = '/api/'
    reversed_url = tp.reverse('api')
    assert expected_url == reversed_url

 
pip install django-test-plus

django-test-plus has long been a useful helper library with Django. Now we've added some simple pytest fixture support to make it even more useful!

pytest support for django-test-plus

With companies moving to microservices, it's becoming common to have a system scattered across multiple repositories. It's frequent to abstract common patterns and code into private repositories that are then included in each service. But using packages from private repos with Python can be tricky. This guide will guide you through the necessary steps.

<div class="w-block-markdown block-markdown">
 <div class="prose prose-lg max-w-none">
 
 With companies moving to microservices, it's becoming common to have a system scattered across multiple repositories. It's frequent to abstract common patterns and code into private repositories that are then included in each service.
 
 
 But using packages from private repos with Python can be tricky. This guide will walk you through the necessary steps to correctly use private packages with
 <code>
 pip
 </code>
 or
 <code>
 pipenv
 </code>
 . We have also built
 <a href="https://revsys.com/pip-private-packages/">
 a little app
 </a>
 that will generate the right incantations for you to copy and paste.
 
 <h2>
 Figuring out the URL
 </h2>
 
 First, you need to choose which format you want to use: tarball (ie:
 <code>
 .zip
 </code>
 ) or vcs (most likely
 <code>
 git
 </code>
 ). I prefer just downloading the tarball because it doesn't require any additional vcs software. But if use
 <code>
 pipenv
 </code>
 , you'll have to use vcs (see below).
 
 <h3>
 Authentication
 </h3>
 
 In order to have access to the private repo or tarball, you'll need an authentication token. GitHub and other providers allow you to create a private access token for your user, but because we are going to hardcode such token in code, I strongly recommend to create a 'bot' user and use its access token.
 
 <ul>
 <li>
 Github: access token at
 <code>
 https://github.com/settings/tokens
 </code>
 . You can select just the
 <code>
 repo
 </code>
 scopes.
 </li>
 <li>
 Githost: deploy token at
 <code>
 https://&lt;myorg&gt;.githost.io/&lt;org&gt;/&lt;project&gt;/settings/repository
 </code>
 . You can just select
 <code>
 read_repository
 </code>
 scope.
 </li>
 <li>
 Gitlab: deploy token at
 <code>
 https://gitlab.com/profile/&lt;org&gt;/&lt;project&gt;/settings/repository
 </code>
 . You can just select
 <code>
 read_repository
 </code>
 scope.
 </li>
 <li>
 Bitbucket: access token at
 <code>
 https://bitbucket.org/account/user/&lt;botusername&gt;/app-passwords
 </code>
 . You can just select
 <code>
 repositories/read
 </code>
 scope.
 </li>
 </ul>
 <h3>
 Versioning
 </h3>
 
 You will also need to decide which version of your package you want to install. We'll call this the
 <code>
 ref
 </code>
 . You could target a branch, a commit, or a tag/version. I prefer to use version because it gives me a human-readable point of reference.
 
 <h2>
 Building the url
 </h2>
 
 Once you have your access token and your ref, the next step is figuring out a URL at which
 <code>
 pip
 </code>
 can download or clone the code.
 
 <h3>
 Github
 </h3>
 <ul>
 <li>
 Tarball URL will be
 <code>
 https://&lt;access_token&gt;@github.com/&lt;myorg&gt;/&lt;myproject&gt;/archive/&lt;ref&gt;.zip
 </code>
 </li>
 <li>
 VCS Tarball will be
 <code>
 git+https://&lt;access_token&gt;@github.com/&lt;myorg&gt;/&lt;myproject&gt;.git@&lt;ref&gt;
 </code>
 </li>
 </ul>
 <h3>
 Githost
 </h3>
 <ul>
 <li>
 Tarball URL will be
 <code>
 https://&lt;myorg&gt;.githost.io/&lt;myteam&gt;/&lt;myproject&gt;/repository/archive.zip?private_token=&lt;access_token&gt;&amp;ref=&lt;ref&gt;
 </code>
 </li>
 <li>
 VCS Tarball will be
 <code>
 git+https://&lt;token_username&gt;:&lt;deploy_token@&lt;myorg&gt;.githost.io/&lt;myteam&gt;/&lt;myproject&gt;.git@&lt;ref&gt;
 </code>
 </li>
 </ul>
 <h3>
 Gitlab
 </h3>
 <ul>
 <li>
 Tarball URL will be
 <code>
 https://gitlab.com/&lt;myteam&gt;/&lt;myproject&gt;/repository/archive.zip?private_token=&lt;access_token&gt;&amp;ref=&lt;ref&gt;
 </code>
 </li>
 <li>
 VCS Tarball will be
 <code>
 git+https://&lt;token_username&gt;:&lt;deploy_token&gt;@gitlab.com/&lt;myorg&gt;/&lt;myproject&gt;.git@&lt;ref&gt;
 </code>
 </li>
 </ul>
 <h3>
 Bitbucket
 </h3>
 <ul>
 <li>
 Tarball URL will be
 <code>
 https://&lt;botusername&gt;:&lt;access_token&gt;@bitbucket.org/&lt;myorg&gt;/&lt;myproject&gt;/get/&lt;ref&gt;.zip
 </code>
 </li>
 <li>
 VCS Tarball will be
 <code>
 git+https://&lt;botusername&gt;:&lt;access_token&gt;@bitbucket.org/&lt;myorg&gt;/&lt;myproject&gt;.git@&lt;ref&gt;
 </code>
 </li>
 </ul>
 <h2>
 Specifying the dependency
 </h2>
 <h3>
 <code>
 pip
 </code>
 requirements file
 </h3>
 
 Once you have the URL, you can just add it to you requirements files, or use with
 <code>
 pip install
 </code>
 :
 
 <div class="codehilite">
 <pre><code>$ pip install "&lt;URL&gt;" # You'll need the quotes to escape the URL
</code></pre>
 </div>
 <h3>
 <code>
 Pipfile
 </code>
 </h3>
 
 To add the URL to you
 <code>
 Pipfile
 </code>
 , use this syntax:
 
 <div class="codehilite">
 <pre><code>[packages]
revsys-teams = {file = "&lt;URL&gt;"}
</code></pre>
 </div>
 <h3>
 Private dependency of a private package
 </h3>
 
 If you need to specify your private package as a dependency of another package, you'll have to add the URL in your the dependency's
 <code>
 setup.py
 </code>
 . Note that the exact syntax of the specification is a little tricky:
 
 <div class="codehilite">
 <pre><code>import os
from setuptools import setup, find_packages

setup(
 name='upstreamprivatepackage',
 install_requires=[
 # The 'version' is required. Doesn't have to be an actual version of the package.
 # but it must match what's in the `egg` fragment of its entry in `dependency_links`.
 "&lt;myproject&gt;==0.0.1",
 ],
 dependency_links=[
 # there must be a version identifier in the `egg=` fragment and it must match what's
 # in `install_requires`
 '&lt;url&gt;#egg=&lt;myproject&gt;-0.0.1',
 ],
)
</code></pre>
 </div>
 
 When installing the upstream package, you'll also have to tell
 <code>
 pip
 </code>
 to actually parse the entries in
 <code>
 dependency_links
 </code>
 :
 
 <div class="codehilite">
 <pre><code># `pip` has to be run with the `PIP_PROCESS_DEPENDENCY_LINKS` env var or with `--process-dependency-links` option.
# As of v9.0.1, `pip` will print a big red scary deprecation warning, but that's the only way.
PIP_PROCESS_DEPENDENCY_LINKS=1 pip install ...

# or
pip install ... --process-dependency-links
</code></pre>
 </div>
 </div>
</div>

$ pip install "<URL>" # You'll need the quotes to escape the URL

[packages]
revsys-teams = {file = "<URL>"}

import os
from setuptools import setup, find_packages

setup(
 name='upstreamprivatepackage',
 install_requires=[
 # The 'version' is required. Doesn't have to be an actual version of the package.
 # but it must match what's in the `egg` fragment of its entry in `dependency_links`.
 "<myproject>==0.0.1",
 ],
 dependency_links=[
 # there must be a version identifier in the `egg=` fragment and it must match what's
 # in `install_requires`
 '<url>#egg=<myproject>-0.0.1',
 ],
)

# `pip` has to be run with the `PIP_PROCESS_DEPENDENCY_LINKS` env var or with `--process-dependency-links` option.
# As of v9.0.1, `pip` will print a big red scary deprecation warning, but that's the only way.
PIP_PROCESS_DEPENDENCY_LINKS=1 pip install ...

# or
pip install ... --process-dependency-links

Using Private Packages in Python

Lacey didn&#x27;t have the opportunity to work with Docker at her last job. In this tidbit she steps you through getting started with Docker for Django developers.

FROM python:3.6

ENV PYTHONUNBUFFERED 1
ENV DJANGO_ENV dev
ENV DOCKER_CONTAINER 1

COPY ./requirements.txt /code/requirements.txt
RUN pip install -r /code/requirements.txt

COPY . /code/
WORKDIR /code/

EXPOSE 8000

 
FROM python:3.6

 
ENV PYTHONUNBUFFERED 1

 
ENV DJANGO_ENV dev

 
ENV DOCKER_CONTAINER 1

 
COPY ./requirements.txt /code/requirements.txt

 
RUN pip install -r /code/requirements.txt

 
COPY . /code/

 
WORKDIR /code/

 
EXPOSE 8000

 
version: '3'

services:
  db:
    image: postgres:9.6.5
    volumes:
      - postgres_data:/var/lib/postgresql/data/
  web:
    build: .
    command: bash -c "python /code/manage.py migrate --noinput && python /code/manage.py runserver 0.0.0.0:8000"
    volumes:
      - .:/code
    ports:
      - "8000:8000"
    depends_on:
      - db

volumes:
  postgres_data:

 
version: '3'

 
services:

 
db:
    image: postgres:9.6.5
    volumes:
      - postgres_data:/var/lib/postgresql/data/

 
web:
    build: .
    command: bash -c "python /code/manage.py migrate --noinput && python /code/manage.py runserver 0.0.0.0:8000"
    volumes:
      - .:/code
    ports:
      - "8000:8000"
    depends_on:
      - db

 
volumes:
      - .:/code

 
ports:
      - "8000:8000"

 
depends_on:
      - db

 
volumes:
  postgres_data:

 


 
  Finally, Compose has a rule where you have to list your named volumes in a top-level
  
   volumes
  
  key, so we have done that.
 
 
  Save the docker-compose.yml file.
  
 
 
  In Terminal or your console and from the same directory that contains your Dockerfile and docker-compose.yml file, run
  
   docker-compose up
  
  .
  
 
 
  Assuming you have no errors, navigate to
  
   http://localhost:8000/
  
  in a browser and see your blue screen of success once again!
  
 
 
  Ready for your next step? Check out
  
   Docker: Useful Command Line Stuff
  
  next!
 
 
  Resources
 
 
 
 
 
 
  
   
    Get started with Docker
   
   
  
  
   
    Get started with Docker Compose
   
   
  
  
   
    Quickstart: Compose and Django
   
   
  
  
   
    Best practices for writing Dockerfiles
   
  
  
   
    Dockerizing Django, uWSGI and Postgres the Serious Way
   
   , Oliver Eidel
   
  
 
 
  
   Thanks to Frank Wiles and Jeff Triplett for reviewing drafts of this post.

Lacey didn't have the opportunity to work with Docker at her last job. In this tidbit she steps you through getting started with Docker for Django developers.

A Brief Intro to Docker for Djangonauts

Smaller and faster Python 3.6.x images for you to take advantage of.

<div class="w-block-content block-content">
 
 Turns out there are some optimizations you can do when compiling Python 3.6 and Python 3.7 that give you some significant speed improvements without any real downside.
 
 
 We've paired with with Google's base Debian image that rips out systemd and all it's dependencies, which you weren't going to be using in a Docker container anyway, which yields a MUCH smaller image.
 
 
 How do you use it? Pretty simple just start your Dockerfile with:
 
</div>
<div class="w-block-code block-code">
 <div class="codehilite">
 <pre>FROM revolutionsystems/python:3.7.1-wee-optimized-lto
</pre>
 </div>
</div>
<div class="w-block-content block-content">
 
 And rebuild your image.
 
 
 How much faster is it? Depending on the benchmark in question it's up to 19% faster and nearly 75% smaller than the official Python images.
 
 
 You can get more details on benchmarks and how all this works on the
 <a href="https://hub.docker.com/r/revolutionsystems/python/">
 dockerhub page
 </a>
 . Information on
 <a href="https://github.com/revsys/optimized-python-docker">
 build details
 </a>
 is also available.
 
 
 Enjoy!
 
 
 Need help effectively using Docker with Python or Django? We can help!
 <a href="https://www.revsys.com/contact/">
 Contact us
 </a>
 or
 <a href="mailto:sales@revsys.com">
 send us a quick email
 </a>
 .
 
</div>

FROM revolutionsystems/python:3.7.1-wee-optimized-lto

 


 
  And rebuild your image.
 
 
  How much faster is it? Depending on the benchmark in question it's up to 19% faster and nearly 75% smaller than the official Python images.
 
 
  You can get more details on benchmarks and how all this works on the
  
   dockerhub page
  
  . Information on
  
   build details
  
  is also available.
 
 
  Enjoy!
 
 
  Need help effectively using Docker with Python or Django? We can help!
  
   Contact us
  
  or
  
   send us a quick email
  
  .

Optimized Python

Sometimes you just need to run a command on a remote host with ssh and Python 3 without any real fuss. Here is the simplest way we have found to do that and retrieve the output. 

<div class="w-block-content block-content">
 
 <a href="http://www.fabfile.org/">
 Fabric
 </a>
 is great, but isn't 100% compatible with Python 3. &nbsp;There is
 <a href="https://github.com/mathiasertl/fabric/">
 Fabric3
 </a>
 , which does work, but sometimes you don't want to have to issue
 
 `fab &lt;something&gt;`
 
 &nbsp;but instead want to issue a command as part of a larger bit of Python. &nbsp;
 
 
 I ran into this situation today where I wanted to issue a few commands over ssh to a remote host as part of a
 <a href="http://click.pocoo.org/6/">
 Click
 </a>
 command I'm building to do some ops automation here at REVSYS.&nbsp;
 
 
 It's probably not perfect in terms of error handling, but it sure is simple! Here is all you need to do:
 
</div>
<div class="w-block-code block-code">
 <div class="codehilite">
 <pre>from fabric.api import env
from fabric.operations import run as fabric_run
from fabric.context_managers import settings, hide

def run(user, host, command):
 """ Run a command on the host. Assumes user has SSH keys setup """
 env.user = user
 env.use_ssh_config = True

 with settings(hide('everything'), host_string=host):
 results = fabric_run(command)

 return results
</pre>
 </div>
</div>
<div class="w-block-content block-content">
 
 You'll need to install the Fabric3 library with a simple:
 
</div>
<div class="w-block-code block-code">
 <div class="codehilite">
 <pre>pip install Fabric3
</pre>
 </div>
</div>
<div class="w-block-content block-content">
 
 And away you go! All this is doing is re-using Fabric3's run() function while silencing all of the normal Fabric style output and forcing the user, host, and to use the calling environment's ssh configuration settings (like your ~/.ssh/config). &nbsp;
 
 
 We could accomplish this more directly with Paramiko or work some subprocess magic to call our ssh, but this was the simplest and fastest way I solved the ssh problem today.
 
 
 Hope you find this little trick useful! Happy Hacking!&nbsp;
 
</div>

from fabric.api import env
from fabric.operations import run as fabric_run
from fabric.context_managers import settings, hide

def run(user, host, command):
    """ Run a command on the host. Assumes user has SSH keys setup """
    env.user = user
    env.use_ssh_config = True

    with settings(hide('everything'), host_string=host):
        results = fabric_run(command)

    return results

 
pip install Fabric3

 


 
  And away you go! All this is doing is re-using Fabric3's run() function while silencing all of the normal Fabric style output and forcing the user, host, and to use the calling environment's ssh configuration settings (like your ~/.ssh/config).  
 
 
  We could accomplish this more directly with Paramiko or work some subprocess magic to call our ssh, but this was the simplest and fastest way I solved the ssh problem today.
 
 
  Hope you find this little trick useful! Happy Hacking!

Sometimes you just need to run a command on a remote host with ssh and Python 3 without any real fuss. Here is the simplest way we have found to do that and retrieve the output.

Python 3 run command over SSH

Python's atexit handler is one of those things people should use more often.  Some developers aren't even aware that it exists and that makes me sad. 

<div class="w-block-content block-content">
 
 Did you know Python can automatically do things for you when the process exits? The
 <a href="https://docs.python.org/3/library/atexit.html#atexit-example">
 atexit
 </a>
 &nbsp;is one of those handy, but often forgotten about, aspects of Python. &nbsp;I had cause to use it yet again today with a certain process that was being naughty. &nbsp;
 
 
 The idea is simple, atexit&nbsp;instructs Python to run your function just before the process exits. &nbsp;So when would you want to do that? I'm sure there are other situations, but I typically end up using them when:
 
 
 
 <ul>
 <li>
 I want to log both process startup AND exit
 </li>
 <li>
 Alert someone that a process has died, especially cron jobs that are easily forgotten about
 </li>
 </ul>
 
 A quick example to show you the idea:
 
 
 
</div>
<div class="w-block-code block-code">
 <div class="codehilite">
 <pre>import atexit


@atexit.register
def closing():
 print("=== Closing down ===")


while True:
 print("whee!")
</pre>
 </div>
</div>
<div class="w-block-content block-content">
 
 Using the atexit decorator, we wrap a little function that just prints out that we're closing down. Our while loop, er I mean our
 
 whee loop,
 
 will run until we interrupt the process with a Ctrl-C. &nbsp;The output looks like this:
 
</div>
<div class="w-block-code block-code">
 <div class="codehilite">
 <pre>whee!
whee!
whee!
whee!
whee!
whee!
whee!
whee!
whee!
whee!
whee!
whee!
whee^Cwhee!
whee!
Traceback (most recent call last):
 File "example.py", line 10, in &lt;module&gt;
 print("whee!")
KeyboardInterrupt
=== Closing down ===
</pre>
 </div>
</div>
<div class="w-block-content block-content">
 
 We see our usual KeyboardInterrupt traceback and then our custom closing message.&nbsp;
 
 
 You can also register your atexit functions without the decorator so you can pass args and kwargs to it to be flexible. &nbsp;
 
 
 Seems pretty perfect, doesn't it? It's not all roses. &nbsp;Python only calls these atexit&nbsp;functions in a couple of scenarios, specifically if:
 
 
 
 <ul>
 <li>
 sys.exit() is called
 </li>
 <li>
 The process completes and is exiting normally
 </li>
 </ul>
 
 However, it&nbsp;
 
 does not
 
 &nbsp;call your atexit handles if Python is killed by a signal, when Python itself has a fatal internal error, or if os._exit() is called directly. &nbsp;So it's not useful to detect kills or kill -9s for sure, but in most other usual exit scenarios it can be handy.&nbsp;
 
 
 Hope you find a use for it!&nbsp;
 
 
 
</div>

import atexit


@atexit.register
def closing():
 print("=== Closing down ===")


while True:
 print("whee!")

 
whee!
whee!
whee!
whee!
whee!
whee!
whee!
whee!
whee!
whee!
whee!
whee!
whee^Cwhee!
whee!
Traceback (most recent call last):
 File "example.py", line 10, in <module>
 print("whee!")
KeyboardInterrupt
=== Closing down ===

 


 
 We see our usual KeyboardInterrupt traceback and then our custom closing message. 
 
 
 You can also register your atexit functions without the decorator so you can pass args and kwargs to it to be flexible.  
 
 
 Seems pretty perfect, doesn't it? It's not all roses.  Python only calls these atexit functions in a couple of scenarios, specifically if:
 
 
 
 
 
 sys.exit() is called
 
 
 The process completes and is exiting normally
 
 
 
 However, it 
 
 does not
 
  call your atexit handles if Python is killed by a signal, when Python itself has a fatal internal error, or if os._exit() is called directly.  So it's not useful to detect kills or kill -9s for sure, but in most other usual exit scenarios it can be handy. 
 
 
 Hope you find a use for it!

Python's atexit handler is one of those things people should use more often.  Some developers aren't even aware that it exists and that makes me sad.

Python: atexit

Getting your configuration from the system environment for your Python and/or Django apps is often the best way to provide security and flexibility. envparse makes it easy. 

<div class="w-block-content block-content">
 
 Making&nbsp;
 <a href="https://12factor.net/">
 12-Factor apps
 </a>
 &nbsp;is all the rage these days and not without good reasons. Using the system environment to get your app's configuration is probably the most flexible way to pass configuration information once you get used to it.
 
 
 One mental barrier I know I had was I didn't want to
 
 
 always
 
 
 &nbsp;have to pass a bunch of variables just to run my code. The ideal setup is:
 
 
 
 <ul>
 <li>
 Load the env vars from a file in the current directory or any directory above
 
 </li>
 <li>
 Have any variables from the actual environment included and give the ability to override them on a per execution basis on the command line
 </li>
 </ul>
 
 Luckily, the
 <a href="https://pypi.python.org/pypi/envparse/0.2.0">
 envparse
 </a>
 &nbsp;library for Python gives us this pretty easily. &nbsp;There are two ways you can use it. &nbsp;The "standard" way or more manual way. &nbsp;Or you can define up front a schema of variables you expect and ensure they are cast to the types you need.&nbsp;
 
 
 Let's explore the standard way first.&nbsp;
 
 
 
</div>
<div class="w-block-code block-code">
 <div class="codehilite">
 <pre>from envparse import env

env.read_envfile()
</pre>
 </div>
</div>
<div class="w-block-content block-content">
 
 That's all you need to do to load up the environment. The library will walk your file system path looking for a .env file, if it finds one it will load it. &nbsp;If it doesn't it will issue a warning you can safely ignore.&nbsp;
 
 
 You're on your own for validation, however. &nbsp;If you go to access a variable that doesn't exist you'll get a KeyError.&nbsp;
 
 
 Let's assume we have the following in a .env&nbsp;in the current directory:
 
</div>
<div class="w-block-code block-code">
 <div class="codehilite">
 <pre>AWS_ACCESS_KEY='blahblahblahblah'
AWS_SECRET_KEY='WeAreNotThatDumb'
CONSULTING_LEVEL='Pro'
</pre>
 </div>
</div>
<div class="w-block-content block-content">
 
 If we then called our script like this:
 
</div>
<div class="w-block-code block-code">
 <div class="codehilite">
 <pre>OTHER='bar' CONSULTING_LEVEL='Expert' python envparse-example.py
</pre>
 </div>
</div>
<div class="w-block-content block-content">
 
 Now let's look at envparse-example.py, what do you think it would output?
 
</div>
<div class="w-block-code block-code">
 <div class="codehilite">
 <pre>from envparse import env

env.read_envfile()

print('ACCESS =', env.str('AWS_ACCESS_KEY'))
print('SECRET =', env.str('AWS_SECRET_KEY'))
print('CONSULTING_LEVEL =', env.str('CONSULTING_LEVEL'))
print('OTHER =', env.str('OTHER', default='foo'))
</pre>
 </div>
</div>
<div class="w-block-content block-content">
 
 You win a cookie or something if you correctly guessed:
 
</div>
<div class="w-block-code block-code">
 <div class="codehilite">
 <pre>ACCESS = AKIAJI4KRV67OPXQZS7Q
SECRET = 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
CONSULTING_LEVEL = Expert
OTHER = bar
</pre>
 </div>
</div>
<div class="w-block-content block-content">
 
 I want to point out two things about this. First, as you probably already understood the fact we have 'CONSULTING_LEVEL' set in both the file and on the command line, the command line version takes precedence.&nbsp;
 
 
 The second thing I want to point out is the last line in the source above. &nbsp;We overrode 'OTHER' on the command line, but if we hadn't it would be set to the default of 'foo'. &nbsp;This helps allow you to set sane defaults in the absence of the variable being set anywhere at all.&nbsp;
 
 <h2>
 envparse&nbsp;using a Schema
 </h2>
 
 Now that we've covered the more manual and direct way of using envparse, let's play with the Schema support real quick.&nbsp;
 
</div>
<div class="w-block-code block-code">
 <div class="codehilite">
 <pre>from envparse import Env

env = Env(
 AWS_ACCESS_KEY=str,
 AWS_SECRET_KEY=str,
 CONSULTING_LEVEL=str,
 OTHER=dict(cast=str, default='foo')

)
env.read_envfile()

print('ACCESS =', env.str('AWS_ACCESS_KEY'))
print('SECRET =', env.str('AWS_SECRET_KEY'))
print('CONSULTING_LEVEL =', env.str('CONSULTING_LEVEL'))
print('OTHER =', env.str('OTHER'))
</pre>
 </div>
</div>
<div class="w-block-content block-content">
 
 Assuming the same .env file and overrides on the command line this will produce the exact same results. The different here is we import 'Env' (note the capitalization) and set the variables we expect to have.&nbsp;
 
 
 If something is missing from our environment, envparse&nbsp;raises a envparse.ConfigurationError rather than getting the more generic KeyError when we go to access to a missing value.
 
 
 Using a schema like this also gives us options for settings defaults as part of the definition rather than when calling for the value itself. &nbsp;envparse has a few other useful options for casting more complex types and giving you the ability to pre or post process the values when they are read which can be useful to help normalize values. &nbsp;
 
 <h2>
 Using envparse with Click
 </h2>
 
 I'm a huge fan of
 <a href="http://click.pocoo.org/6/">
 Click
 </a>
 &nbsp;and while it has great support for
 <a href="http://click.pocoo.org/6/options/#values-from-environment-variables">
 pulling values out of the environment for you
 </a>
 , it doesn't support using a .env&nbsp;file directly. &nbsp;Lucky for us it's straightforward to use envparse with Click. If you need options that can come from a .env file, the environment directly, or passed on the command line you can do this:
 
 
 
 
</div>
<div class="w-block-code block-code">
 <div class="codehilite">
 <pre>import click

from envparse import Env

env = Env(
 AWS_ACCESS_KEY=str,
 AWS_SECRET_KEY=str,
 CONSULTING_LEVEL=str,
 OTHER=dict(cast=str, default='foo')

)
env.read_envfile()

@click.command()
@click.option('--access-key', default=env.str('AWS_ACCESS_KEY'))
@click.option('--secret-key', default=env.str('AWS_SECRET_KEY'))
def testing(access_key, secret_key):
 print(access_key)
 print(secret_key)

if __name__ == '__main__':
 testing()
</pre>
 </div>
</div>
<div class="w-block-content block-content">
 
 The trick here is to use envparse to read in your expected values, whether they come from a .env file or are passed on in via the environment it doesn't matter. &nbsp;We then use the value from envparse to set the default for Click. &nbsp;If we need to, we can still override this on the command line and everyone is happy.
 
 
 Hope this helps you make your Python app's configuration more flexible!&nbsp;
 
</div>

from envparse import env

env.read_envfile()

 
AWS_ACCESS_KEY='blahblahblahblah'
AWS_SECRET_KEY='WeAreNotThatDumb'
CONSULTING_LEVEL='Pro'

 
OTHER='bar' CONSULTING_LEVEL='Expert' python envparse-example.py

 
from envparse import env

env.read_envfile()

print('ACCESS =', env.str('AWS_ACCESS_KEY'))
print('SECRET =', env.str('AWS_SECRET_KEY'))
print('CONSULTING_LEVEL =', env.str('CONSULTING_LEVEL'))
print('OTHER =', env.str('OTHER', default='foo'))

 
ACCESS = AKIAJI4KRV67OPXQZS7Q
SECRET = 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
CONSULTING_LEVEL = Expert
OTHER = bar

 
from envparse import Env

env = Env(
    AWS_ACCESS_KEY=str,
    AWS_SECRET_KEY=str,
    CONSULTING_LEVEL=str,
    OTHER=dict(cast=str, default='foo')

)
env.read_envfile()

print('ACCESS =', env.str('AWS_ACCESS_KEY'))
print('SECRET =', env.str('AWS_SECRET_KEY'))
print('CONSULTING_LEVEL =', env.str('CONSULTING_LEVEL'))
print('OTHER =', env.str('OTHER'))

 
import click

from envparse import Env

env = Env(
    AWS_ACCESS_KEY=str,
    AWS_SECRET_KEY=str,
    CONSULTING_LEVEL=str,
    OTHER=dict(cast=str, default='foo')

)
env.read_envfile()

@click.command()
@click.option('--access-key', default=env.str('AWS_ACCESS_KEY'))
@click.option('--secret-key', default=env.str('AWS_SECRET_KEY'))
def testing(access_key, secret_key):
    print(access_key)
    print(secret_key)

if __name__ == '__main__':
    testing()

 


 
  The trick here is to use envparse to read in your expected values, whether they come from a .env file or are passed on in via the environment it doesn't matter.  We then use the value from envparse to set the default for Click.  If we need to, we can still override this on the command line and everyone is happy.
 
 
  Hope this helps you make your Python app's configuration more flexible!

Getting your configuration from the system environment for your Python and/or Django apps is often the best way to provide security and flexibility. envparse makes it easy.

Python 12-factor apps with envparse

Have a URL but can't figure out what view it is supposed to use? Use resolve() to easily and quickly determine it just as Django does.

<div class="w-block-content block-content">
 
 One common practice in Django is reversing URL:
 
 
 
</div>
<div class="w-block-code block-code">
 <div class="codehilite">
 <pre>from django.urls import reverse

reverse('my_url_name', kwargs={'pk': 1})
</pre>
 </div>
</div>
<div class="w-block-content block-content">
 
 But very often, in large projects, it's hard to remember exactly what the name of the URL is. Furthermore, you maybe using 3rd party libraries that include URLs you want to refer to.
 
 
 If you know what the URL looks like, but you can't remember the name or its arguments, a quick and easy way to find them out is use the `resolve` function:
 
 
 
</div>
<div class="w-block-code block-code">
 <div class="codehilite">
 <pre>from django.urls import resolve

resolve('/myurl/2')
# ResolverMatch(func=myapp.views.MyModelViewSet, args=(), kwargs={u'pk': u'2'}, url_name=mymodel-detail, app_name=None, namespaces=['api'])
</pre>
 </div>
</div>
<div class="w-block-content block-content">
 
 As you can see, the output includes all you need to&nbsp;reverse&nbsp;the URL, including arguments,&nbsp;app_name&nbsp;and&nbsp;namespaces.
 
</div>
<div class="w-block-html block-html">
 <div class="block-content">
 <div class="rich-text">
 <h6>
 UPDATE
 </h6>
 
 The
 <code>
 django-extensions
 </code>
 package provides another easy way to find out which view. After installing it, you can simply use its
 <code>
 show_urls
 </code>
 and
 <code>
 grep
 </code>
 ping for the URL you're looking for:
 
 </div>
 </div>
</div>
<div class="w-block-code block-code">
 <div class="codehilite">
 <pre>$ ./manage.py show_urls | grep "/myurl/&lt;pk&gt;"
</pre>
 </div>
</div>
<div class="w-block-content block-content">
 
 Thanks to
 
 @bmihelac
 
 &nbsp;for the suggestion!
 
 
 
 
</div>

from django.urls import reverse

reverse('my_url_name', kwargs={'pk': 1})

 
from django.urls import resolve

resolve('/myurl/2')
# ResolverMatch(func=myapp.views.MyModelViewSet, args=(), kwargs={u'pk': u'2'}, url_name=mymodel-detail, app_name=None, namespaces=['api'])

 
$ ./manage.py show_urls | grep "/myurl/<pk>"

 


 
 Thanks to
 
 @bmihelac
 
  for the suggestion!

reverse()'s soulmate resolve()

You can use a variable to set the name of the template you are extending when using Django templates.

<div class="w-block-content block-content">
 
 One of my all favorite template tricks goes back to my LJ World days.
 
 
 Most people don't realize that the&nbsp;
 <a href="https://docs.djangoproject.com/en/1.9/ref/templates/builtins/#extends">
 extends
 </a>
 &nbsp;tag accepts a variable and not just a hard-coded literal value. This allows for magical things to happen when combined with a default.
 
 
 
 The following block extends a base template named "base.html" if the&nbsp;
 
 base_template
 
 &nbsp;is not set.
 
</div>
<div class="w-block-code block-code">
 <div class="codehilite">
 <pre>{% extends base_template|default:"base.html" %}
</pre>
 </div>
</div>
<div class="w-block-content block-content">
 
 For any page that you want to programmatically override the base template, you may do so by adding a&nbsp;base_template context variable to your views context values which are exposed to your template.
 
</div>
<div class="w-block-code block-code">
 <div class="codehilite">
 <pre>from django.views.generic import TemplateView


class MarketingView(TemplateView):

 template_name = 'marketing_landing.html'

 def get_context_data(self, **kwargs):
 context = super(MarketingView, self).get_context_data(**kwargs)
 context['base_template'] = 'base_marketing.html'
 return context
</pre>
 </div>
</div>
<div class="w-block-content block-content">
 
 This simple trick opens the door to do fun things with your website based on the day of the year or something useful like A/B test two different versions of your website's landing page. The possibilities are endless.
 
</div>

{% extends base_template|default:"base.html" %}

 
from django.views.generic import TemplateView


class MarketingView(TemplateView):

    template_name = 'marketing_landing.html'

    def get_context_data(self, **kwargs):
        context = super(MarketingView, self).get_context_data(**kwargs)
        context['base_template'] = 'base_marketing.html'
        return context

 


 
  This simple trick opens the door to do fun things with your website based on the day of the year or something useful like A/B test two different versions of your website's landing page. The possibilities are endless.

Extend Django Templates Smartly

Our recap of the Django&#x27;s 10th Birthday Party

<div class="w-block-markdown block-markdown">
 <div class="prose prose-lg max-w-none">
 <h2>
 Happy 10th Birthday Django!
 </h2>
 
 <img alt="" src="http://photos.frankwiles.com/Events/Django-Birthday/i-pRMN7Sr/0/M/150711jasondailey841-M.jpg"/>
 
 
 Whew, what a conference! The talks were great, the venue was comfortable, and the BBQ was amazing. But that was only part of it. Many of the talks focused on the Django community and you could feel it in the room. We laughed from our bellies and teared up each time someone mentioned
 <a href="https://www.djangoproject.com/weblog/2013/sep/16/announcing-malcolm-tredinnick-memorial-prize/">
 Malcolm
 </a>
 .
 
 
 I had the honor of giving the closing address, which was really just a thinly veiled way for me to have a Steve Jobs "oh and one more thing" moment. We had secretly conspired with The World Company, the place where Django was born, to open up the building so all of the attendees could get a quick tour and see
 
 EXACTLY
 
 where Django originated. Liberty Hall, where we held the conference, is conveniently across the alley from the Journal-World building so we all stood up, grabbed our bags, and walked over. The photo above is most of us in front of the News Center building where Django was born.
 
 
 The night before the conference we had a very loosely planned evening of dinner and drinks which also included a quick Skype session with the Django Girls group in Australia to kick off the celebration. It was a great relaxed way to catch up with old friends.
 
 <h2>
 Videos
 </h2>
 
 The
 <a href="http://pyvideo.org/category/71/django-birthday">
 videos are up on PyVideo.org
 </a>
 for you all to enjoy. As always
 <a href="http://nextdayvideo.com/">
 Next Day Video
 </a>
 had them up so fast it was a constant stream of "OMG my talk is already up" Tweets.
 
 <h2>
 Photos
 </h2>
 
 Even before Adrian mentioned in his talk about taking more photos of your projects for nostalgia purposes, we had it in our plans for Django Birthday. There is no telling if Django will make it to 2025, but assuming it does we have plenty of ammo for the next birthday party. I took a bunch of photos and we hired
 <a href="http://www.daileyimages.com/">
 Dailey Images
 </a>
 to ensure we captured all of the right moments.
 
 
 Some initial quickly edited shots from Jason Dailey, the photos I took, and some from my friend Joe Griffin are in my
 <a href="http://photos.frankwiles.com/Events/Django-Birthday/">
 Django Birthday
 </a>
 photo album.
 
 
 Kenneth Love also setup an
 <a href="https://www.icloud.com/sharedalbum/#B04GWZuqDGv43mF">
 iCloud album
 </a>
 of some of his shots. Including one of the newspaper the next day with a front page photo of the block party.
 
 <h2>
 The Party
 </h2>
 
 <img alt="" src="http://photos.frankwiles.com/Events/Django-Birthday/i-nTMNj9g/0/X3/Django_July11_2015-9-X3.jpg"/>
 
 
 Eldarion and RevSys sponsored the party with Lincoln Loop sponsoring the birthday cake.
 
 
 While it was a bit warmer than I would like, the party was epic. We had all of the things you would expect at a 10 year old geeky kids birthday party. Snow cones, balloon animals, Gizmo the robot wandering around, and of course a bouncy castle. The Batmobile even made an appearance, but that wasn't planned... it's just
 <a href="https://www.google.com/webhp?sourceid=chrome-instant&amp;ion=1&amp;espv=2&amp;es_th=1&amp;ie=UTF-8#q=%23LFK&amp;es_th=1">
 #LFK
 </a>
 
 
 For the more adult of us, we also had 3 musical performances and started the night off with Adrian himself playing Django Reinhardt. Yep that's right, the guy who created Django, playing Django, at Django's 10th birthday.
 
 
 Our friends at
 <a href="http://www.thesandbar.com">
 The SandBar
 </a>
 were awesome enough to help us throw this block party and showed everyone an amazing time. We had the usual things you would expect, but also a special Python/Django themed craft cocktail tent and this awesome shrimp &amp; rice in a pineapple food option that I have to try to recreate at home now.
 
 
 When we started planning this party Jacob made me promise to not have any ponies, but I couldn't resist one. So in addition to our big cakes for all attendees we had the baker create a small cutting cake and let Adrian have the honor of cutting the pony in half.
 
 
 I want to personally thank everyone who made the trip out to our little weird bit of the world. I hope you enjoyed yourselves and come back again very soon. Things went so well, I'm not sure I can wait until 2025. We may have to celebrate Django's Sweet 16 or something in the mean time.
 
 <h2>
 Many thanks to our great sponsors for helping us put on a great and fun event!
 </h2>
 <ul>
 <li>
 <a href="http://eldarion.com/">
 Eldarion
 </a>
 </li>
 <li>
 <a href="https://p.ota.to/">
 Potato
 </a>
 </li>
 <li>
 <a href="http://plushrugs.com/">
 PlushRugs.com
 </a>
 </li>
 <li>
 <a href="https://lincolnloop.com/">
 Lincoln Loop
 </a>
 </li>
 <li>
 <a href="https://reelio.com/">
 Reelio
 </a>
 </li>
 <li>
 <a href="https://teamtreehouse.com/">
 Treehouse
 </a>
 </li>
 </ul>
 
 And our travel sponsors:
 
 <ul>
 <li>
 <a href="https://www.python.org/psf/">
 The Python Software Foundation
 </a>
 </li>
 <li>
 <a href="https://www.djangoproject.com/">
 The Django Software Foundation
 </a>
 </li>
 </ul>
 </div>
</div>

Happy 10th Birthday Django!
  
  
   
  
  
   Whew, what a conference! The talks were great, the venue was comfortable, and the BBQ was amazing. But that was only part of it. Many of the talks focused on the Django community and you could feel it in the room. We laughed from our bellies and teared up each time someone mentioned
   
    Malcolm
   
   .
  
  
   I had the honor of giving the closing address, which was really just a thinly veiled way for me to have a Steve Jobs "oh and one more thing" moment. We had secretly conspired with The World Company, the place where Django was born, to open up the building so all of the attendees could get a quick tour and see
   
    EXACTLY
   
   where Django originated. Liberty Hall, where we held the conference, is conveniently across the alley from the Journal-World building so we all stood up, grabbed our bags, and walked over. The photo above is most of us in front of the News Center building where Django was born.
  
  
   The night before the conference we had a very loosely planned evening of dinner and drinks which also included a quick Skype session with the Django Girls group in Australia to kick off the celebration. It was a great relaxed way to catch up with old friends.
  
  
   Videos
  
  
   The
   
    videos are up on PyVideo.org
   
   for you all to enjoy. As always
   
    Next Day Video
   
   had them up so fast it was a constant stream of "OMG my talk is already up" Tweets.
  
  
   Photos
  
  
   Even before Adrian mentioned in his talk about taking more photos of your projects for nostalgia purposes, we had it in our plans for Django Birthday. There is no telling if Django will make it to 2025, but assuming it does we have plenty of ammo for the next birthday party. I took a bunch of photos and we hired
   
    Dailey Images
   
   to ensure we captured all of the right moments.
  
  
   Some initial quickly edited shots from Jason Dailey, the photos I took, and some from my friend Joe Griffin are in my
   
    Django Birthday
   
   photo album.
  
  
   Kenneth Love also setup an
   
    iCloud album
   
   of some of his shots. Including one of the newspaper the next day with a front page photo of the block party.
  
  
   The Party
  
  
   
  
  
   Eldarion and RevSys sponsored the party with Lincoln Loop sponsoring the birthday cake.
  
  
   While it was a bit warmer than I would like, the party was epic. We had all of the things you would expect at a 10 year old geeky kids birthday party. Snow cones, balloon animals, Gizmo the robot wandering around, and of course a bouncy castle. The Batmobile even made an appearance, but that wasn't planned... it's just
   
    #LFK
   
  
  
   For the more adult of us, we also had 3 musical performances and started the night off with Adrian himself playing Django Reinhardt. Yep that's right, the guy who created Django, playing Django, at Django's 10th birthday.
  
  
   Our friends at
   
    The SandBar
   
   were awesome enough to help us throw this block party and showed everyone an amazing time. We had the usual things you would expect, but also a special Python/Django themed craft cocktail tent and this awesome shrimp & rice in a pineapple food option that I have to try to recreate at home now.
  
  
   When we started planning this party Jacob made me promise to not have any ponies, but I couldn't resist one. So in addition to our big cakes for all attendees we had the baker create a small cutting cake and let Adrian have the honor of cutting the pony in half.
  
  
   I want to personally thank everyone who made the trip out to our little weird bit of the world. I hope you enjoyed yourselves and come back again very soon. Things went so well, I'm not sure I can wait until 2025. We may have to celebrate Django's Sweet 16 or something in the mean time.
  
  
   Many thanks to our great sponsors for helping us put on a great and fun event!
  
  
   
    
     Eldarion
    
   
   
    
     Potato
    
   
   
    
     PlushRugs.com
    
   
   
    
     Lincoln Loop
    
   
   
    
     Reelio
    
   
   
    
     Treehouse
    
   
  
  
   And our travel sponsors:
  
  
   
    
     The Python Software Foundation
    
   
   
    
     The Django Software Foundation

Our recap of the Django's 10th Birthday Party

Django Birthday: Recap

Django is turning 10 years old this summer and we&#x27;re throwing it a birthday party in the city where it was born.

<div class="w-block-markdown block-markdown">
 <div class="prose prose-lg max-w-none">
 
 So as you may know, Django is turning 10 years old this summer and we&rsquo;re throwing it a
 <a href="https://djangobirthday.com">
 birthday party
 </a>
 . It&rsquo;s a one day tech conference and one day of sprints mere feet from where Adrian, Simon, Jacob, and Wilson toiled away on &ldquo;
 
 the CMS
 
 &rdquo;, as it was called then. That code became the first of a few amazing refreshes of
 <a href="http://www.lawrence.com">
 Lawrence.com
 </a>
 and the
 
 very first
 
 Django website.
 
 
 We&rsquo;ve got an amazing line up of
 <a href="https://djangobirthday.com/speakers/">
 speakers
 </a>
 of course. And we want to celebrate Django obviously, but secretly we have another small motivation. See we love Lawrence, despite the fact it&rsquo;s surrounded by Kansas, and we&rsquo;ve grown tired of answering questions along the lines of &ldquo;Ummm why are you in Kansas?&rdquo; So we&rsquo;re going to expose our Django community friends to as much as this area has to offer to help explain why we choose to live here. And who knows maybe one of them will fall in love, as we have, and build their next startup here.
 
 <h2>
 So what can you expect?
 </h2>
 
 Most everyone agrees the best part of Lawrence is our downtown, so we&rsquo;re having the event downtown and encouraging everyone to stay in or near the area. This makes everything special in Lawrence within walking distance. There&rsquo;s great local restaurants, quirky stores, and being a college town lots of interesting places to grab a libation should you so choose.
 
 
 We&rsquo;re having the event at
 <a href="https://www.google.com/search?q=liberty+hall+lawrence&amp;espv=2&amp;biw=1439&amp;bih=1010&amp;source=lnms&amp;tbm=isch&amp;sa=X&amp;ei=MfOBVZLRFpSZoQTFhoDIDw&amp;ved=0CAcQ_AUoAg">
 Liberty Hall
 </a>
 which has acted as a community-meeting house and venue since 1856! If you&rsquo;re interested you can
 <a href="http://www.libertyhall.net/about/history">
 read more about its history
 </a>
 . Lawrence is weird and funky, which is reflected in the wide variety of entertainers that have performed there. Some highlights include: Oscar Wilde, Tina Turner, Wu Tang Clan, Henry Rollins, Tori Amos, The Civil Wars, and for its 100th anniversary party
 <a href="http://www.pitch.com/FastPitch/archives/2012/06/22/the-flaming-lips-circus-last-night-at-liberty-hall">
 The Flaming Lips
 </a>
 which I&rsquo;m quite sad to have missed. Lawrence has a long and great music history, but much of that happened in the 90s at
 <a href="https://en.wikipedia.org/wiki/Outhouse_(venue)">
 The Outhouse
 </a>
 (which is now sadly a strip club so we
 
 won&rsquo;t
 
 be going there) and another venue down the street named the
 <a href="http://thebottlenecklive.com/about/">
 Bottleneck
 </a>
 .
 
 
 Kansas is known for its BBQ, so of course we&rsquo;re having Kansas City&rsquo;s best
 <a href="http://www.joeskc.com/about/">
 Oklahoma Joe&rsquo;s
 </a>
 cater lunch. In 2009, Anthony Bourdain listed Joe&rsquo;s in his &ldquo;Thirteen Places to Eat Before You Die&rdquo;. Yeah it&rsquo;s that good.
 
 <h2>
 But it&rsquo;s a party right?
 </h2>
 
 Oh yes it&rsquo;s a party all right! We&rsquo;ve gotten special permission from the City of Lawrence to close down a block of downtown and hold a block party! We&rsquo;ll have a few live bands, including Adrian, food, beer, a special cocktail tent for our more refined guests, snow cones, and some other fun family friendly things for the kids.
 
 
 See many in Lawrence don&rsquo;t know about Django and we want to change that. By having a block party all of Lawrence can come down and help celebrate, instead of having it just be a small gathering of us geeks.
 
 
 Oh and there will be cake of course&hellip; it&rsquo;s not a birthday party without cake.
 
 <h2>
 Many thanks to our great sponsors for helping us put on a great and fun event!
 </h2>
 <ul>
 <li>
 <a href="https://p.ota.to/">
 Potato
 </a>
 </li>
 <li>
 <a href="http://plushrugs.com/">
 PlushRugs.com
 </a>
 </li>
 <li>
 <a href="https://lincolnloop.com/">
 Lincoln Loop
 </a>
 </li>
 <li>
 <a href="https://reelio.com/">
 Reelio
 </a>
 </li>
 <li>
 <a href="https://teamtreehouse.com/">
 Treehouse
 </a>
 </li>
 </ul>
 
 And our travel sponsors:
 
 <ul>
 <li>
 <a href="https://www.python.org/psf/">
 The Python Software Foundation
 </a>
 </li>
 <li>
 <a href="https://www.djangoproject.com/">
 The Django Software Foundation
 </a>
 </li>
 </ul>
 </div>
</div>

So as you may know, Django is turning 10 years old this summer and we’re throwing it a
   
    birthday party
   
   . It’s a one day tech conference and one day of sprints mere feet from where Adrian, Simon, Jacob, and Wilson toiled away on “
   
    the CMS
   
   ”, as it was called then. That code became the first of a few amazing refreshes of
   
    Lawrence.com
   
   and the
   
    very first
   
   Django website.
  
  
   We’ve got an amazing line up of
   
    speakers
   
   of course. And we want to celebrate Django obviously, but secretly we have another small motivation. See we love Lawrence, despite the fact it’s surrounded by Kansas, and we’ve grown tired of answering questions along the lines of “Ummm why are you in Kansas?” So we’re going to expose our Django community friends to as much as this area has to offer to help explain why we choose to live here. And who knows maybe one of them will fall in love, as we have, and build their next startup here.
  
  
   So what can you expect?
  
  
   Most everyone agrees the best part of Lawrence is our downtown, so we’re having the event downtown and encouraging everyone to stay in or near the area. This makes everything special in Lawrence within walking distance. There’s great local restaurants, quirky stores, and being a college town lots of interesting places to grab a libation should you so choose.
  
  
   We’re having the event at
   
    Liberty Hall
   
   which has acted as a community-meeting house and venue since 1856! If you’re interested you can
   
    read more about its history
   
   . Lawrence is weird and funky, which is reflected in the wide variety of entertainers that have performed there. Some highlights include: Oscar Wilde, Tina Turner, Wu Tang Clan, Henry Rollins, Tori Amos, The Civil Wars, and for its 100th anniversary party
   
    The Flaming Lips
   
   which I’m quite sad to have missed. Lawrence has a long and great music history, but much of that happened in the 90s at
   
    The Outhouse
   
   (which is now sadly a strip club so we
   
    won’t
   
   be going there) and another venue down the street named the
   
    Bottleneck
   
   .
  
  
   Kansas is known for its BBQ, so of course we’re having Kansas City’s best
   
    Oklahoma Joe’s
   
   cater lunch. In 2009, Anthony Bourdain listed Joe’s in his “Thirteen Places to Eat Before You Die”. Yeah it’s that good.
  
  
   But it’s a party right?
  
  
   Oh yes it’s a party all right! We’ve gotten special permission from the City of Lawrence to close down a block of downtown and hold a block party! We’ll have a few live bands, including Adrian, food, beer, a special cocktail tent for our more refined guests, snow cones, and some other fun family friendly things for the kids.
  
  
   See many in Lawrence don’t know about Django and we want to change that. By having a block party all of Lawrence can come down and help celebrate, instead of having it just be a small gathering of us geeks.
  
  
   Oh and there will be cake of course… it’s not a birthday party without cake.
  
  
   Many thanks to our great sponsors for helping us put on a great and fun event!
  
  
   
    
     Potato
    
   
   
    
     PlushRugs.com
    
   
   
    
     Lincoln Loop
    
   
   
    
     Reelio
    
   
   
    
     Treehouse
    
   
  
  
   And our travel sponsors:
  
  
   
    
     The Python Software Foundation
    
   
   
    
     The Django Software Foundation

Django is turning 10 years old this summer and we're throwing it a birthday party in the city where it was born.

Django Birthday Party

Improve your Django Tests using django-test-plus&#x27; helpful additions to the standard Django TestCase

<div class="w-block-markdown block-markdown">
 <div class="prose prose-lg max-w-none">
 
 You write tests. Or at least you should be writing tests! As Jacob always says,
 
 "Code without tests is broken as designed."
 
 
 
 Unfortunately you have to write a ton of boilerplate code to test even the simplest of things. This isn't true of just Django, but most languages and frameworks. There are lots of tools like
 <a href="https://factoryboy.readthedocs.org/en/latest/">
 Factory Boy
 </a>
 and
 <a href="http://pytest.org/">
 pytest
 </a>
 that help, but we can do better.
 
 
 If you're looking for how to get started writing Django tests I'd suggest reading the
 <a href="https://docs.djangoproject.com/en/dev/topics/testing/">
 excellent testing documentation
 </a>
 first and also consider picking up Harry Percival's excellent book
 <a href="http://www.amazon.com/gp/product/1449364829/ref=as_li_tl?ie=UTF8&amp;camp=1789&amp;creative=390957&amp;creativeASIN=1449364829&amp;linkCode=as2&amp;tag=revosystblog-20&amp;linkId=VG3GUGRNAQXLZRVR">
 Test-Driven Development with Python
 </a>
 <img alt="" src="http://ir-na.amazon-adsystem.com/e/ir?t=revosystblog-20&amp;l=as2&amp;o=1&amp;a=1449364829"/>
 
 
 After the 400th time of copying over the same base test methods into various personal and client projects, I finally got around to packaging up and releasing all of our useful test additions into a module
 <a href="https://pypi.python.org/pypi/django-test-plus">
 django-test-plus
 </a>
 . I confess, I did this so I didn't have to copy it around anymore as much as wanting to share it with the world. My laziness is your gain. Come be lazy with me!
 
 <h2>
 Let's have some testing fun
 </h2>
 
 I think this module is best explained by examples. So let's just dive right in. First off you need to install django-test-plus with:
 
 <div class="codehilite">
 <pre><code>pip install django-test-plus
</code></pre>
 </div>
 <h3>
 Then instead of this...
 </h3>
 <div class="codehilite">
 <pre><code>from django.test import TestCase
from django.core.urlresolvers import reverse

class MyViewTest(TestCase):

 def test_some_view(self):
 url = reverse('my-url-name')
 response = self.client.get(url)
 self.assertEqual(response.status_code, 200)
</code></pre>
 </div>
 <h3>
 Do this...
 </h3>
 <div class="codehilite">
 <pre><code>from test_plus.test import TestCase

class MyViewTest(TestCase):

 def test_some_view(self):
 response = self.get('my-url-name')
 self.response_200(response)
</code></pre>
 </div>
 
 For the sake of brevity, I'll be excluding the imports and test class in the rest of these examples and just focus on the methods themselves.
 
 <h3>
 Or even this...
 </h3>
 
 Getting a named URL and checking that status is 200 is such a common pattern, it has it's own helper. This will get the named URL, check that the status is 200 and return the response for other checks.
 
 <div class="codehilite">
 <pre><code>def test_some_view(self):
 response = self.get_check_200('my-url-name')
</code></pre>
 </div>
 <h3>
 Instead of...
 </h3>
 <div class="codehilite">
 <pre><code>def test_with_reverse_args(self):
 url = reverse('some-url', kwargs={
 'year': 2015, 
 'slug': 'my-this-is-getting-long'
 })
 response = self.client.get(url)
</code></pre>
 </div>
 <h3>
 Do this...
 </h3>
 <div class="codehilite">
 <pre><code>def test_with_reverse_args(self):
 response = self.get('some-url', year=2015, slug='much-better')
</code></pre>
 </div>
 <h2>
 HTTP Posts work the same way
 </h2>
 <h3>
 Instead of...
 </h3>
 <div class="codehilite">
 <pre><code>def post_with_reverse_args(self):
 url = reverse('some-url', kwargs={
 'year': 2015, 
 'slug': 'my-this-is-getting-long'
 })
 data = {'body': 'long-way'}
 response = self.client.post(url. data=data)
</code></pre>
 </div>
 
 `
 
 <h3>
 Save your keystrokes for another day with...
 </h3>
 <div class="codehilite">
 <pre><code>def test_post_better(self):
 response = self.post('some-url', 
 year=2015, 
 slug='much-better', 
 data={'body': 'lazy-way'}
 )
</code></pre>
 </div>
 <h3>
 Sometimes you still need reverse...
 </h3>
 
 So it's included for you. No need to import it yourself.
 
 <div class="codehilite">
 <pre><code>def test_reversing(self):
 response = self.get('my-named-url')
 test_url = self.reverse('some-other-url', pk=12)
 self.assertEqual(response.context['next'], test_url)
</code></pre>
 </div>
 
 That's better, but that's still a bit rough on the hands. Let's make it a touch better with this version which uses our
 <code>
 get_context()
 </code>
 method:
 
 <div class="codehilite">
 <pre><code>def test_reversing(self):
 response = self.get('my-named-url')
 test_url = self.reverse('some-other-url', pk=12)
 self.assertEqual(self.get_context('next'), test_url)
</code></pre>
 </div>
 
 We keep the last response received in
 <code>
 self.last_response
 </code>
 for you so there isn't a reason to have to pass it around all over the place.
 
 
 Often you need to test the values of several context variables, so let's make that a bit easier.
 
 <div class="codehilite">
 <pre><code>def test_several_values(self):
 self.get('my-view')
 self.assertContext('key1', 'value1')
 self.assertContext('key2', 'value2')
 self.assertContext('key3', False)
</code></pre>
 </div>
 <h2>
 What about other statuses?
 </h2>
 
 Don't worry we've got you covered there are several
 <code>
 response_XXX()
 </code>
 methods to test for other common status code, for example:
 
 <div class="codehilite">
 <pre><code>def test_not_found(self):
 response = self.get('no-there')
 self.response_404(respones)
</code></pre>
 </div>
 <h2>
 Authentication and Users
 </h2>
 
 When testing out Django views you often need to make some users, login in as them, and poke around. Let's make that easier too!
 
 <div class="codehilite">
 <pre><code>def test_needs_login(self):
 # Make a user 
 user = self.make_user('testuser')

 # Make sure we protected the view
 self.assertLoginRequired('my-protected-view')

 with self.login(user):
 self.get('my-protected-view')
 self.assertContext('secret', True)
</code></pre>
 </div>
 <h3>
 Performance
 </h3>
 
 It's easy to make a few template changes that seem inconsequential only to have your database query count blow up to the size of Warren Buffett's checking account. Django provides the
 <code>
 assertNumQueries
 </code>
 assertion context to check the query count, but that is a static count. Often the results are slightly variable, so django-test-plus as
 <code>
 assertNumQueriesLessThan
 </code>
 .
 
 <div class="codehilite">
 <pre><code>def test_does_not_get_crazy(self):
 with self.assertNumQueriesLessThan(25):
 variable_query_count_function()
</code></pre>
 </div>
 <h3>
 Ultimate Lazy
 </h3>
 
 Maybe it's a toy project or you're adding tests to a project that previously didn't have any. Some tests are better than no tests right? So we provide a quick view check with
 <code>
 assertGoodView
 </code>
 :
 
 <div class="codehilite">
 <pre><code>def test_better_than_nothing(self):
 self.assertGoodView('first-view')
 self.assertGoodView('second-view')
 self.assertGoodView('something-else')
</code></pre>
 </div>
 
 What's this do? It gets the view at the named URL, ensures the status code is 200 and tests that the number of queries run is less than 50. Oh and it returns the response if you want it for other purposes. Not amazing, but better than having no test coverage.
 
 <h2>
 The future
 </h2>
 
 I've been using various versions of this module for years, but there is always room for improvement. Happy to take pull requests submissions for new methods that will be generally useful. Or maybe one day I'll be less lazy and see about merging these into Django core.
 
 
 Hope you like the module and find it useful. Happy Testing!
 
 </div>
</div>

pip install django-test-plus

from django.test import TestCase
from django.core.urlresolvers import reverse

class MyViewTest(TestCase):

    def test_some_view(self):
        url = reverse('my-url-name')
        response = self.client.get(url)
        self.assertEqual(response.status_code, 200)

from test_plus.test import TestCase

class MyViewTest(TestCase):

    def test_some_view(self):
        response = self.get('my-url-name')
        self.response_200(response)

def test_some_view(self):
    response = self.get_check_200('my-url-name')

def test_with_reverse_args(self):
    url = reverse('some-url', kwargs={
        'year': 2015, 
        'slug': 'my-this-is-getting-long'
    })
    response = self.client.get(url)

def test_with_reverse_args(self):
    response = self.get('some-url', year=2015, slug='much-better')

def post_with_reverse_args(self):
    url = reverse('some-url', kwargs={
        'year': 2015, 
        'slug': 'my-this-is-getting-long'
    })
    data = {'body': 'long-way'}
    response = self.client.post(url. data=data)

def test_post_better(self):
    response = self.post('some-url', 
        year=2015, 
        slug='much-better', 
        data={'body': 'lazy-way'}
    )

def test_reversing(self):
    response = self.get('my-named-url')
    test_url = self.reverse('some-other-url', pk=12)
    self.assertEqual(response.context['next'], test_url)

def test_reversing(self):
    response = self.get('my-named-url')
    test_url = self.reverse('some-other-url', pk=12)
    self.assertEqual(self.get_context('next'), test_url)

def test_several_values(self):
    self.get('my-view')
    self.assertContext('key1', 'value1')
    self.assertContext('key2', 'value2')
    self.assertContext('key3', False)

def test_not_found(self):
    response = self.get('no-there')
    self.response_404(respones)

def test_needs_login(self):
    # Make a user 
    user = self.make_user('testuser')

    # Make sure we protected the view
    self.assertLoginRequired('my-protected-view')

    with self.login(user):
          self.get('my-protected-view')
          self.assertContext('secret', True)

def test_does_not_get_crazy(self):
    with self.assertNumQueriesLessThan(25):
        variable_query_count_function()

def test_better_than_nothing(self):
    self.assertGoodView('first-view')
    self.assertGoodView('second-view')
    self.assertGoodView('something-else')

  
  
   What's this do? It gets the view at the named URL, ensures the status code is 200 and tests that the number of queries run is less than 50. Oh and it returns the response if you want it for other purposes. Not amazing, but better than having no test coverage.
  
  
   The future
  
  
   I've been using various versions of this module for years, but there is always room for improvement. Happy to take pull requests submissions for new methods that will be generally useful. Or maybe one day I'll be less lazy and see about merging these into Django core.
  
  
   Hope you like the module and find it useful. Happy Testing!

Improve your Django Tests using django-test-plus' helpful additions to the standard Django TestCase

Improved Django Tests

Four simple steps you can take to improve the performance of your Django applications

<div class="w-block-markdown block-markdown">
 <div class="prose prose-lg max-w-none">
 
 Optimizing for performance often comes with a bunch of headaches and hard problems, but it doesn&rsquo;t always have to be that way.
 
 
 Below are four simple things you can do to quickly and easily improve the performance of your sites. They&rsquo;re so simple you should make them part of your standard setup.
 
 <h2>
 Persistent database connections
 </h2>
 
 New in Django 1.6 Django has built-in database connection persistence. Most everyone uses PostgreSQL for their production systems, and while connecting to PG isn&rsquo;t particularly slow, it is something we can optimize.
 
 
 Without persistent connections every request to your site also makes a connection to your database, authenticates the user, etc. This can easily take as much as 20&ndash;75ms if your database is on another host even on a fast network.
 
 
 To setup connection persistence you add the
 <code>
 CONN_MAX_AGE
 </code>
 parameter to your
 <code>
 DATABASES
 </code>
 setting to look like this:
 
 <div class="codehilite">
 <pre><code>DATABASES = {
 'default': {
 'ENGINE': 'django.db.backends.postgresql_psycopg2',
 'NAME': 'whoohoodb',
 'CONN_MAX_AGE': 600,
 }
}
</code></pre>
 </div>
 
 With this,we&rsquo;ve setup persistent connections to stay alive for 10 minutes at a time. This helps to cut down on any possible memory leaks or a flaky connection causing a long term problem. You can certainly use a longer value, but I wouldn&rsquo;t go much beyond an hour as there will be little benefit.
 
 
 You can read more details about
 <a href="https://docs.djangoproject.com/en/1.8/ref/databases/#persistent-connections">
 persistent database connections
 </a>
 in the Django documentation.
 
 <h2>
 Template Loading
 </h2>
 
 By default Django comes configured to use the two standard template loaders:
 
 <div class="codehilite">
 <pre><code>TEMPLATE_LOADERS = (
 'django.template.loaders.filesystem.Loader',
 'django.template.loaders.app_directories.Loader',
)
</code></pre>
 </div>
 
 These loaders search the file system and parse your templates on every request. Kinda surprising that it works as quickly as it does right?
 
 
 You can turn on cached loading, so Django only has to find and parse your templates one time, by simply changing your configuration to be:
 
 <div class="codehilite">
 <pre><code>TEMPLATE_LOADERS = (
 ('django.template.loaders.cached.Loader', (
 'django.template.loaders.filesystem.Loader',
 'django.template.loaders.app_directories.Loader',
 )),
)
</code></pre>
 </div>
 
 Now, don&rsquo;t be silly and turn on cached loading in your development environments as it will get annoying. You&rsquo;ll be forced to restart runserver (or whatever you&rsquo;re using locally) on each and every template change.
 
 <h2>
 Optimize Django Sessions
 </h2>
 
 If you poll Django users nearly 82% of them have no idea where their sessions are stored. Ok, I admit I made that number up, but in our experience it&rsquo;s pretty true.
 
 
 By default, Django stores your user sessions in your database and expects you to occasionally prune out old entries. Very few people do this.
 
 
 So on each request you&rsquo;re doing a SQL query to get the session data and another to grab the User object information. If you&rsquo;re the sort of person who&rsquo;s concerned about performance you likely already have memcached or Redis setup to cache some things. You can switch to storing your sessions in your cache and easily remove a SQL query from every single request to your site with:
 
 <div class="codehilite">
 <pre><code>SESSION_ENGINE = 'django.contrib.sessions.backends.cache'
</code></pre>
 </div>
 
 Now this assumes you can safely lose any data you store in your user sessions. If that&rsquo;s not the case, you can still get some benefit from using:
 
 <div class="codehilite">
 <pre><code>SESSION_ENGINE = 'django.contrib.sessions.backends.cached_db'
</code></pre>
 </div>
 
 This backend caches your data, but also writes any changes to your database for long term storage.
 
 <h3>
 select_related() and prefetch_related()
 </h3>
 
 The first two tips above were super easy, right? Unfortunately this last one, while still easy compared to totally changing up your data storage, does require a bit more than a single configuration setting change.
 
 
 <code>
 select_related()
 </code>
 and
 <code>
 prefetch_related()
 </code>
 aren&rsquo;t in everyone&rsquo;s bag of tricks but they are incredibly useful for cutting down on the number of SQL queries your ORM code is firing.
 
 
 The situation that usually happens is you have some
 <code>
 BlogPost
 </code>
 model with a ForeignKey to User. Then in a list view you use
 <code>
 queryset = BlogPost.objects.active
 </code>
 . Then in your templates you have something like:
 
 <div class="codehilite">
 <pre><code>&lt;ul&gt;
{% for post in object_list %}
 &lt;li&gt;{{ post.title }} - {{ post.user.email }}&lt;/li&gt;
{% endfor %}
&lt;/ul&gt;
</code></pre>
 </div>
 
 This ends up doing the expected SELECT against the blog_post table, but then is forced in the template loop to do another SELECT against auth_user
 
 for each and every post in the list
 
 .
 
 
 Instead, we can ask the Django ORM to go ahead and JOIN in the auth_user table to our results so that
 <code>
 object.user
 </code>
 is already an object for us. Taking us from
 <code>
 BlogPost.objects.active().count()
 </code>
 queries down to a single query.
 
 
 We can fix it by simply changing the view&rsquo;s queryset to be:
 
 <div class="codehilite">
 <pre><code>queryset = BlogPost.objects.select_related().active()
</code></pre>
 </div>
 
 While not a simple settings change, still pretty simple to accomplish.
 
 
 <code>
 prefetch_related
 </code>
 is the same mechanism, but for ManyToManyField relationships. Prefetch can&rsquo;t use a join here, so the
 
 join
 
 happens in Python at the ORM level. This can be a HUGE performance win for situations where you have a small number of rows in your ManyToManyField() vs a large number of rows in the model itself. However, in general, it&rsquo;s less useful and thus used less often that
 <code>
 select_related()
 </code>
 .
 
 
 So when do you use these you ask? The easiest thing to do is fire up
 <a href="https://django-debug-toolbar.readthedocs.org/en/1.3/">
 django-debug-toolbar
 </a>
 with your project and some representative data. By representative I mean you shouldn&rsquo;t have a single BlogPost and a single User, try more like 100+ of each.
 
 
 Then all you need to do is browse around your app locally with the debug toolbar open looking specifically at the query counts and times. If you see anything over 5&ndash;10 queries going on this is your clue to investigate.
 
 
 Most of the time with 100s of objects in your database you&rsquo;ll stumble across a view that is doing 200&ndash;500+ queries where one of these two options will be a huge win for you. Change your queryset, refresh, and make sure that both your query counts decreased but also your overall time. Occasionally you&rsquo;ll have a situation where it&rsquo;s actually faster to do the extra queries vs using joins it happens rarely, but still worth checking the overall query time to make sure you aren&rsquo;t making things worse.
 
 </div>
</div>

DATABASES = {
 'default': {
 'ENGINE': 'django.db.backends.postgresql_psycopg2',
 'NAME': 'whoohoodb',
 'CONN_MAX_AGE': 600,
 }
}

TEMPLATE_LOADERS = (
 'django.template.loaders.filesystem.Loader',
 'django.template.loaders.app_directories.Loader',
)

TEMPLATE_LOADERS = (
 ('django.template.loaders.cached.Loader', (
 'django.template.loaders.filesystem.Loader',
 'django.template.loaders.app_directories.Loader',
 )),
)

SESSION_ENGINE = 'django.contrib.sessions.backends.cache'

SESSION_ENGINE = 'django.contrib.sessions.backends.cached_db'

<ul>
{% for post in object_list %}
 <li>{{ post.title }} - {{ post.user.email }}</li>
{% endfor %}
</ul>

queryset = BlogPost.objects.select_related().active()

 
 
 While not a simple settings change, still pretty simple to accomplish.
 
 
 
 prefetch_related
 
 is the same mechanism, but for ManyToManyField relationships. Prefetch can’t use a join here, so the
 
 join
 
 happens in Python at the ORM level. This can be a HUGE performance win for situations where you have a small number of rows in your ManyToManyField() vs a large number of rows in the model itself. However, in general, it’s less useful and thus used less often that
 
 select_related()
 
 .
 
 
 So when do you use these you ask? The easiest thing to do is fire up
 
 django-debug-toolbar
 
 with your project and some representative data. By representative I mean you shouldn’t have a single BlogPost and a single User, try more like 100+ of each.
 
 
 Then all you need to do is browse around your app locally with the debug toolbar open looking specifically at the query counts and times. If you see anything over 5–10 queries going on this is your clue to investigate.
 
 
 Most of the time with 100s of objects in your database you’ll stumble across a view that is doing 200–500+ queries where one of these two options will be a huge win for you. Change your queryset, refresh, and make sure that both your query counts decreased but also your overall time. Occasionally you’ll have a situation where it’s actually faster to do the extra queries vs using joins it happens rarely, but still worth checking the overall query time to make sure you aren’t making things worse.

Django Performance: 4 Simple Things

Is Wagtail the best Django CMS? Review of Wagtail 1.0 features and our experiences using it.

from django.db import models
from wagtail.wagtailcore.models import Page
from wagtail.wagtailcore.fields import RichTextField
from wagtail.wagtailadmin.edit_handlers import FieldPanel

class BlogPage(Page):
    sub_title = models.CharField(max_length=500, blank=True)
    published = models.DateField()
    author = models.CharField(max_length=100)
    summary = models.RichTextField(blank=True)
    body = models.RichTextField()
    closing_content = models.RichTextField(blank=True)

    content_panels = [
        FieldPanel(‘title’),
        FieldPanel(‘sub_title’),
        FieldPanel(‘published’),
        FieldPanel(‘author’),
        FieldPanel(‘summary’),
        FieldPanel(‘body’),
        FieldPanel(‘closing_content’)
    ]

DEPARTMENT_CHOICES = (
    (‘admin’, ‘Administration’),
    (‘accounting’, ‘Accounting’),
    (‘marketing’, ‘Marketing’),
    (‘sales’, ‘Sales’),
    (‘engineer’, ‘Engineering’),
)

class StaffPage(Page):
    first_name = models.CharField(max_length=50)
    last_name = models.CharField(max_length=50)
    active = models.BooleanField(default=True)
    start_date = models.DateField()
    end_date = models.DateField(blank=True, null=True)
    department = models.CharField(max_length=50, choices=DEPARTMENT_CHOICES)
    email = models.EmailField()
    twitter = models.CharField(max_length=50)
    short_bio = models.RichTextField(blank=True)
    bio = models.RichTextField(blank=True)
    education = models.RichTextField(blank=True)
    work_history = models.RichTextField(blank=True)

    # Panel options left out for brevity

  
  
   As you can see the StaffPage model has quite a bit more fields, most of them optional, which allows the Staff member to update their information over time and not get strangled into putting ‘Bio coming soon’ into an otherwise required field.
  
  
   Pretty simple right? You’re probably thinking there is some catch, luckily you’re wrong. It’s really pretty much that simple. Easy things easy right?
  
  
   Harder things in Wagtail
  
  
   So what are the
   
    hard
   
   things in Wagtail? Well it’s mostly just getting familiar with the system in general. A few things that may trip you up are:
  
  
   
    You can’t have a field named
    
     url
    
    on your derived models as Wagtail uses that field name in the parent Page model. Unfortunately if you do add one, which I’ve done more times than I care to admit, you get a not very useful error “
    
     can’t set attribute
    
    ” and not much else to go on.
   
   
    On many listing type pages it’s fine to simply show all of the items, with pagination, in some sort of chronological order. Other times users want to be able to manually curate what shows up on given pages. Wagtail makes this relatively easy as you can define a ForeignKey relationship using a through type model to other pages and use a
    
     PageChoosePanel
    
    to give the user a nice interface for doing this. The user can also manually order them right in the admin, no additional work necessary.
   
   
    Limiting which pages can be created as children (aka beneath) a page can be handled by setting a list named
    
     parent_page_types
    
    on the child model. Then it can only be added below pages of those defined types. On complex sites with lots of different page types this helps keep the Page Choosing and Creation option choices to a manageable level for the users. And it also obviously helps to keep users from creating the wrong types of pages in the wrong parts of the site.
   
   
    Wagtail currently doesn’t have a great story around building navigation menus for you, but there are a dozen reusable Django apps to help handle that. Often a site’s menu remains relatively static and isn’t edited day-to-day anyway.
   
   
    Supporting multiple sites with the same CMS. This isn’t technically hard, but more conceptually difficult to grok. Wagtail supports having multiple sites, via it’s wagtailsites app. The way this works is you simply set the Root page for each hostname and it basically takes it from there. However, in most circumstances it’s probably easier and cleaner to just have two different instances of Wagtail and use different databases.
   
  
  
   Images and Documents
  
  
   Documents are files of any type you want to be able to upload into the system. This handles any sort of situations where a user needs to upload a PDF, Excel, or Word document and be able to link to it from any other bit of content.
  
  
   Images are exactly what you think, however you can define your own base model for this if you choose and attach additional fields for things like copyright, license, attribution, or even something like EXIF data if you wanted.
  
  
   With both Documents and Images having tagging support via
   
    django-taggit
   
   and a really nicely designed look and UX for them in the admin interface.
  
  
   And yes, before you ask, it has built in support for flexible thumbnails in your templates andthe ability for you to mainually define the main focal point in the image to avoid cropping things weirdly.
  
  
   
  
  
   Form Builder interface
  
  
   Wagtail also has a nice
   
    form builder
   
   built into it that can easily suffice for your typical contact form scenarios or more complicated collection needs.
  
  
   Much like Pages, you simply subclass from Wagtail and define what fields you want to collect. On your model you can also override the
   
    process_form_submission
   
   method to do more complex validation or in a more common case to email the interested parties that there is a new submission.
  
  
   One great feature of the form builder that is also built in, is the viewing and downloading interface. Viewing the data that has come in is great, but you just know your users are going to want to pull it out and use it for some other purpose. Wagtail smartly anticipates this and allows the user to download the submitted data, by date range, as a CSV file anytime they want.
  
  
   Snippets
  
  
   
    Wagtail Snippets
   
   are reusable bits of content that aren’t full web pages. Often these are used for things like sidebar content, advertisements, or calls to action.
  
  
   Unlike with Pages or Forms, you don’t subclass a model but instead define a model and simply register it as a snippet. You’re then free to give your users the option of attaching snippets of the types you want to other pages. Or if you just want to give them the ability to edit things like footer content for example, you can just manually include the snippet data and the Snippet admin UI is really just becomes their editing interface.
  
  
   Best Feature? Content Streams
  
  
   While being able to define your own Page types with their own fields goes a long way, it’s quite a stretch from truly free form content. New in version 1.0 is Wagtail’s killer feature the
   
    StreamField
   
   .
  
  
   Users want free form content while developers, designers, and even ops want nicely structured data. StreamFields satisfies both camps.
  
  
   When you define a StreamField on a Page you set what types of
   
    blocks
   
   are available to be added into that stream. A block can be something as simple as a CharField of text or as complicated as a Staff type record like above using
   
    structural block types
   
   .
  
  
   The end user can then add say few headers of various types, some rich text content blocks and have it all interspersed with a few images and code blocks. Each of these block types you define can then be styled differently CSS and/or have their markup structured entirely differently if needed.
  
  
   Prior to this feature being added to 1.0, I had to resort to complicated Page relationships that weren’t actually pages we intended to make visible on the site. We just subverted the Page Choosing features of Wagtail to give the users the flexibility they needed and keep it all in the same admin interface.
  
  
   Here is what the admin UI looks like for StreamFields. Here we've defined a field named Body that has header, content, and code block types. Each of these
   
    lines
   
   are different blocks. The top and bottom being headers. As you can see you can simply click the plus icons to add new blocks in between others or use the arrows on the right to move block around. They are currently a bit hard to see due to a CSS bug I anticipate being fixed quickly.
  
  
   
  
  
   Wagtail’s Future
  
  
   I think Wagtail has a
   
    VERY
   
   bright future in general and especially inside the Django community. However, like any 1.0 product there are definitely some things I would like to see in future versions. The two main things I hope to see are:
  
  
   
    A community collection of high quality and flexible common Page and Block types to make most sites more of a Lego exercise than a coding one.
   
   
    The ability to more easily customize and control the Publish/Moderate/Save as Draft options that appear at the bottom of the screen while editing content. On many smaller sites or those with a flat workflow it should be trivial to make ‘Publish’ or ‘Submit for Moderation’ be the default action presented to the user.

Wagtail 1.0 (beta) best Django CMS?

How to load files into Django models, not via a file upload, but from a script or Python shell.

<div class="w-block-markdown block-markdown">
 <div class="prose prose-lg max-w-none">
 
 I don&rsquo;t know about you, but I run into a situation every so often where I need to programmatically push a file from the local file system or remotely via a URL into a Django model. Using a Python shell, a one off script, or a Django management command.
 
 
 This should be easy, but with a typical web app you do it infrequently. Mostly I find we do it when we&rsquo;re converting someone&rsquo;s site from something else to Django and need to associate an image with another Content item for example.
 
 
 If you start Googling around for answers to how to do this you get the default docs for
 <a href="https://docs.djangoproject.com/en/dev/topics/http/file-uploads/">
 File Uploads
 </a>
 ,
 <a href="https://docs.djangoproject.com/en/dev/ref/files/file/">
 File objects
 </a>
 , and
 <a href="https://docs.djangoproject.com/en/dev/topics/files/">
 Managing Files
 </a>
 which are all great. However, they don&rsquo;t
 
 exactly
 
 spell out how you&rsquo;re supposed to do this.
 
 
 My first inclination is to just assign the model field to a file, but this is missing the relative file path information Django needs.
 
 
 I took the time to write this post mostly for my own benefit so I don&rsquo;t have to figure this out yet again in the future, but hopefully you&rsquo;ll get some use out of it as well.
 
 <h2>
 Steps for loading Django ImageField with a local file
 </h2>
 
 Here is a quick example showing you the moving pieces. The steps you need to perform are:
 
 <ol>
 <li>
 Retrieve the file (if remote) and store it locally
 </li>
 <li>
 Open that file as a normal Python file object
 </li>
 <li>
 Covert that open file to a Django
 <a href="https://docs.djangoproject.com/en/dev/ref/files/file/">
 File
 </a>
 </li>
 <li>
 Attach it to your model field
 </li>
 </ol>
 <h2>
 Example Model
 </h2>
 
 Suppose we have a model like this:
 
 <div class="codehilite">
 <pre><code>from django.db import models

class Company(models.Model):
 name = models.CharField(max_length=100)
 logo = models.ImageField()
</code></pre>
 </div>
 
 Let&rsquo;s create an entry for RevSys, pulling the logo in with requests:
 
 <div class="codehilite">
 <pre><code>import requests
from django.core.files import File

from .models import Company

r = requests.get(&ldquo;http://media.revsys.com/img/revsys-logo.png&rdquo;)

with open(&ldquo;/tmp/revsys-logo.png&rdquo;, &ldquo;wb&rdquo;) as f:
 f.write(r.content)

reopen = open(&ldquo;/tmp/revsys-logo.png&rdquo;, &ldquo;rb&rdquo;)
django_file = File(reopen)

revsys = Company()
revsys.name = &ldquo;Revolution Systems&rdquo;
revsys.logo.save(&ldquo;revsys-logo.png&rdquo;, django_file, save=True)
</code></pre>
 </div>
 
 The last line here is the important bit.
 <code>
 save
 </code>
 is given three arguments here:
 
 <ol>
 <li>
 The relative path and filename for inside MEDIA_ROOT
 </li>
 <li>
 The open file using Django&rsquo;s File object
 </li>
 <li>
 Whether or not we want to save the revsys
 <code>
 Company
 </code>
 instance after the image is saved.
 </li>
 </ol>
 
 Now if you&rsquo;re doing this for a bunch of images, you&rsquo;ll want to parse your URLs or filenames to build the paths for you. Something like this would be a good starting point, note this is using Python 3:
 
 <div class="codehilite">
 <pre><code>import os
from urllib.parse import urlparse
# from urlparse import urlparse for Python 2.7

url = &ldquo;http://media.revsys.com/img/revsys-logo.png&rdquo;
filename = os.path.basename(urlparse(url).path)
# This returns &lsquo;revsys-logo.png&rsquo; from the URL
&hellip;

revsys.logo.save(filename, django_file, save=True)
</code></pre>
 </div>
 
 Hope this helps!
 
 </div>
</div>

from django.db import models

class Company(models.Model):
    name = models.CharField(max_length=100)
    logo = models.ImageField()

import requests
from django.core.files import File

from .models import Company

r = requests.get(“http://media.revsys.com/img/revsys-logo.png”)

with open(“/tmp/revsys-logo.png”, “wb”) as f:
    f.write(r.content)

reopen = open(“/tmp/revsys-logo.png”, “rb”)
django_file = File(reopen)

revsys = Company()
revsys.name = “Revolution Systems”
revsys.logo.save(“revsys-logo.png”, django_file, save=True)

import os
from urllib.parse import urlparse
# from urlparse import urlparse for Python 2.7

url = “http://media.revsys.com/img/revsys-logo.png”
filename = os.path.basename(urlparse(url).path)
# This returns ‘revsys-logo.png’ from the URL
…

revsys.logo.save(filename, django_file, save=True)

  
  
   Hope this helps!

Loading Django FileField and ImageFields from the file system

Our recommendation for the optimal Django project layout for your files, settings, and templates.

<div class="w-block-markdown block-markdown">
 <div class="prose prose-lg max-w-none">
 <h2>
 What&rsquo;s the optimal layout for your Django applications, settings files, and various other associated directories?
 </h2>
 
 When Django 1.4 was released it included an
 <a href="https://docs.djangoproject.com/en/dev/releases/1.4/#updated-default-project-layout-and-manage-py">
 updated project layout
 </a>
 which went a long way to improving the default Django project&rsquo;s layout, but here are some tips for making it even better.
 
 
 This is a question we get asked all of the time so I wanted to take a bit of time and write down exactly how we feel about this subject so we can easily refer clients to this document. Note that this was written using Django version 1.7.1, but can be applied to any Django version after 1.4 easily.
 
 <h2>
 Why this layout is better
 </h2>
 
 The project layout we&rsquo;re recommending here has several advantages namely:
 
 <ul>
 <li>
 Allows you to pick up, repackage, and reuse individual Django applications for use in other projects. Often it isn&rsquo;t clear as you are building an app whether or not it is even a candidate for reuse. Building it this way from the start makes it much easier if that time comes.
 </li>
 <li>
 Encourages designing applications for reuse
 </li>
 <li>
 Environment specific settings. No more
 <code>
 if DEBUG==True
 </code>
 nonsense in a single monolithic settings file. This allows to easily see which settings are shared and what is overridden on a per environment basis.
 </li>
 <li>
 Environment specific PIP requirements
 </li>
 <li>
 Project level templates and static files that can, if necessary, override app level defaults.
 </li>
 <li>
 Small more specific test files which are easier to read and understand.
 </li>
 </ul>
 
 Assuming you have two apps
 
 blog
 
 and
 
 users
 
 and 2 environments dev and prod your project layout should be structured like this:
 
 <div class="codehilite">
 <pre><code>myproject/
 manage.py
 myproject/
 __init__.py
 urls.py
 wsgi.py
 settings/
 __init__.py
 base.py
 dev.py
 prod.py
 blog/
 __init__.py
 models.py
 managers.py
 views.py
 urls.py
 templates/
 blog/
 base.html
 list.html
 detail.html
 static/
 &hellip;
 tests/
 __init__.py
 test_models.py
 test_managers.py
 test_views.py
 users/
 __init__.py
 models.py
 views.py
 urls.py
 templates/
 users/
 base.html
 list.html
 detail.html
 static/
 &hellip;
 tests/
 __init__.py
 test_models.py
 test_views.py
 static/
 css/
 &hellip;
 js/
 &hellip;
 templates/
 base.html
 index.html
 requirements/
 base.txt
 dev.txt
 test.txt
 prod.txt
</code></pre>
 </div>
 
 The rest of this article explains how to move a project to this layout and why this layout is better.
 
 <h2>
 Current Default Layout
 </h2>
 
 We&rsquo;re going to call our example project
 
 foo
 
 , yes I realize it&rsquo;s a very creative name. We&rsquo;re assuming here that we&rsquo;re going to be launching
 
 foo.com
 
 but while we like to have our project names reflect the ultimate domain(s) the project will live on this isn&rsquo;t by any means required.
 
 
 If you kick off your project using
 <code>
 django-admin.py startproject foo
 </code>
 you get a directory structure like this:
 
 <div class="codehilite">
 <pre><code> foo/
 manage.py
 foo/
 __init__.py
 settings.py
 urls.py
 wsgi.py
</code></pre>
 </div>
 
 This layout is a great starting place, we have a top level directory
 
 foo
 
 which contains our manage.py the
 
 project
 
 directory
 
 foo/foo/
 
 inside it. This is the directory you would check into your source control system such as git.
 
 
 You should think of this
 
 foo/foo/
 
 subdirectory as being
 
 the project
 
 where everything else is either a Django application or ancillary files related to the project.
 
 <h2>
 Fixing Settings
 </h2>
 
 We&rsquo;re on a mission to fix your bad settings files here. We show this layout to new clients and I&rsquo;m constantly surprised how few people know this is even possible to do. I blame the fact that while everyone knows that settings are just Python code, they don&rsquo;t think about them
 
 as
 
 Python code.
 
 
 So let&rsquo;s fix up our settings. For our foo project we&rsquo;re going to have 4 environments: dev, stage, jenkins, and production. So let&rsquo;s give each it&rsquo;s own file. The process to do this is:
 
 <ol>
 <li>
 In
 
 foo/foo/
 
 make a settings directory and create an empty
 <code>
 __init__.py
 </code>
 file inside it.
 </li>
 <li>
 Move
 
 foo/foo/settings.py
 
 into
 
 foo/foo/settings/base.py
 
 </li>
 <li>
 Create the individual
 
 dev.py
 
 ,
 
 stage.py
 
 ,
 
 jenkins.py
 
 , and
 
 prod.py
 
 files in
 
 foo/foo/settings/
 
 . Each of these 4 environment specific files should simply contain the following:
 <div class="codehilite">
 <pre><code>from base import *
</code></pre>
 </div>
 </li>
 </ol>
 
 So why is this important? Well for local development you want
 <code>
 DEBUG=True
 </code>
 , but it&rsquo;s pretty easy to accidentally push out production code with it on, so just open up
 
 foo/foo/settings/prod.py
 
 and after the initial import from base just add
 <code>
 DEBUG=False
 </code>
 . Now if your production site is safe from that silly mistake.
 
 
 What else can you customize? Well it should be pretty obvious you&rsquo;ll likely have staging, jenkins, and production all pointing at different databases, likely even on different hosts. So adjust those settings in each environment file.
 
 <h2>
 Using these settings
 </h2>
 
 Using these settings is easy, no matter which method you typically use. To use the OS&rsquo;s environment you just do:
 
 <div class="codehilite">
 <pre><code>export DJANGO_SETTINGS_MODULE=&ldquo;foo.settings.jenkins&rdquo;
</code></pre>
 </div>
 
 And boom, you&rsquo;re now using the jenkins configuration.
 
 
 Or maybe you prefer to pass them in as a commandline option like this:
 
 <div class="codehilite">
 <pre><code>./manage.py migrate &mdash;settings=foo.settings.production
</code></pre>
 </div>
 
 Same if you&rsquo;re using gunicorn:
 
 <div class="codehilite">
 <pre><code>gunicorn -w 4 -b 127.0.0.1:8001 &mdash;settings=foo.settings.dev
</code></pre>
 </div>
 <h2>
 What else should be customized about settings?
 </h2>
 
 Another useful tip with Django settings is to change several of the default settings
 
 collections
 
 from being tuples to being lists. For example
 <code>
 INSTALLED_APPS
 </code>
 , by changing it from:
 
 <div class="codehilite">
 <pre><code>INSTALLED_APPS = (
 &hellip;
)
</code></pre>
 </div>
 
 to:
 
 <div class="codehilite">
 <pre><code>INSTALLED_APPS = [
 &hellip;
]
</code></pre>
 </div>
 
 In
 
 foo/settings/base.py
 
 we can now more easily add and remove apps based on each environment specific settings file. For example, maybe you only want django-debug-toolbar installed in dev, but not your other environments.
 
 
 This trick is also often useful for the
 <code>
 TEMPLATE_DIRS
 </code>
 and
 <code>
 MIDDLEWARE_CLASSES
 </code>
 settings.
 
 
 Another useful trick we often use is to break up your apps into two lists, one your prerequisites and another for your actual project applications. So like this:
 
 <div class="codehilite">
 <pre><code>PREREQ_APPS = [
 &lsquo;django.contrib.auth&rsquo;,
 &lsquo;django.contrib.contenttypes&rsquo;,
 &hellip;
 &lsquo;debug_toolbar&rsquo;,
 &lsquo;imagekit&rsquo;,
 &lsquo;haystack&rsquo;,
]

PROJECT_APPS = [
 &lsquo;homepage&rsquo;,
 &lsquo;users&rsquo;,
 &lsquo;blog&rsquo;,
]

INSTALLED_APPS = PREREQ_APPS + PROJECT_APPS
</code></pre>
 </div>
 
 Why is this useful? For one it helps better distinguish between Django core apps, third party apps, and your own internal project specific applications. However,
 <code>
 PROJECT_APPS
 </code>
 often comes in handy as a list of your specific apps for things like testing and code coverage. You have a list of
 
 your
 
 apps, so you can easily and automagically make sure their tests are run and coverage is recorded just for them, not including any third party apps, without having to maintain the list in two separate places.
 
 <h2>
 Fixing requirements
 </h2>
 
 Most projects have a single
 <code>
 requirements.txt
 </code>
 file that is installed like this:
 
 <div class="codehilite">
 <pre><code>pip install -r requirements.txt
</code></pre>
 </div>
 
 This is sufficient for small simple projects, but a little known feature of requirements files is that you can use the
 <code>
 -r
 </code>
 flag to include other files. So we can have a base.txt of all the common requirements and then if we need to be able to run tests have a specific
 
 requirements/test.txt
 
 that looks like this:
 
 <div class="codehilite">
 <pre><code>-r base.txt
pytest==2.5.2
coverage==3.7.1
</code></pre>
 </div>
 
 I&rsquo;ll admit this is not a
 
 HUGE
 
 benefit, but it does help separate out what is a requirement in which environment. And for the truly performance conscience it reduces your pip install time in production a touch by not installing a bunch of things that won&rsquo;t actually be used in production.
 
 <h2>
 Test Files
 </h2>
 
 Why did we separate out the tests files so much? One main reason, if you&rsquo;re writing enough tests a single tests.py file per application will end up being one huge honking file. This is bad for readability, but also just for the simple fact you have to spend time scrolling around a lot in your editor.
 
 
 You&rsquo;ll also end up with less merge conflicts when working with other developers which is a nice side benefit. Small files are your friends.
 
 <h2>
 URLs
 </h2>
 
 For small projects it&rsquo;s tempting to put all of your url definitions in
 
 foo/urls.py
 
 to keep them all in one place. However, if your goal is clarity and reusability you want to define your urls in each app and include them into your main project. So instead of:
 
 <div class="codehilite">
 <pre><code>urlpatterns = patterns(&lsquo;&rsquo;,
 url(r&rsquo;^$&rsquo;, HomePageView.as_view(), name=&lsquo;home&rsquo;),
 url(r&rsquo;^blog/$&rsquo;, BlogList.as_view(), name=&lsquo;blog_list&rsquo;),
 url(r&rsquo;^blog/(?P&lt;pk&gt;\d+)/$&rsquo;, BlogDetail.as_view(), name=&lsquo;blog_detail&rsquo;),
 &hellip;
 url(r&rsquo;^user/list/$&rsquo;, UserList.as_view(), name=&lsquo;user_list&rsquo;),
 url(r&rsquo;^user/(?P&lt;username&gt;\w+)/$&rsquo;, UserDetail.as_view(), name=&lsquo;user_detail&rsquo;),
)
</code></pre>
 </div>
 
 you should do this:
 
 <div class="codehilite">
 <pre><code>urlpatterns = patterns(&lsquo;&rsquo;,
 url(r&rsquo;^$&rsquo;, HomePageView.as_view(), name=&lsquo;home&rsquo;),
 url(r&rsquo;^blog/&lsquo;, include(&lsquo;blog.urls&rsquo;)),
 url(r&rsquo;^user/&lsquo;, include(&lsquo;user.urls&rsquo;)),
)
</code></pre>
 </div>
 <h2>
 Templates and static media
 </h2>
 
 Having per app
 
 templates/
 
 and
 
 static/
 
 directories gives us the ability to reuse an application basically
 
 as is
 
 in another project.
 
 
 We get the default templates the app provides and any associated static media like special Javascript for that one cool feature all in one package.
 
 
 However, it
 
 also
 
 gives us the ability to override those templates on a per project basis in the main
 
 foo/templates/
 
 directory. By adding a
 <code>
 templates/blog/detail.html
 </code>
 template we override, or mask, the default
 <code>
 blog/templates/blog/detail.html
 </code>
 template.
 
 <h2>
 Reusing a Django application
 </h2>
 
 So assuming you&rsquo;ve been using this layout for awhile, one day you&rsquo;ll realize that your new project needs a blog and the one from your
 
 foo
 
 project would be perfect for it. So you copy and paste the files in&hellip;
 
 pssst WRONG!
 
 . Now you have two copies of the application out there. Bug fixes or new features in one have to manually be moved between the projects, and that assumes you even remember to do that.
 
 
 Instead, make a new repo for your blog and put the
 
 foo/blog/
 
 directory in it. And adjust both your existing foo project and your new project to pip install it.
 
 
 They can still both track different versions of the app, if necessary, or keep up to date and get all of your bug fixes and new features as they develop. You still can override the templates and static media as you need to on a per project basis, so there really isn&rsquo;t any real issues doing this.
 
 <h2>
 Additional Resources
 </h2>
 
 Our friends Danny and Audrey over at
 <a href="http://www.cartwheelweb.com">
 CartWheel Web
 </a>
 reminded us about
 <a href="https://github.com/audreyr/cookiecutter">
 Cookie Cutter
 </a>
 and specifically Danny's
 <a href="https://github.com/pydanny/cookiecutter-django">
 cookiecutter-django
 </a>
 as useful tools for making your initial project creations easier and repeatable.
 
 
 Also, if you're looking for all around great Django tips and best practices, you can't go wrong with their book
 <a href="http://www.amazon.com/gp/product/098146730X/ref=as_li_qf_sp_asin_tl?ie=UTF8&amp;camp=1789&amp;creative=9325&amp;creativeASIN=098146730X&amp;linkCode=as2&amp;tag=revosystblog-20">
 Two Scoops of Django: Best Practices For Django 1.6
 </a>
 <img alt="" src="http://ir-na.amazon-adsystem.com/e/ir?t=revosystblog-20&amp;l=as2&amp;o=1&amp;a=098146730X"/>
 which we recommend to all of our clients.
 
 <h2>
 Feedback
 </h2>
 
 We hope you find this improved project layout useful. If you find any bugs, have a suggestion, or just want to chat feel free to reach out to us. Thanks for reading!
 
 </div>
</div>

myproject/
 manage.py
 myproject/
 __init__.py
 urls.py
 wsgi.py
 settings/
 __init__.py
 base.py
 dev.py
 prod.py
 blog/
 __init__.py
 models.py
 managers.py
 views.py
 urls.py
 templates/
 blog/
 base.html
 list.html
 detail.html
 static/
 …
 tests/
 __init__.py
 test_models.py
 test_managers.py
 test_views.py
 users/
 __init__.py
 models.py
 views.py
 urls.py
 templates/
 users/
 base.html
 list.html
 detail.html
 static/
 …
 tests/
 __init__.py
 test_models.py
 test_views.py
 static/
 css/
 …
 js/
 …
 templates/
 base.html
 index.html
 requirements/
 base.txt
 dev.txt
 test.txt
 prod.txt

 foo/
 manage.py
 foo/
 __init__.py
 settings.py
 urls.py
 wsgi.py

from base import *

export DJANGO_SETTINGS_MODULE=“foo.settings.jenkins”

./manage.py migrate —settings=foo.settings.production

gunicorn -w 4 -b 127.0.0.1:8001 —settings=foo.settings.dev

INSTALLED_APPS = (
 …
)

INSTALLED_APPS = [
 …
]

PREREQ_APPS = [
 ‘django.contrib.auth’,
 ‘django.contrib.contenttypes’,
 …
 ‘debug_toolbar’,
 ‘imagekit’,
 ‘haystack’,
]

PROJECT_APPS = [
 ‘homepage’,
 ‘users’,
 ‘blog’,
]

INSTALLED_APPS = PREREQ_APPS + PROJECT_APPS

pip install -r requirements.txt

-r base.txt
pytest==2.5.2
coverage==3.7.1

urlpatterns = patterns(‘’,
 url(r’^$’, HomePageView.as_view(), name=‘home’),
 url(r’^blog/$’, BlogList.as_view(), name=‘blog_list’),
 url(r’^blog/(?P<pk>\d+)/$’, BlogDetail.as_view(), name=‘blog_detail’),
 …
 url(r’^user/list/$’, UserList.as_view(), name=‘user_list’),
 url(r’^user/(?P<username>\w+)/$’, UserDetail.as_view(), name=‘user_detail’),
)

urlpatterns = patterns(‘’,
 url(r’^$’, HomePageView.as_view(), name=‘home’),
 url(r’^blog/‘, include(‘blog.urls’)),
 url(r’^user/‘, include(‘user.urls’)),
)

 
 
 Templates and static media
 
 
 Having per app
 
 templates/
 
 and
 
 static/
 
 directories gives us the ability to reuse an application basically
 
 as is
 
 in another project.
 
 
 We get the default templates the app provides and any associated static media like special Javascript for that one cool feature all in one package.
 
 
 However, it
 
 also
 
 gives us the ability to override those templates on a per project basis in the main
 
 foo/templates/
 
 directory. By adding a
 
 templates/blog/detail.html
 
 template we override, or mask, the default
 
 blog/templates/blog/detail.html
 
 template.
 
 
 Reusing a Django application
 
 
 So assuming you’ve been using this layout for awhile, one day you’ll realize that your new project needs a blog and the one from your
 
 foo
 
 project would be perfect for it. So you copy and paste the files in…
 
 pssst WRONG!
 
 . Now you have two copies of the application out there. Bug fixes or new features in one have to manually be moved between the projects, and that assumes you even remember to do that.
 
 
 Instead, make a new repo for your blog and put the
 
 foo/blog/
 
 directory in it. And adjust both your existing foo project and your new project to pip install it.
 
 
 They can still both track different versions of the app, if necessary, or keep up to date and get all of your bug fixes and new features as they develop. You still can override the templates and static media as you need to on a per project basis, so there really isn’t any real issues doing this.
 
 
 Additional Resources
 
 
 Our friends Danny and Audrey over at
 
 CartWheel Web
 
 reminded us about
 
 Cookie Cutter
 
 and specifically Danny's
 
 cookiecutter-django
 
 as useful tools for making your initial project creations easier and repeatable.
 
 
 Also, if you're looking for all around great Django tips and best practices, you can't go wrong with their book
 
 Two Scoops of Django: Best Practices For Django 1.6
 
 
 which we recommend to all of our clients.
 
 
 Feedback
 
 
 We hope you find this improved project layout useful. If you find any bugs, have a suggestion, or just want to chat feel free to reach out to us. Thanks for reading!

Recommended Django Project Layout

Simple way to use virtaualenvwrapper&#x27;s postactivate and deactivate hooks to execute arbitrary Python code for any of your Python shells while working on that project.

<div class="w-block-markdown block-markdown">
 <div class="prose prose-lg max-w-none">
 <h2>
 Do you find yourself re-doing the same Python shell stuff?
 </h2>
 
 I know I do. I'm constantly doing it and replaying some of my IPython history isn't cutting it for me anymore. IPython has
 <a href="http://ipython.org/ipython-doc/dev/config/intro.html#profiles">
 profiles
 </a>
 but they're not terribly easy to get working with Django and your typical
 <code>
 manage.py shell
 </code>
 command.
 
 <h2>
 Site wide is easy, but what about per project?
 </h2>
 
 You can load things and execute code for
 
 ALL
 
 your Python shells system wide pretty easily, but what about when you need different things loaded depending on the project?
 
 
 Turns out it's pretty easy to do with an environment variable and
 <a href="http://virtualenvwrapper.readthedocs.org/en/latest/">
 virtualenvwrapper
 </a>
 . If you aren't already using virtualenvwrapper you should really check it out. I use it even outside of Python projects just to have an easy mechanism for different "shell environments" to have certain things executed when I start and finish working on a project.
 
 <h2>
 The situation I was in...
 </h2>
 
 While the issue that lead me to figure this out was a Django project, nothing about this technique is Django specific. I have a pretty common workflow when doing green field Django development where I setup Fabric tasks to rebuild my local dev database and repopulate it with certain database objects I need to make the site functional enough for testing. For example, I typically setup a User object with username 'frank' (I know big surprise there) that is a superuser. So if I launch a shell 9 out of 10 times I probably need my
 
 frank
 
 User object for something.
 
 
 This is complicated however by the fact that on some projects I work on the User model is customized to not have a username so in some projects I need to run:
 
 <div class="codehilite">
 <pre><code>frank = User.objects.get(username='frank')
</code></pre>
 </div>
 
 And in others I need:
 
 <div class="codehilite">
 <pre><code>frank = User.objects.get(email='frank@revsys.com')
</code></pre>
 </div>
 
 What I realized was Python honors and executes the code in any file you specify in the
 <code>
 PYTHONSTARTUP
 </code>
 environment variable in your shell. So if you adjust your virtualenv's
 <a href="http://virtualenvwrapper.readthedocs.org/en/latest/plugins.html#plugins-post-activate">
 postactivate and postdeactivate
 </a>
 hooks to set the path to code you want run on a per project basis. For example:
 
 <h3>
 In ~/.virtualenvs/projectX/bin/postactivate
 </h3>
 <div class="codehilite">
 <pre><code> #!/bin/bash
 export PYTHONSTARTUP=/Users/frank/work/src/projectX/startup.py
 export DJANGO_SETTINGS_MODULE="projectX.settings.dev"
 cd /Users/frank/work/src/projectX
</code></pre>
 </div>
 <h3>
 In ~/.virtualenvs/projectX/bin/postdeactivate
 </h3>
 <div class="codehilite">
 <pre><code> #!/bin/bash
 unset $DJANGO_SETTINGS_MODULE
 unset $PYTHONSTARTUP
</code></pre>
 </div>
 <h2>
 So what can you do with this?
 </h2>
 
 Well you can execute any arbitrary Python code so you could could do a lot, even go so far as to hit your time tracking software's API to log the time, but realize anything you run will slow down the startup time of each Python shell you launch so keep it to what you really need. On this particular project I kept it to a few small things:
 
 <div class="codehilite">
 <pre><code> # Import datetime and some of my commonly used Models 
 import datetime
 from django.contrib.auth import get_user_model
 from app1.models import ThatOneModel
 from app2.models import TheOtherCommonModel

 try:
 User = get_user_model()
 frank = User.objects.get(username='frank')
 except User.DoesNotExist:
 # Frank not being in the DB shouldn't raise an error
 pass
</code></pre>
 </div>
 
 Hope this helps you stay in flow more and delay your carpal tunnel a few days at least!
 
 </div>
</div>

frank = User.objects.get(username='frank')

frank = User.objects.get(email='frank@revsys.com')

  #!/bin/bash
  export PYTHONSTARTUP=/Users/frank/work/src/projectX/startup.py
  export DJANGO_SETTINGS_MODULE="projectX.settings.dev"
  cd /Users/frank/work/src/projectX

  #!/bin/bash
  unset $DJANGO_SETTINGS_MODULE
  unset $PYTHONSTARTUP

  # Import datetime and some of my commonly used Models 
  import datetime
  from django.contrib.auth import get_user_model
  from app1.models import ThatOneModel
  from app2.models import TheOtherCommonModel

  try:
      User = get_user_model()
      frank = User.objects.get(username='frank')
  except User.DoesNotExist:
      # Frank not being in the DB shouldn't raise an error
      pass

  
  
   Hope this helps you stay in flow more and delay your carpal tunnel a few days at least!

Simple way to use virtaualenvwrapper's postactivate and deactivate hooks to execute arbitrary Python code for any of your Python shells while working on that project.

Python Dev Tip: DRY your shell with PYTHONSTARTUP

A little bookmarklet that redirects you from the current page to the same path on http://localhost:8000/. Helpful when debugging Django.

<div class="w-block-markdown block-markdown">
 <div class="prose prose-lg max-w-none">
 
 So this is one of those little things where you think, why the hell didn't I think of this years ago?
 
 
 I'm sure many Django developers end up doing this on a fairly regular basis. You're working on a site, go to the production/staging version, navigate around and then want to view the same URL path but on your local development server.
 
 
 If you're like me you cut-n-paste the current URL into a new tab and manually remove the host and replace it with
 <code>
 http://localhost:8000
 </code>
 . That way you have a browser tab of the real site and a tab with the local version for comparision. I've done this at least a billion times. It's a silly little annoyance because it's easy to mess up the URL leaving off the slash after 8000 for example.
 
 
 This morning while working on a site I realized,
 
 WTF
 
 I can do this with a bookmarklet! They're super easy as it turns out.
 
 <h2>
 Version that replaces the current tab:
 </h2>
 <div class="codehilite">
 <pre><code>javascript:(function() { window.location.replace("http://localhost:8000" + window.location.pathname); }())
</code></pre>
 </div>
 <h2>
 Version that creates a new tab:
 </h2>
 <div class="codehilite">
 <pre><code>javascript:(function() { window.open("http://localhost:8000" + window.location.pathname, '_blank'); }())
</code></pre>
 </div>
 
 I still can't quite believe I didn't think to do this long ago! I wonder how many hours of time I would have saved over the years? I've included two links below that you can create your own bookmarks from, you'll obviously need to edit them if you don't use the Django defaults of
 
 localhost:8000
 
 .
 
 
 Simply drag which ever version you prefer into your bookmarks and you're ready to go.
 
 
 <a href="javascript:(function() { window.location.replace(" title='http://localhost:8000" + window.location.pathname); }()) "Make localhost'>
 Make localhost:8000
 </a>
 
 
 <a href="javascript:(function() { window.open(" title="http://localhost:8000&quot; + window.location.pathname, '_blank'); }()) &quot;Make localhost tab">
 Make localhost:8000 tab
 </a>
 
 
 You could easily modify these to create bookmarklets that take you to your staging or QA system and even when working with development frameworks other than Django.
 
 
 Hope this saves you as much time as it is going to save me!
 
 
 
 UPDATE:
 
 <a href="http://jacobian.org">
 Jacob
 </a>
 sent me a great additional add-on tip to this.
 <a href="http://softwareas.com/how-to-use-different-favicons-for-development-staging-and-production/">
 Michael Mahemoff
 </a>
 has a good tip on how to change the Favicon in your browser between different environments like staging and production. So you can combine these two techniques for maximal win!
 
 </div>
</div>

javascript:(function() { window.location.replace("http://localhost:8000" + window.location.pathname); }())

javascript:(function() {  window.open("http://localhost:8000" + window.location.pathname, '_blank'); }())

  
  
   I still can't quite believe I didn't think to do this long ago! I wonder how many hours of time I would have saved over the years? I've included two links below that you can create your own bookmarks from, you'll obviously need to edit them if you don't use the Django defaults of
   
    localhost:8000
   
   .
  
  
   Simply drag which ever version you prefer into your bookmarks and you're ready to go.
  
  
   
    Make localhost:8000
   
  
  
   
    Make localhost:8000 tab
   
  
  
   You could easily modify these to create bookmarklets that take you to your staging or QA system and even when working with development frameworks other than Django.
  
  
   Hope this saves you as much time as it is going to save me!
  
  
   
    UPDATE:
   
   
    Jacob
   
   sent me a great additional add-on tip to this.
   
    Michael Mahemoff
   
   has a good tip on how to change the Favicon in your browser between different environments like staging and production. So you can combine these two techniques for maximal win!

Django Debugging Bookmarklet Trick

<div class="w-block-markdown block-markdown">
 <div class="prose prose-lg max-w-none">
 
 Nearly a year in the making, Django 1.3 is now shipping. It includes a ton of bugfixes along with a bunch of major new features:
 
 <blockquote>
 <ul>
 <li>
 Class-based views.
 </li>
 <li>
 Better support of Python&rsquo;s logging tools.
 </li>
 <li>
 A new tool to help with handling static files.
 </li>
 <li>
 Greatly improved testing utilities via the unittest2 library.
 </li>
 <li>
 Configurable on-delete behavior.
 </li>
 <li>
 And more!
 </li>
 </ul>
 </blockquote>
 
 To help people get a jump on upgrading, I'll be
 <a href="http://www.eventbrite.com/event/1008507473">
 holding a webinar next week
 </a>
 . We'll talk about the new features, go over the steps to follow for a safe and easy upgrade, and cover the "gotchas" to avoid as you upgrade.
 
 
 <a href="http://www.eventbrite.com/event/1008507473">
 You should join us
 </a>
 &mdash; March 31 from 1-3 (central). It'll be a blast.
 
 </div>
</div>

Nearly a year in the making, Django 1.3 is now shipping. It includes a ton of bugfixes along with a bunch of major new features:
  
  
   
    
     Class-based views.
    
    
     Better support of Python’s logging tools.
    
    
     A new tool to help with handling static files.
    
    
     Greatly improved testing utilities via the unittest2 library.
    
    
     Configurable on-delete behavior.
    
    
     And more!
    
   
  
  
   To help people get a jump on upgrading, I'll be
   
    holding a webinar next week
   
   . We'll talk about the new features, go over the steps to follow for a safe and easy upgrade, and cover the "gotchas" to avoid as you upgrade.
  
  
   
    You should join us
   
   — March 31 from 1-3 (central). It'll be a blast.

Django 1.3 is out - time to upgrade!

<div class="w-block-markdown block-markdown">
 <div class="prose prose-lg max-w-none">
 
 Want to buff up your Django chops? Learn some cool new tricks for the new year? You should come to one of my new Django classes next month! I'll be teaching two new one-day classes, both of which evolved from common questions we get here at Revsys. Each class is going to be offered twice, once in LA and once in Boston.
 
 
 For the impatient here are some links and quick info:
 
 <blockquote>
 <ul>
 <li>
 <a href="https://deploydjango-la.eventbrite.com/">
 Django Deployment Workshop - Los Angeles
 </a>
 - December 6, 2010.
 </li>
 <li>
 <a href="https://deploydjango-bos.eventbrite.com/">
 Django Deployment Workshop - Boston
 </a>
 - December 10, 2010.
 </li>
 <li>
 <a href="https://djangoeco-la.eventbrite.com/">
 The Best of the Django Ecosystem - Los Angeles
 </a>
 - December 7, 2010.
 </li>
 <li>
 <a href="https://djangoeco-bos.eventbrite.com/">
 The Best of the Django Ecosystem - Boston
 </a>
 - December 9, 2010.
 </li>
 </ul>
 </blockquote>
 
 For more details on each class, read on...
 
 
 The first class attempts to answer this question: "So... I've written this cool Django site, and it works on my laptop... but now what?"
 
 
 We've increasingly been spending our time helping clients learn how to successfully deploy Django, so I've taken the things we've learned and turned them into the
 
 Django Deployment Workshop
 
 . You're invited to bring your own Django sites and I'll show you how to put them into production. At the end of the class, you'll walk away with a production environment that could easily serve most small-to-medium Django sites.
 
 
 To sign up:
 
 <blockquote>
 <ul>
 <li>
 <a href="https://deploydjango-la.eventbrite.com/">
 Django Deployment Workshop - Los Angeles
 </a>
 - December 6, 2010.
 </li>
 <li>
 <a href="https://deploydjango-bos.eventbrite.com/">
 Django Deployment Workshop - Boston
 </a>
 - December 10, 2010.
 </li>
 </ul>
 </blockquote>
 
 The second class focuses on Django's app ecosystem. Our clients tell us that they love all the apps available out there, but that they have a really hard time finding the
 
 best
 
 ones and figuring out how to use them. So I've put together
 
 The Best of the Django Ecosystem
 
 , a class that tries to answer both questions. We'll look at some of the coolest tools out there in the ecosystem (including
 <a href="http://south.aeracode.org/">
 South
 </a>
 ,
 <a href="http://celeryproject.org/">
 Celery
 </a>
 , and
 <a href="http://haystacksearch.org/">
 Haystack
 </a>
 ) and learn how to integrate them with an existing site. Again, you're invited to bring your existing Django site; I'll give plenty of time and hands-on help getting these apps wired up..
 
 
 Dates and details:
 
 <blockquote>
 <ul>
 <li>
 <a href="https://djangoeco-la.eventbrite.com/">
 The Best of the Django Ecosystem - Los Angeles
 </a>
 - December 7, 2010.
 </li>
 <li>
 <a href="https://djangoeco-bos.eventbrite.com/">
 The Best of the Django Ecosystem - Boston
 </a>
 - December 9, 2010.
 </li>
 </ul>
 </blockquote>
 
 
 Early-bird registration is open now
 
 . You'll save over $100 if you register before November 22nd, so get on it. There's also a combo ticket available in both cities if you'd like to attend both classes and group rates if you'd like to bring the whole team.
 
 
 Oh, and to answer the inevitable FAQ: yes, we'd love to bring these classes to other cities! We're looking at a bunch of locations for 2011, and if you'd like to throw your hometown in the ring we've got a
 <a href="https://spreadsheets.google.com/a/jacobian.org/viewform?hl=en&amp;formkey=cmJSYWNvZEZqN085QjBCaUtKRTh4bUE6MA..">
 quick little survey
 </a>
 to help us figure out where to host future classes.
 
 
 If you have other questions, please
 <a href="/contact/">
 get in touch
 </a>
 , or drop into our
 <a href="https://www.revsys.com/blog/2010/nov/02/open-office-hours/">
 office hours
 </a>
 tomorrow afternoon.
 
 </div>
</div>

Want to buff up your Django chops? Learn some cool new tricks for the new year? You should come to one of my new Django classes next month! I'll be teaching two new one-day classes, both of which evolved from common questions we get here at Revsys. Each class is going to be offered twice, once in LA and once in Boston.
  
  
   For the impatient here are some links and quick info:
  
  
   
    
     
      Django Deployment Workshop - Los Angeles
     
     - December 6, 2010.
    
    
     
      Django Deployment Workshop - Boston
     
     - December 10, 2010.
    
    
     
      The Best of the Django Ecosystem - Los Angeles
     
     - December 7, 2010.
    
    
     
      The Best of the Django Ecosystem - Boston
     
     - December 9, 2010.
    
   
  
  
   For more details on each class, read on...
  
  
   The first class attempts to answer this question: "So... I've written this cool Django site, and it works on my laptop... but now what?"
  
  
   We've increasingly been spending our time helping clients learn how to successfully deploy Django, so I've taken the things we've learned and turned them into the
   
    Django Deployment Workshop
   
   . You're invited to bring your own Django sites and I'll show you how to put them into production. At the end of the class, you'll walk away with a production environment that could easily serve most small-to-medium Django sites.
  
  
   To sign up:
  
  
   
    
     
      Django Deployment Workshop - Los Angeles
     
     - December 6, 2010.
    
    
     
      Django Deployment Workshop - Boston
     
     - December 10, 2010.
    
   
  
  
   The second class focuses on Django's app ecosystem. Our clients tell us that they love all the apps available out there, but that they have a really hard time finding the
   
    best
   
   ones and figuring out how to use them. So I've put together
   
    The Best of the Django Ecosystem
   
   , a class that tries to answer both questions. We'll look at some of the coolest tools out there in the ecosystem (including
   
    South
   
   ,
   
    Celery
   
   , and
   
    Haystack
   
   ) and learn how to integrate them with an existing site. Again, you're invited to bring your existing Django site; I'll give plenty of time and hands-on help getting these apps wired up..
  
  
   Dates and details:
  
  
   
    
     
      The Best of the Django Ecosystem - Los Angeles
     
     - December 7, 2010.
    
    
     
      The Best of the Django Ecosystem - Boston
     
     - December 9, 2010.
    
   
  
  
   
    Early-bird registration is open now
   
   . You'll save over $100 if you register before November 22nd, so get on it. There's also a combo ticket available in both cities if you'd like to attend both classes and group rates if you'd like to bring the whole team.
  
  
   Oh, and to answer the inevitable FAQ: yes, we'd love to bring these classes to other cities! We're looking at a bunch of locations for 2011, and if you'd like to throw your hometown in the ring we've got a
   
    quick little survey
   
   to help us figure out where to host future classes.
  
  
   If you have other questions, please
   
    get in touch
   
   , or drop into our
   
    office hours
   
   tomorrow afternoon.

Two new Django classes: deployment and the ecosystem

<div class="w-block-markdown block-markdown">
 <div class="prose prose-lg max-w-none">
 
 Packt Publishing recently asked me to read and review
 <a href="https://www.packtpub.com/django-1-1-testing-and-debugging/book">
 Django 1.1 Testing and Debugging
 </a>
 and I have to admit I really enjoyed reading this book. Often I find myself debating whether or not to purchase a new development book. I'm usually thinking "If I spend $XX.XX on this book, will I really learn anything worth that much?". Especially considering most answers are a few Google searches away. I can happily attest this book is definitely worth the cost.
 
 
 The book starts off with the usual introduction to testing, discussing both Doctests and Unittest, which is obviously required for a book on this subject. However, this book differs greatly from many in that it walks you through testing and debugging your application as you would when building a
 
 real
 
 application. Many tech books strive to be a great tutorial, but often fall short of the mark and end up just being some verbiage around a rehashing of the available documentation. They end up being more reference than tutorial.
 <a href="https://www.packtpub.com/django-1-1-testing-and-debugging/book">
 Django 1.1 Testing and Debugging
 </a>
 however does a great job of walking you through real world scenarios. For example, it covers topics (and in the proper order in my opinion) like:
 
 <ul>
 <li>
 Test coverage
 </li>
 <li>
 Everything you would ever want to know about the default Django debug page
 </li>
 <li>
 How to debug urls.py issues
 </li>
 <li>
 How debugging differs between the development server and what to do in production
 </li>
 <li>
 Getting ahold of what database queries were actually generated on a given page
 </li>
 <li>
 Using the wonderful
 <a href="http://github.com/robhudson/django-debug-toolbar">
 Django Debug Toolbar
 </a>
 </li>
 <li>
 Using logging to help debug your code
 </li>
 <li>
 Effectively using the Python Debugger
 </li>
 <li>
 How to report Django bugs
 </li>
 </ul>
 
 All of these tools and techniques should be in all of our development arsenals, but this is the first book I've seen that puts them all together in a way that is accessible to Django developers of nearly any skill level. Beginners should read the book, even if they don't understand everything right away, just to know what options they do have at their disposal. However, more advanced developers will likely find a few golden nuggets of wisdom that will pay productivity dividends for years to come.
 
 
 Hence the title of this post
 
 Investing in Yourself
 
 . While Packt was nice enough to give me a free copy of the book to review, had I paid full list price the time savings from the few little nuggets of wisdom I either didn't know about or for whatever reason never clicked with me before would have paid for the book many times over in just the first few weeks.
 
 
 I highly encourage you to pick this up and with it improve your code going forward. You can check out a
 <a href="https://www.packtpub.com/sites/default/files/7566_Django%201.1%20Testing%20and%20Debugging_SampleChapter_1.pdf">
 sample chapter
 </a>
 of the book to whet your appetite a bit.
 
 </div>
</div>

Packt Publishing recently asked me to read and review
   
    Django 1.1 Testing and Debugging
   
   and I have to admit I really enjoyed reading this book. Often I find myself debating whether or not to purchase a new development book. I'm usually thinking "If I spend $XX.XX on this book, will I really learn anything worth that much?". Especially considering most answers are a few Google searches away. I can happily attest this book is definitely worth the cost.
  
  
   The book starts off with the usual introduction to testing, discussing both Doctests and Unittest, which is obviously required for a book on this subject. However, this book differs greatly from many in that it walks you through testing and debugging your application as you would when building a
   
    real
   
   application. Many tech books strive to be a great tutorial, but often fall short of the mark and end up just being some verbiage around a rehashing of the available documentation. They end up being more reference than tutorial.
   
    Django 1.1 Testing and Debugging
   
   however does a great job of walking you through real world scenarios. For example, it covers topics (and in the proper order in my opinion) like:
  
  
   
    Test coverage
   
   
    Everything you would ever want to know about the default Django debug page
   
   
    How to debug urls.py issues
   
   
    How debugging differs between the development server and what to do in production
   
   
    Getting ahold of what database queries were actually generated on a given page
   
   
    Using the wonderful
    
     Django Debug Toolbar
    
   
   
    Using logging to help debug your code
   
   
    Effectively using the Python Debugger
   
   
    How to report Django bugs
   
  
  
   All of these tools and techniques should be in all of our development arsenals, but this is the first book I've seen that puts them all together in a way that is accessible to Django developers of nearly any skill level. Beginners should read the book, even if they don't understand everything right away, just to know what options they do have at their disposal. However, more advanced developers will likely find a few golden nuggets of wisdom that will pay productivity dividends for years to come.
  
  
   Hence the title of this post
   
    Investing in Yourself
   
   . While Packt was nice enough to give me a free copy of the book to review, had I paid full list price the time savings from the few little nuggets of wisdom I either didn't know about or for whatever reason never clicked with me before would have paid for the book many times over in just the first few weeks.
  
  
   I highly encourage you to pick this up and with it improve your code going forward. You can check out a
   
    sample chapter
   
   of the book to whet your appetite a bit.

Investing in Yourself - A review of Django 1.1 Testing and Debugging by Karen M. Tracey

<div class="w-block-markdown block-markdown">
 <div class="prose prose-lg max-w-none">
 
 Early registration ends Friday for the March
 <a href="http://www.revsys.com/training/django/march-2010/">
 Advanced Django Class
 </a>
 I'm teaching, so if you're planning on coming, you should
 <a href="http://revys-training-march-2010.eventbrite.com/">
 sign up
 </a>
 soon!
 
 
 I'm really excited about this class: we'll get to dive really deep into the good parts of Django. We'll cover all sorts of advanced usage of Django's APIs, spend a bunch of time playing with all the cool stuff out there in the Django ecosystem, and actually spend a whole day setting up and configuring a real-world deployment stack.
 
 
 Many more details are over
 <a href="http://www.revsys.com/training/django/march-2010/">
 on the class description page
 </a>
 , along with
 <a href="http://www.revsys.com/training/django/march-2010/#questions">
 contact info
 </a>
 if you've got any questions.
 
 </div>
</div>

Early registration ends Friday for the March
   
    Advanced Django Class
   
   I'm teaching, so if you're planning on coming, you should
   
    sign up
   
   soon!
  
  
   I'm really excited about this class: we'll get to dive really deep into the good parts of Django. We'll cover all sorts of advanced usage of Django's APIs, spend a bunch of time playing with all the cool stuff out there in the Django ecosystem, and actually spend a whole day setting up and configuring a real-world deployment stack.
  
  
   Many more details are over
   
    on the class description page
   
   , along with
   
    contact info
   
   if you've got any questions.

Early registration for our Advanced Django class ends soon

<div class="w-block-markdown block-markdown">
 <div class="prose prose-lg max-w-none">
 
 The new year's shaping up to be a great one for Django: Django 1.2 is on track to ship this March, and there's no doubt in my mind that it'll be the best release ever.
 
 
 Because of this, we expect to see a lot of new people wanting to learn Django next year, so we're stepping up our
 <a href="https://revsys.com/training/django/">
 training offerings
 </a>
 in the new year.
 
 
 This Friday, I'll be holding
 <a href="https://hwebdjmc01.eventbrite.com/">
 a one-day Django workshop in New York City
 </a>
 . The class is mostly full, but
 <a href="https://hwebdjmc01.eventbrite.com/">
 a few spots are still available
 </a>
 . This is the second in a series of workshops we've been holding in conjunction with
 <a href="https://holdenweb.com/">
 HoldenWeb
 </a>
 , and we plan to offer more (see below)
 
 
 In March, I'll be teaching
 <a href="https://www.revsys.com/training/django/march-2010/">
 a week-long advanced Django class
 </a>
 in Kansas City. This class is perfect for folks who know Django but want to
 
 really
 
 learn all the ins and outs. We'll look at a whole bunch of advanced uses of Django, learn how to use optional add-ons like GeoDjango, Piston, Haystack, and even dig into the internals of Django a bit. We'll also spend a whole day setting up a real-world Django production environment and tuning it for performance.
 
 
 I'm really looking forward to this class: there's some really fantastic bits buried deep in Django that I don't get much call to talk about.
 
 
 Finally, we're currently scouting locations and topics for future classes and
 <a href="https://spreadsheets.google.com/a/jacobian.org/viewform?hl=en&amp;formkey=cmJSYWNvZEZqN085QjBCaUtKRTh4bUE6MA..">
 would like your input
 </a>
 ! If you'd be interested in attending a future Django class there's
 <a href="https://spreadsheets.google.com/a/jacobian.org/viewform?hl=en&amp;formkey=cmJSYWNvZEZqN085QjBCaUtKRTh4bUE6MA..">
 a quick questionnaire you can fill out
 </a>
 .
 
 </div>
</div>

The new year's shaping up to be a great one for Django: Django 1.2 is on track to ship this March, and there's no doubt in my mind that it'll be the best release ever.
  
  
   Because of this, we expect to see a lot of new people wanting to learn Django next year, so we're stepping up our
   
    training offerings
   
   in the new year.
  
  
   This Friday, I'll be holding
   
    a one-day Django workshop in New York City
   
   . The class is mostly full, but
   
    a few spots are still available
   
   . This is the second in a series of workshops we've been holding in conjunction with
   
    HoldenWeb
   
   , and we plan to offer more (see below)
  
  
   In March, I'll be teaching
   
    a week-long advanced Django class
   
   in Kansas City. This class is perfect for folks who know Django but want to
   
    really
   
   learn all the ins and outs. We'll look at a whole bunch of advanced uses of Django, learn how to use optional add-ons like GeoDjango, Piston, Haystack, and even dig into the internals of Django a bit. We'll also spend a whole day setting up a real-world Django production environment and tuning it for performance.
  
  
   I'm really looking forward to this class: there's some really fantastic bits buried deep in Django that I don't get much call to talk about.
  
  
   Finally, we're currently scouting locations and topics for future classes and
   
    would like your input
   
   ! If you'd be interested in attending a future Django class there's
   
    a quick questionnaire you can fill out
   
   .

Django Training in 2010

<div class="w-block-markdown block-markdown">
 <div class="prose prose-lg max-w-none">
 
 <a href="http://www.packtpub.com/django-1.0-template-design-practical-guide/mid/210909irxdzd?utm_source=revsys.com&amp;utm_medium=bookrev&amp;utm_content=other&amp;utm_campaign=mdb_000707">
 Django 1.0 Template Development
 </a>
 by Scott Newman actually surprised me in it's depth of covering the topic.
 
 
 I assumed it would be written with the absolute Django Template beginner in mind. While it is definitely an appropriate book for beginners, it also covers more advanced topics such as:
 
 <ul>
 <li>
 Writing custom template tags and filters, Chapter 7
 </li>
 <li>
 Covers pagination quite well in Chapter 8
 </li>
 <li>
 Customizing the Django admin's look and feel in Chapter 9
 </li>
 <li>
 And gives a good designer intro to caching in Chapter 10
 </li>
 </ul>
 
 This book should definitely be required reading for web designers that are looking to use Django. As someone who has read and re-read the wonderful
 <a href="http://www.djangoproject.com/en/dev/">
 Django Documentation
 </a>
 many times this book does a great job of distilling all of that reference knowledge into book form.
 
 </div>
</div>

Django 1.0 Template Development
   
   by Scott Newman actually surprised me in it's depth of covering the topic.
  
  
   I assumed it would be written with the absolute Django Template beginner in mind. While it is definitely an appropriate book for beginners, it also covers more advanced topics such as:
  
  
   
    Writing custom template tags and filters, Chapter 7
   
   
    Covers pagination quite well in Chapter 8
   
   
    Customizing the Django admin's look and feel in Chapter 9
   
   
    And gives a good designer intro to caching in Chapter 10
   
  
  
   This book should definitely be required reading for web designers that are looking to use Django. As someone who has read and re-read the wonderful
   
    Django Documentation
   
   many times this book does a great job of distilling all of that reference knowledge into book form.

Django 1.0 Template Development Review

<div class="w-block-markdown block-markdown">
 <div class="prose prose-lg max-w-none">
 
 I'm pleased to announce that this September, Revolution Systems will be offering a
 <a href="https://www.revsys.com/training/django/sept-2009/">
 four-day Django training course
 </a>
 here in Kansas City, taught by Jacob Kaplan-Moss. Jacob has taught a number of similar courses privately in the past, but this is the first time we're opening this up to the public.
 
 
 If you've been wanting to really learn Django, this is a great chance. The class is intended for relative beginners to Django, but by the time you leave you'll know enough to create some pretty awesome websites.
 
 
 The class will be Monday, September 21 - Friday, September 25, 2009 in Kansas City, MO.
 
 
 For more, or to sign up,
 <a href="https://www.revsys.com/training/django/sept-2009/">
 check out the full details
 </a>
 .
 
 </div>
</div>

I'm pleased to announce that this September, Revolution Systems will be offering a
   
    four-day Django training course
   
   here in Kansas City, taught by Jacob Kaplan-Moss. Jacob has taught a number of similar courses privately in the past, but this is the first time we're opening this up to the public.
  
  
   If you've been wanting to really learn Django, this is a great chance. The class is intended for relative beginners to Django, but by the time you leave you'll know enough to create some pretty awesome websites.
  
  
   The class will be Monday, September 21 - Friday, September 25, 2009 in Kansas City, MO.
  
  
   For more, or to sign up,
   
    check out the full details
   
   .

Django Training

<div class="w-block-markdown block-markdown">
 <div class="prose prose-lg max-w-none">
 
 <a href="http://www.linuxpromagazine.com">
 LinuxPro Magazine
 </a>
 just released my latest article, an introduction to
 <a href="http://www.djangoproject.com">
 Django
 </a>
 and some discussion about the newly created
 <a href="http://www.djangoproject.com/foundation/">
 Django Software Foundation
 </a>
 . Being a life long Perl user, I didn't think I would enjoy Django at all. I have to admit that it is a VERY polished system. It has great PostgreSQL support, in fact the core developers smartly prefer it over MySQL for their own systems.
 
 
 You can download a PDF copy of the article at,
 <a href="http://www.linuxpromagazine.com/issues/2008/95/fret_free">
 Fret Free -- Django and the Django Software Foundation
 </a>
 . The print issue will hit the stands in October. Hope you enjoy it!
 
 </div>
</div>

LinuxPro Magazine
   
   just released my latest article, an introduction to
   
    Django
   
   and some discussion about the newly created
   
    Django Software Foundation
   
   . Being a life long Perl user, I didn't think I would enjoy Django at all. I have to admit that it is a VERY polished system.  It has great PostgreSQL support, in fact the core developers smartly prefer it over MySQL for their own systems.
  
  
   You can download a PDF copy of the article at,
   
    Fret Free -- Django and the Django Software Foundation
   
   .  The print issue will hit the stands in October.  Hope you enjoy it!

Fret Free -- Introduction to Django and the Django Software Foundation

<div class="w-block-markdown block-markdown">
 <div class="prose prose-lg max-w-none">
 
 The
 <a href="http://www.djangoproject.com">
 Django
 </a>
 guys have scored another milestone today with the creation of the
 <a href="http://www.djangoproject.com/foundation/">
 Django Software Foundation
 </a>
 . Being a huge Open Source advocate I love hearing great news like this. You can read more about the foundation at:
 
 <ul>
 <li>
 <a href="http://www.djangoproject.com/weblog/2008/jun/17/foundation/">
 Announcing the Django Software Foundation
 </a>
 </li>
 <li>
 <a href="http://www2.ljworld.com/news/2008/jun/17/new_foundation_django/">
 New Foundation for Django
 </a>
 </li>
 </ul>
 
 On an unrelated note, an article I wrote for
 <a href="http://www.linux-magazine.com">
 LinuxPro Magazine
 </a>
 last November about using
 <a href="http://www.danga.com/perlbal">
 Perlbal
 </a>
 was recently put online. You can read it at
 <a href="http://w3.linux-magazine.com/issue/84/Perlbal.pdf">
 The Juggler -- Let the nimble Perlbal webserver keep your traffic in balance
 </a>
 .
 
 </div>
</div>

The
   
    Django
   
   guys have scored another milestone today with the creation of the
   
    Django Software Foundation
   
   . Being a huge Open Source advocate I love hearing great news like this.  You can read more about the foundation at:
  
  
   
    
     Announcing the Django Software Foundation
    
   
   
    
     New Foundation for Django
    
   
  
  
   On an unrelated note, an article I wrote for
   
    LinuxPro Magazine
   
   last November about using
   
    Perlbal
   
   was recently put online.  You can read it at
   
    The Juggler -- Let the nimble Perlbal webserver keep your traffic in balance
   
   .

Django Software Foundation

<div class="medium-feed-item"><a href="https://python.plainenglish.io/the-python-data-science-stack-is-changing-heres-what-developers-should-learn-in-2026-0743af8c8b73?source=rss----78073def27b8---4"><img src="https://cdn-images-1.medium.com/max/1400/0*rJHcKMAOKTFZD5fb" width="1400"></a>For years, learning Python for data science meant learning a fairly predictable stack.<a href="https://python.plainenglish.io/the-python-data-science-stack-is-changing-heres-what-developers-should-learn-in-2026-0743af8c8b73?source=rss----78073def27b8---4">Continue reading on Python in Plain English »</a></div>

For years, learning Python for data science meant learning a fairly predictable stack.
Continue reading on Python in Plain English »

The Python Data Science Stack Is Changing. Here’s What Developers Should Learn in 2026

<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*IeJtLTnT09AZhbu0HbVfqA.png" /></figure><h3>Can You Explain Your Code? And Does it Work?</h3>A Python script can run without its author fully understanding it.Copy an example, change a few variables, and get the expected output. That is a useful starting point. But change the input or ask what happens when something goes wrong and the gaps become visible.Consider a small task: replacing spaces in filenames with underscores.Before running that script on your Downloads folder, could you explain what happens if two files end up with the same name? Whether folders are affected? Or how you would check the results without changing anything?Those questions turn a coding exercise into an engineering discussion.<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*k6lALR3mF5gLHQPFGjZNHA.png" /><figcaption>Image Generated by AI</figcaption></figure>Let’s work through a small example you can inspect, run, and modify. It previews filename changes without actually renaming files.<h3>Start with a result you can describe</h3>Our requirements are deliberately narrow:<ul><li>Examine files directly inside one folder.</li><li>Replace spaces in each filename with underscores.</li><li>Leave subfolders alone.</li><li>Flag proposed names that already exist.</li><li>Print the plan without changing files.</li></ul>For example:<pre>meeting notes.txt → meeting_notes.txt</pre>But if meeting_notes.txt already exists, the script should report a conflict.Writing these rules first gives us something concrete to check. “It works” now means more than “Python didn’t display an error.”<h3>Build a safe folder to experiment with</h3>Save this setup code as make_demo.py:<pre>from pathlib import Path from tempfile import mkdtemp</pre><pre>folder = Path(mkdtemp(prefix=&quot;python_rename_demo_&quot;))</pre><pre>samples = { &quot;meeting notes.txt&quot;: &quot;Discuss the next project.&quot;, &quot;shopping list.txt&quot;: &quot;Rice, beans, coffee.&quot;, &quot;shopping_list.txt&quot;: &quot;An existing file.&quot;, &quot;readme.txt&quot;: &quot;Already has a simple name.&quot;, }</pre><pre>for filename, content in samples.items(): (folder / filename).write_text(content, encoding=&quot;utf-8&quot;)</pre><pre>(folder / &quot;project files&quot;).mkdir()</pre><pre>print(f&quot;Demo folder: {folder}&quot;)</pre>Run it:<pre>python make_demo.py</pre>Depending on your installation, you may need python3 instead of python.The setup creates a new temporary folder and prints its location. It includes an intentional conflict: shopping list.txt and shopping_list.txt.We also created a subfolder so we can check whether our preview leaves directories alone.Use the printed folder path in the next step.<h3>Write the preview</h3>Save this as rename_preview.py:<pre>import argparse from pathlib import Path </pre><pre>def preview_renames(folder: Path) -&gt; None: if not folder.is_dir(): raise ValueError(f&quot;Not an existing folder: {folder}&quot;)</pre><pre> for source in sorted(folder.iterdir()): if source.is_symlink() or not source.is_file(): continue</pre><pre> new_name = source.name.replace(&quot; &quot;, &quot;_&quot;)</pre><pre> if new_name == source.name: continue</pre><pre> target = source.with_name(new_name)</pre><pre> if target.exists() or target.is_symlink(): print( f&quot;CONFLICT: {source.name!r} -&gt; {new_name!r} &quot; &quot;(destination exists)&quot; ) continue</pre><pre> print(f&quot;PREVIEW: {source.name!r} -&gt; {new_name!r}&quot;) </pre><pre>def main() -&gt; None: parser = argparse.ArgumentParser( description=&quot;Preview filename changes without renaming files.&quot; ) parser.add_argument( &quot;folder&quot;, type=Path, help=&quot;Folder containing the files to inspect.&quot;, ) args = parser.parse_args()</pre><pre> try: preview_renames(args.folder) except (ValueError, OSError) as error: parser.exit(1, f&quot;Error: {error}\n&quot;) </pre><pre>if __name__ == &quot;__main__&quot;: main()</pre>Run it with the folder path printed by the setup script:<pre>python rename_preview.py &quot;/path/to/your/demo-folder&quot;</pre>The expected output for our sample files is:<pre>PREVIEW: &#39;meeting notes.txt&#39; -&gt; &#39;meeting_notes.txt&#39; CONFLICT: &#39;shopping list.txt&#39; -&gt; &#39;shopping_list.txt&#39; (destination exists)</pre>Nothing is renamed. The preview contains no call to rename(), replace(), or a file-writing method.That distinction matters: this script describes a proposed action. It does not perform it.<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*KZdWDg4t3-gNKSiZc1gF9w.png" /><figcaption>Image Generated With AI</figcaption></figure><h3>Question 1: What does the script receive?</h3>The input is a folder path supplied on the command line:<pre>parser.add_argument(&quot;folder&quot;, type=Path)</pre>argparse handles the command-line argument. Path gives us an object with filesystem operations, rather than making us assemble paths by hand.The next check establishes a basic requirement:<pre>if not folder.is_dir(): raise ValueError(f&quot;Not an existing folder: {folder}&quot;)</pre>If someone supplies a missing folder or a file path, the script reports an error.That is a better explanation than “this part imports a library.” It connects the implementation to a requirement.<h3>Question 2: What does it deliberately ignore?</h3>This line examines only the folder’s immediate contents:<pre>folder.iterdir()</pre>It does not recursively walk through subfolders.Then we skip symbolic links and anything that is not a regular file:<pre>if source.is_symlink() or not source.is_file(): continue</pre>A symbolic link points to another filesystem location. Excluding links keeps this beginner exercise easier to reason about.We also skip filenames that would remain unchanged:<pre>if new_name == source.name: continue</pre>That is why readme.txt does not appear in the output.Being able to explain what your program excludes is part of understanding its behavior.<h3>Question 3: What could go wrong?</h3>The obvious problem is a naming conflict:<pre>shopping list.txt shopping_list.txt</pre>Replacing spaces in the first name produces the second name.We check for that before printing an ordinary preview:<pre>if target.exists() or target.is_symlink():</pre>The symbolic-link check also catches a destination that is a broken link, which exists() alone may not identify as existing.But this check has limits.It checks the filesystem at the moment the preview runs. Another program could create a file afterward. Filesystem case sensitivity can also affect whether two names count as the same.If you later add more transformations — lowercasing, removing punctuation, or collapsing repeated underscores — multiple source files may map to the same destination even when that destination does not exist yet. You would need to validate the complete plan for duplicate targets.A good explanation includes those boundaries. A preview is useful; it is not a guarantee that every future rename will succeed.<h3>Question 4: How do you know it behaves correctly?</h3>Start with specific cases instead of repeatedly trying one convenient folder.CaseExpected behaviorFilename contains a spacePrint the proposed nameFilename already uses underscoresSkip itProposed destination existsPrint a conflictEntry is a subfolderSkip itFolder is emptyPrint no proposalsFolder does not existReport an errorRun the preview twiceLeave filenames unchanged both timesOur demo covers several of these immediately.You can also run:<pre>python rename_preview.py --help</pre>That checks whether a new user can discover how to use the script without reading its source.For a larger project, turn these cases into automated tests. For this exercise, the important habit is making your expectations explicit before interpreting the results.<h3>Question 5: Can you explain a design decision?</h3>Imagine someone asks:“Why didn’t you just rename the files?”A useful answer would be:<blockquote>I separated inspection from modification so the user can review the proposed changes first. This version only prints a plan. A version that performs renames would need additional handling for conflicts, permissions, and files changing between inspection and execution.</blockquote>That answer explains the tradeoff.It is more informative than “I used pathlib because it is cleaner,” and more honest than calling a short demonstration production-ready.The same habit applies to other projects:<ul><li>Why did you choose a CSV file instead of a database?</li><li>Why does this function stop on an error?</li><li>Why did you avoid a third-party dependency?</li><li>Which inputs did you check, and which remain unsupported?</li></ul>Your choices become easier to discuss when you can connect them to a real constraint.<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7pwU5lJ__UCBPDlBMdc_qg.png" /><figcaption>Image Generated With AI</figcaption></figure><h3>Use AI to question your understanding</h3>AI can help you generate a first draft. The next step is to interrogate it.Try prompts such as:<ul><li>“Which assumptions does this script make about its input?”</li><li>“Give me three cases that could break this approach.”</li><li>“Does this code change files? Identify the exact operations.”</li><li>“Explain why checking a destination now does not guarantee it will be available later.”</li></ul>Then verify the answers against the code, documentation, and your own experiments.A fluent explanation from an AI is not evidence that the program behaves as described. The output still needs checking.<h3>Try one change yourself</h3>Modify the preview so it considers only files whose extension is .txt, including .TXT.Hint: inspect this expression:<pre>source.suffix.lower()</pre>Before running your change, write down:<ol><li>Where the new condition belongs.</li><li>Which files should now be skipped.</li><li>A sample filename that will prove your condition works.</li></ol>Then add holiday photo.jpg and meeting notes.TXT to your demo and compare the output with your prediction.That small cycle — predict, change, run, explain — is a practical way to move beyond copying code.<h3>Practice a two-minute explanation</h3>For this project, your explanation might sound like this:<blockquote>The script previews filename changes inside one folder. It replaces spaces with underscores, skips directories and symbolic links, and flags destinations that already exist. It prints the proposed changes without renaming anything. I would check ordinary filenames, conflicts, empty folders, and invalid paths. Before adding actual renaming, I would address execution-time errors and validate the complete plan.</blockquote>You do not need to memorize those words. You need to understand the decisions behind them.Being able to explain a small, working program gives you something concrete to discuss. It does not guarantee a job, but it makes your skills easier for someone else to evaluate.<h3>Want another small project to practice with?</h3>My <a href="https://denzell84.gumroad.com/l/lvwvn?utm_source=medium&amp;utm_medium=article&amp;utm_campaign=explain_your_code">Python Weekend Cheat Sheet is available for $7 on Gumroad</a>. Check the product page for what is included before deciding whether it fits your next step.You can complete everything in this article without buying it.What happened when you tried the preview — and which case would you check next?<img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1774e04228d0" width="1" height="1" alt=""><hr><a href="https://python.plainenglish.io/the-python-skill-employers-actually-notice-1774e04228d0">The Python Skill Employers Actually Notice:</a> was originally published in <a href="https://python.plainenglish.io">Python in Plain English</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.

Can You Explain Your Code? And Does it Work?
A Python script can run without its author fully understanding it.
Copy an example, change a few variables, and get the expected output. That is a useful starting point. But change the input or ask what happens when something goes wrong and the gaps become visible.
Consider a small task: replacing spaces in filenames with underscores.
Before running that script on your Downloads folder, could you explain what happens if two files end up with the same name? Whether folders are affected? Or how you would check the results without changing anything?
Those questions turn a coding exercise into an engineering discussion.
Image Generated by AI
Let’s work through a small example you can inspect, run, and modify. It previews filename changes without actually renaming files.
Start with a result you can describe
Our requirements are deliberately narrow:

Examine files directly inside one folder.
Replace spaces in each filename with underscores.
Leave subfolders alone.
Flag proposed names that already exist.
Print the plan without changing files.

For example:
meeting notes.txt → meeting_notes.txt
But if meeting_notes.txt already exists, the script should report a conflict.
Writing these rules first gives us something concrete to check. “It works” now means more than “Python didn’t display an error.”
Build a safe folder to experiment with
Save this setup code as make_demo.py:
from pathlib import Path
from tempfile import mkdtemp
folder = Path(mkdtemp(prefix="python_rename_demo_"))
samples = {
    "meeting notes.txt": "Discuss the next project.",
    "shopping list.txt": "Rice, beans, coffee.",
    "shopping_list.txt": "An existing file.",
    "readme.txt": "Already has a simple name.",
}
for filename, content in samples.items():
    (folder / filename).write_text(content, encoding="utf-8")
(folder / "project files").mkdir()
print(f"Demo folder: {folder}")
Run it:
python make_demo.py
Depending on your installation, you may need python3 instead of python.
The setup creates a new temporary folder and prints its location. It includes an intentional conflict: shopping list.txt and shopping_list.txt.
We also created a subfolder so we can check whether our preview leaves directories alone.
Use the printed folder path in the next step.
Write the preview
Save this as rename_preview.py:
import argparse
from pathlib import Path

def preview_renames(folder: Path) -> None:
    if not folder.is_dir():
        raise ValueError(f"Not an existing folder: {folder}")
    for source in sorted(folder.iterdir()):
        if source.is_symlink() or not source.is_file():
            continue
        new_name = source.name.replace(" ", "_")
        if new_name == source.name:
            continue
        target = source.with_name(new_name)
        if target.exists() or target.is_symlink():
            print(
                f"CONFLICT: {source.name!r} -> {new_name!r} "
                "(destination exists)"
            )
            continue
        print(f"PREVIEW: {source.name!r} -> {new_name!r}")

def main() -> None:
    parser = argparse.ArgumentParser(
        description="Preview filename changes without renaming files."
    )
    parser.add_argument(
        "folder",
        type=Path,
        help="Folder containing the files to inspect.",
    )
    args = parser.parse_args()
    try:
        preview_renames(args.folder)
    except (ValueError, OSError) as error:
        parser.exit(1, f"Error: {error}\n")

if __name__ == "__main__":
    main()
Run it with the folder path printed by the setup script:
python rename_preview.py "/path/to/your/demo-folder"
The expected output for our sample files is:
PREVIEW: 'meeting notes.txt' -> 'meeting_notes.txt'
CONFLICT: 'shopping list.txt' -> 'shopping_list.txt' (destination exists)
Nothing is renamed. The preview contains no call to rename(), replace(), or a file-writing method.
That distinction matters: this script describes a proposed action. It does not perform it.
Image Generated With AI
Question 1: What does the script receive?
The input is a folder path supplied on the command line:
parser.add_argument("folder", type=Path)
argparse handles the command-line argument. Path gives us an object with filesystem operations, rather than making us assemble paths by hand.
The next check establishes a basic requirement:
if not folder.is_dir():
    raise ValueError(f"Not an existing folder: {folder}")
If someone supplies a missing folder or a file path, the script reports an error.
That is a better explanation than “this part imports a library.” It connects the implementation to a requirement.
Question 2: What does it deliberately ignore?
This line examines only the folder’s immediate contents:
folder.iterdir()
It does not recursively walk through subfolders.
Then we skip symbolic links and anything that is not a regular file:
if source.is_symlink() or not source.is_file():
    continue
A symbolic link points to another filesystem location. Excluding links keeps this beginner exercise easier to reason about.
We also skip filenames that would remain unchanged:
if new_name == source.name:
    continue
That is why readme.txt does not appear in the output.
Being able to explain what your program excludes is part of understanding its behavior.
Question 3: What could go wrong?
The obvious problem is a naming conflict:
shopping list.txt
shopping_list.txt
Replacing spaces in the first name produces the second name.
We check for that before printing an ordinary preview:
if target.exists() or target.is_symlink():
The symbolic-link check also catches a destination that is a broken link, which exists() alone may not identify as existing.
But this check has limits.
It checks the filesystem at the moment the preview runs. Another program could create a file afterward. Filesystem case sensitivity can also affect whether two names count as the same.
If you later add more transformations — lowercasing, removing punctuation, or collapsing repeated underscores — multiple source files may map to the same destination even when that destination does not exist yet. You would need to validate the complete plan for duplicate targets.
A good explanation includes those boundaries. A preview is useful; it is not a guarantee that every future rename will succeed.
Question 4: How do you know it behaves correctly?
Start with specific cases instead of repeatedly trying one convenient folder.
CaseExpected behaviorFilename contains a spacePrint the proposed nameFilename already uses underscoresSkip itProposed destination existsPrint a conflictEntry is a subfolderSkip itFolder is emptyPrint no proposalsFolder does not existReport an errorRun the preview twiceLeave filenames unchanged both times
Our demo covers several of these immediately.
You can also run:
python rename_preview.py --help
That checks whether a new user can discover how to use the script without reading its source.
For a larger project, turn these cases into automated tests. For this exercise, the important habit is making your expectations explicit before interpreting the results.
Question 5: Can you explain a design decision?
Imagine someone asks:
“Why didn’t you just rename the files?”
A useful answer would be:
I separated inspection from modification so the user can review the proposed changes first. This version only prints a plan. A version that performs renames would need additional handling for conflicts, permissions, and files changing between inspection and execution.
That answer explains the tradeoff.
It is more informative than “I used pathlib because it is cleaner,” and more honest than calling a short demonstration production-ready.
The same habit applies to other projects:

Why did you choose a CSV file instead of a database?
Why does this function stop on an error?
Why did you avoid a third-party dependency?
Which inputs did you check, and which remain unsupported?

Your choices become easier to discuss when you can connect them to a real constraint.
Image Generated With AI
Use AI to question your understanding
AI can help you generate a first draft. The next step is to interrogate it.
Try prompts such as:

“Which assumptions does this script make about its input?”
“Give me three cases that could break this approach.”
“Does this code change files? Identify the exact operations.”
“Explain why checking a destination now does not guarantee it will be available later.”

Then verify the answers against the code, documentation, and your own experiments.
A fluent explanation from an AI is not evidence that the program behaves as described. The output still needs checking.
Try one change yourself
Modify the preview so it considers only files whose extension is .txt, including .TXT.
Hint: inspect this expression:
source.suffix.lower()
Before running your change, write down:

Where the new condition belongs.
Which files should now be skipped.
A sample filename that will prove your condition works.

Then add holiday photo.jpg and meeting notes.TXT to your demo and compare the output with your prediction.
That small cycle — predict, change, run, explain — is a practical way to move beyond copying code.
Practice a two-minute explanation
For this project, your explanation might sound like this:
The script previews filename changes inside one folder. It replaces spaces with underscores, skips directories and symbolic links, and flags destinations that already exist. It prints the proposed changes without renaming anything. I would check ordinary filenames, conflicts, empty folders, and invalid paths. Before adding actual renaming, I would address execution-time errors and validate the complete plan.
You do not need to memorize those words. You need to understand the decisions behind them.
Being able to explain a small, working program gives you something concrete to discuss. It does not guarantee a job, but it makes your skills easier for someone else to evaluate.
Want another small project to practice with?
My Python Weekend Cheat Sheet is available for $7 on Gumroad. Check the product page for what is included before deciding whether it fits your next step.
You can complete everything in this article without buying it.
What happened when you tried the preview — and which case would you check next?

The Python Skill Employers Actually Notice: was originally published in Python in Plain English on Medium, where people are continuing the conversation by highlighting and responding to this story.

The Python Skill Employers Actually Notice:

<div class="medium-feed-item"><a href="https://python.plainenglish.io/the-9-python-tools-top-companies-expect-every-developer-to-know-49c65ec39ee7?source=rss----78073def27b8---4"><img src="https://cdn-images-1.medium.com/max/1024/1*5-W8OjUV4G4zImRa9GioDg.jpeg" width="1024"></a>Nine Python tools top companies expect developers to know in 2026, from uv and Ruff to FastAPI and Polars, with code examples for each.<a href="https://python.plainenglish.io/the-9-python-tools-top-companies-expect-every-developer-to-know-49c65ec39ee7?source=rss----78073def27b8---4">Continue reading on Python in Plain English »</a></div>

Nine Python tools top companies expect developers to know in 2026, from uv and Ruff to FastAPI and Polars, with code examples for each.
Continue reading on Python in Plain English »

The 9 Python Tools Top Companies Expect Every Developer to Know

The small habits that make automation harder than it needs to be
Python is deceptively simple.
One can write
print("Hello")
and see some output right away.
That being said, Python is perhaps one of those languages where one can easily pick up some bad habits that will eventually come back to bite them.
The issue is rarely that one knows too little Python.
More often than not, the issue is that one thinks about Python as a language that they need to learn, not as a tool that they need to master to get their job done.
This becomes especially evident when one reaches the point in one’s learning journey when one is supposed to start writing their own automation scripts.
A script that will go through hundreds of files to rename them, extract some information from them and create reports, send requests to an API, or process the data in any other way is not going to care much for one’s understanding of Python.
It will only care if the code is reliable, readable, and has proper error-handling.
I have seen too many beginners trying to automate a five-minute real-world task that takes an hour to code because their script worked for one example but fails for the rest. Luckily for us, most common mistakes fall under a few common themes.
I present them to you below.
1. Putting Everything in One Place: Scripts
One of the first automation scripts that one would write as a beginner would be similar to the following:
import os
files = os.listdir("documents")
for file in files:
    if file.endswith(".txt"):
        old_path = "documents/" + file
        new_name = file.replace(" ", "_")
        new_path = "documents/" + new_name
        os.rename(old_path, new_path)
        print("Renamed:", file, "->", new_name)
While nothing is wrong with this script (and it may even work!), issues start when one wants to change some part of it.
Say, if tomorrow one wants to

rename files in another folder?
add a prefix?
skip files that already have the correct name?
log errors?
test the renaming logic without actually renaming files?

The code above would have to be modified significantly. A more Pythonic and organized approach would be to split the script into functions and perhaps even classes if one wanted to make their code production-ready.
rename files in a different directory,
add a specific prefix,
exclude already-correct files,
log errors to a log file,
or test the renaming procedure without actually renaming any files
The code above would have to be modified significantly. A more Pythonic and organized approach would be to split the script into functions and perhaps even classes if one wanted to make their code production-ready.
Step 1: Turn repeated logic into a function
import os
def rename_file(folder, filename):
    old_path = os.path.join(folder, filename)
    new_name = filename.replace(" ", "_")
    new_path = os.path.join(folder, new_name)
    os.rename(old_path, new_path)
    return new_name
Now the automation logic is isolated.
You can use it elsewhere:
folder = "documents"
for filename in os.listdir(folder):
    if filename.endswith(".txt"):
        new_name = rename_file(folder, filename)
        print(f"Renamed: {filename} -> {new_name}")
This may look like more code.
But more lines do not automatically mean more complexity.
Sometimes a few extra lines make the entire program easier to understand.
Why this matters for automation
Automation projects almost always grow.
You start with:
Rename these files.
Then:
Rename them and organize them.
Then:
Also generate a CSV report.
Then:
Run the process every morning.
If everything lives inside one giant block, every new feature increases the chance of breaking something else.
Functions give each piece of logic a job.
That makes your automation easier to modify and debug.
2. Hard-Coding Everything
This is another beginner favorite:
import os
files = os.listdir("C:/Users/Azba/Desktop/MyFiles")
for file in files:
    print(file)
It works on your computer.
But only your computer.
The moment you move the script somewhere else, the path becomes useless.
This is one of the biggest differences between a quick script and a reusable automation tool.
Step 1: Store changing values separately
import os
folder = "documents"
files = os.listdir(folder)
for file in files:
    print(file)
Now the folder can be changed without touching the actual automation logic.
But we can improve this further.
Step 2: Use command-line arguments
Python provides a simple way to accept input from the user.
import sys
import os
folder = sys.argv[1]
for file in os.listdir(folder):
    print(file)
You could run the script like this:
python list_files.py documents
Now the same program can work with different folders.
Why this matters
Imagine building an automation script that processes invoices.
If the code contains:
folder = "C:/Users/Azba/Desktop/JanuaryInvoices"
then you’ve built an automation script for January.
Not an invoice automation system.
A better design separates configuration from logic.
The same principle applies to:

API keys
database URLs
file paths
email addresses
output directories
processing limits
model names

Don’t bury values that are expected to change inside your logic.
3. Ignoring Errors Because “It Works”
This mistake becomes painful very quickly.
Consider:
number = int(input("Enter a number: "))
print(100 / number)
Try entering:
10
Everything works.
Try:
0
You get:
ZeroDivisionError
Try:
hello
You get another error.
Beginners often respond by thinking:
I’ll just make sure I enter the correct input.
That’s not automation.
The entire purpose of automation is to reduce the amount of human intervention required.
Step 1: Handle expected errors
try:
    number = int(input("Enter a number: "))
    result = 100 / number
    print(result)
except ValueError:
    print("Please enter a valid number.")
except ZeroDivisionError:
    print("The number cannot be zero.")
Now the program doesn’t immediately crash when the input is invalid.
Step 2: Use error handling where failure is realistic
Suppose your automation processes files:
import os
folder = "documents"
for filename in os.listdir(folder):
    try:
        with open(os.path.join(folder, filename), "r") as file:
            content = file.read()
        print(f"Processed {filename}")
    except Exception as error:
        print(f"Could not process {filename}: {error}")
This is particularly useful in automation because external things fail.
Files can be corrupted.
Permissions can change.
APIs can time out.
A folder can contain an unexpected file.
A database can be temporarily unavailable.
Your program should expect the outside world to be imperfect.
But don’t make this mistake
Beginners sometimes write:
try:
    # everything
except:
    pass
This is worse.
Now errors disappear silently.
If something breaks, you won’t know why.
Error handling should help you understand failure, not hide it.
4. Repeating Code Instead of Automating the Pattern
This is probably the most important mistake when your goal is automation.
Suppose you want to calculate totals for three invoices.
A beginner might write:
invoice1 = 1200
invoice2 = 1500
invoice3 = 900
total = invoice1 + invoice2 + invoice3
print(total)
It works.
But what happens when there are 300 invoices?
You don’t want:
invoice1 = ...
invoice2 = ...
invoice3 = ...
all the way to:
invoice300 = ...
You want to identify the pattern.
Step 1: Put related values into a collection
invoices = [1200, 1500, 900]
total = sum(invoices)
print(total)
Now adding another invoice is easy:
invoices = [1200, 1500, 900, 2100]
total = sum(invoices)
print(total)
But automation becomes more interesting when the data comes from somewhere else.
Step 2: Read the data from a CSV file
Imagine invoices.csv contains:
customer,amount
Ali,1200
Sara,1500
Hamza,900
You can process it automatically:
import csv
total = 0
with open("invoices.csv", newline="") as file:
    reader = csv.DictReader(file)
    for row in reader:
        total += float(row["amount"])
print(f"Total invoices: {total}")
Now the number of invoices doesn’t matter.
Ten rows?
It works.
Ten thousand rows?
The same basic logic can process them.
This is the mindset shift automation requires:
Don’t automate individual actions. Automate the pattern behind the actions.
If you’re repeatedly doing the same thing manually, don’t immediately ask:
How do I make Python click this button?
Ask:
What underlying pattern am I repeating?
That question usually leads to a better solution.
5. Using the Wrong Data Structure
Beginners often use lists for everything.
For example:
student = ["Azba", 20, "Computer Science", 3.3]
Technically, this works.
But what does this mean?
student[2]
You have to remember that index 2 represents the degree.
That becomes increasingly difficult as the program grows.
A dictionary is often more appropriate:
student = {
    "name": "Azba",
    "age": 20,
    "degree": "Computer Science",
    "gpa": 3.3
}
Now:
print(student["degree"])
is immediately understandable.
Why this matters in automation
Automation often deals with structured information.
For example, an API might return:
user = {
    "id": 42,
    "name": "Ali",
    "email": "ali@example.com",
    "active": True
}
You can access exactly what you need:
email = user["email"]
print(email)
You could also store multiple users:
users = [
    {
        "name": "Ali",
        "email": "ali@example.com"
    },
    {
        "name": "Sara",
        "email": "sara@example.com"
    }
]for user in users:
    print(user["email"])
This becomes useful when automating tasks such as:

generating reports
processing API responses
cleaning datasets
creating personalized emails
organizing records
extracting information from documents

Choosing the right data structure isn’t about writing sophisticated code.
It’s about representing the problem correctly.
6. Forgetting to Separate Input, Processing, and Output
A surprisingly useful mental model for automation is:
Input → Processing → Output
For example, imagine you’re building a script that cleans filenames.
The input is:
My Report.pdf
Final Report.pdf
Project Notes.pdf
The processing is:
replace spaces with underscores
The output becomes:
My_Report.pdf
Final_Report.pdf
Project_Notes.pdf
Beginners often mix all three stages together.
Instead, separate them.
Step 1: Create the processing function
def clean_filename(filename):
    return filename.replace(" ", "_")
Step 2: Process the input
files = [
    "My Report.pdf",
    "Final Report.pdf",
    "Project Notes.pdf"
]cleaned_files = []
for filename in files:
    cleaned = clean_filename(filename)
    cleaned_files.append(cleaned)
Step 3: Produce the output
for filename in cleaned_files:
    print(filename)
The complete program becomes:
def clean_filename(filename):
    return filename.replace(" ", "_")
files = [
    "My Report.pdf",
    "Final Report.pdf",
    "Project Notes.pdf"
]cleaned_files = []
for filename in files:
    cleaned_files.append(clean_filename(filename))
for filename in cleaned_files:
    print(filename)
This separation becomes extremely valuable when automation projects become larger.
Suppose later you want to change the output.
Instead of changing the entire program, you can modify only the output stage.
Or perhaps the input changes from a Python list to a CSV file.
The processing logic can remain the same.
That is a much more maintainable design.
7. Building Automation Without Measuring Whether It Actually Helps
This is the mistake people rarely talk about.
You can write a technically impressive automation script that saves almost no time.
Imagine a task takes you two minutes manually.
You spend six hours building an automation script for it.
If you perform the task once a month, was the automation worth it?
Probably not.
The important question isn’t:
Can Python automate this?
Python can automate a surprisingly large number of tasks.
The better question is:
Is automating this task worth the engineering cost?
Step 1: Measure the manual process
Suppose you manually rename 200 files.
It takes:
25 minutes
You do this every week.
That’s roughly:
25 × 52 = 1300 minutes
or about:
21.7 hours per year
Now automation becomes interesting.
If your script takes three hours to build and saves roughly 22 hours every year, the economics are straightforward.
Step 2: Measure the automated process
You could time your Python script:
import time
start = time.time()
# automation code goes here
end = time.time()
print(f"Execution time: {end - start:.2f} seconds")
For more accurate performance measurement, Python also provides time.perf_counter():
import time
start = time.perf_counter()
# automation code goes here
end = time.perf_counter()
print(f"Execution time: {end - start:.4f} seconds")
Now you’re not just saying:
My script is faster.
You have a measurement.
The bigger lesson
Automation is not about making everything run through Python.
It is about converting repetitive human effort into reliable computational work.
That distinction matters.
A Better Beginner Workflow for Python Automation
If you’re learning Python specifically because you want to build useful automation projects, don’t start by memorizing every Python feature.
Start with a problem.
Then break it down.
Step 1: Describe the manual task
For example:
Every Friday, I download 50 CSV files and combine them into one report.
Step 2: Identify the input
50 CSV files
Step 3: Identify the transformation
Read each CSV
Extract relevant rows
Combine the data
Step 4: Identify the output
final_report.csv
Step 5: Build the smallest working version
import csv
import glob
files = glob.glob("data/*.csv")
rows = []
for filename in files:
    with open(filename, newline="") as file:
        reader = csv.DictReader(file)
        for row in reader:
            rows.append(row)
print(f"Collected {len(rows)} rows")
Don’t immediately build a graphical interface.
Don’t add a database.
Don’t add AI.
Don’t deploy it to the cloud.
First make the basic workflow work.
Then improve it.
What These Mistakes Have in Common
At first glance, these seven mistakes seem unrelated.
They aren’t.
They all come from treating programming as syntax rather than problem solving.
The beginner asks:
What Python feature should I use?
A better question is:
What exactly am I trying to accomplish?
That difference changes how you write code.
If you’re building an automation system, think in terms of:
Problem
   ↓
Input
   ↓
Processing
   ↓
Output
   ↓
Error handling
   ↓
Measurement
Once you understand that pipeline, Python becomes the tool used to implement it.
Not the starting point.
The Next Level: Turn Scripts Into Systems
A small automation script might look like this:
files = get_files()
for file in files:
    process(file)
A production-oriented system eventually needs more:
def main():
    files = get_files()
for file in files:
        try:
            process(file)
        except Exception as error:
            log_error(file, error)
if __name__ == "__main__":
    main()
The difference isn’t that the second version is “more advanced Python.”
It is that it starts accounting for reality.
Files can fail.
Inputs can change.
Processes can be repeated.
Errors need to be recorded.
The program may need to run without you watching it.
That is where automation starts becoming engineering.
Final Takeaway
The fastest way to become better at Python automation isn’t to collect more syntax.
Build more useful things.
Pick a repetitive problem.
Measure how much time it costs.
Identify the input, transformation, and output.
Build the smallest version that works.
Then make it reusable.
And when something breaks, don’t immediately patch the error.
Ask why your program wasn’t prepared for that situation.
The difference between a beginner Python script and a useful automation tool is rarely the number of Python features involved.
It is the quality of the thinking behind the code.
So the next time you catch yourself asking:
“What can I build with Python?”
Try changing the question to:
“What problem am I repeatedly wasting time on?”
That question will usually lead you somewhere much more interesting.

7 Python Mistakes Beginners Make Without Realizing was originally published in Python in Plain English on Medium, where people are continuing the conversation by highlighting and responding to this story.

7 Python Mistakes Beginners Make Without Realizing

<div class="medium-feed-item"><a href="https://python.plainenglish.io/genome-toolkit-part-4-6-building-structured-scientific-results-51a309b61e17?source=rss----78073def27b8---4"><img src="https://cdn-images-1.medium.com/max/2560/1*Z1sp0n66hKCtLhf8bzWuRg.jpeg" width="2560"></a>Welcome back to the Genome Toolkit series!<a href="https://python.plainenglish.io/genome-toolkit-part-4-6-building-structured-scientific-results-51a309b61e17?source=rss----78073def27b8---4">Continue reading on Python in Plain English »</a></div>

Welcome back to the Genome Toolkit series!
Continue reading on Python in Plain English »

Genome Toolkit. Part 4.6: Building Structured Scientific Results

<div class="medium-feed-item"><a href="https://python.plainenglish.io/9-python-libraries-every-developer-should-master-before-2027-d1bc16afff86?source=rss----78073def27b8---4"><img src="https://cdn-images-1.medium.com/max/1672/1*MWVxCq_8L1h2m7n_m38z1w@2x.jpeg" width="1672"></a>Subtitle: Python has no shortage of libraries. That is the problem.<a href="https://python.plainenglish.io/9-python-libraries-every-developer-should-master-before-2027-d1bc16afff86?source=rss----78073def27b8---4">Continue reading on Python in Plain English »</a></div>

Subtitle: Python has no shortage of libraries. That is the problem.
Continue reading on Python in Plain English »

Hello, Python

Python Jobs 💼